sig_cloud/diskimage-builder
11
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Increase source-repositories support for tarballs

Browse Source 
The existing support for tar files in source-repositories saves
only the contents within subdirectories contained in the archive.

This change makes the selection of contents within the tarball user-
definable based on the REPOREF specified (or overridden in the env).

A REPOREF of '*' (the current default, but deprecated) will select
the contents of subdirectories within the archive, while '.' will
select the entire contents of the archive.

For reference:
http://lists.openstack.org/pipermail/openstack-dev/2014-August/043197.html
http://lists.openstack.org/pipermail/openstack-dev/2014-August/043249.html

Change-Id: I45db42ce66bf1d63d6ab5730090bf458b1b37ce9
This commit is contained in:
Jonathan Brownell 2014-07-03 14:20:08 -07:00
parent 6f7af6f5a2
commit 6e4e2cfeca
2 changed files with 32 additions and 8 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

11
elements/source-repositories/README.md
View File

@ -11,7 +11,12 @@ from git and pbr from a tarball would be
*File : elements/custom-element/source-repository-pbr*
pbr tar /usr/local/pbr http://tarballs.openstack.org/pbr/pbr-master.tar.gz
# <ref> is defined as "*" by default, although this behavior is deprecated.
# A value of "." extracts the entire contents of the tarball.
# A value of "*" extracts the contents within all its subdirectories.
# A value of a subdirectory path may be used to extract only its contents.
# A value of a specific file path within the archive is not supported.
pbr tar /usr/local/pbr http://tarballs.openstack.org/pbr/pbr-master.tar.gz .
diskimage-builder will then retrieve the sources specified and place them
at the directory \<destination\>
@ -49,9 +54,7 @@ may contain environment variables.
Git sources will be cloned to \<destination\>
Tarballs will be extracted to \<destination\>. Tarballs should contain a
single topleval directory, regardless of the name of this top level directory
it will be renamed to \<destination\>
Tarballs will be extracted to \<destination\>.
The package type indicates the element should install from packages onto the
root filesystem of the image build during the install.d phase.

29
elements/source-repositories/extra-data.d/98-source-repositories
View File

@ -29,9 +29,15 @@ function get_repos_for_element(){
local REGEX="^([^ ]+) (git|tar|file|package) ?(/[^ ]+)? ?([^ ]+)? ?([^ ]*)$"
while read line; do
# temporarily turn off globbing '*' (since it may be used as the REPOREF for tarballs)
set -f
# expand variables
line=$(eval echo $line)
# restore globbing
set +f
# ignore blank lines and lines beginning in '#'
[[ "$line" == \#* ]] || [[ -z "$line" ]] && continue
@ -41,7 +47,17 @@ function get_repos_for_element(){
local REPOPATH=${BASH_REMATCH[3]}
local REPOLOCATION=${BASH_REMATCH[4]}
local REPO_ORIG_LOCATION=$REPOLOCATION
local REPOREF=${BASH_REMATCH[5]:-master}
if [ $REPONAME = "tar" -a -z "${BASH_REMATCH[5]:-}" ] ; then
echo "Warning: Default tarball REPOREF of '*' is deprecated; do not rely on it."
fi
# Default of '*' for tar repositories is deprecated; do not rely on it.
local REPOREF_DEFAULT_TAR=*
local REPOREF_DEFAULT_GIT=master
local REPOREF_LOOKUP_DEFAULT=REPOREF_DEFAULT_${REPOTYPE^^}
local REPOREF=${BASH_REMATCH[5]:-${!REPOREF_LOOKUP_DEFAULT:-}}
local REPO_DEST=$TMP_MOUNT_PATH$REPOPATH
local REPO_SUB_DIRECTORY=$(dirname $REPO_DEST)
@ -129,8 +145,7 @@ function get_repos_for_element(){
tar)
# The top level directory of the tarball mightn't have a fixed name i.e.
# it could contain version numbers etc... so we write it to a tmpdir
# the then move the contents into the directory we want it in, this does
# assume the tarball only contains a single top level directory
# for inspection before transferring the contents into the target directory
local tmpdir=$(mktemp --tmpdir=$TMP_MOUNT_PATH/tmp -d)
if [ -n "$CACHE_PATH" ] ; then
echo "Caching $REPONAME tarball from $REPOLOCATION in $CACHE_PATH"
@ -142,9 +157,15 @@ function get_repos_for_element(){
echo "Fetching $REPONAME tarball from $REPOLOCATION"
curl $REPOLOCATION | tar -C $tmpdir -xzf -
fi
sudo mkdir -p $REPO_DEST
sudo mv $tmpdir/*/* $REPO_DEST
# A REPOREF of '.' will select the entire contents of the tarball,
# while '*' will select only the contents of its subdirectories.
sudo rsync -a --remove-source-files $tmpdir/$REPOREF/. $REPO_DEST
rm -rf $tmpdir
;;
file)
sudo mkdir -p $REPO_SUB_DIRECTORY