sig_cloud/diskimage-builder
11
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Increase source-repositories support for tarballs

Browse Source 
The existing support for tar files in source-repositories saves
only the contents within subdirectories contained in the archive.

This change makes the selection of contents within the tarball user-
definable based on the REPOREF specified (or overridden in the env).

A REPOREF of '*' (the current default, but deprecated) will select
the contents of subdirectories within the archive, while '.' will
select the entire contents of the archive.

For reference:
http://lists.openstack.org/pipermail/openstack-dev/2014-August/043197.html
http://lists.openstack.org/pipermail/openstack-dev/2014-August/043249.html

Change-Id: I45db42ce66bf1d63d6ab5730090bf458b1b37ce9
This commit is contained in:
Jonathan Brownell 2014-07-03 14:20:08 -07:00
parent 6f7af6f5a2
commit 6e4e2cfeca
2 changed files with 32 additions and 8 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

11
elements/source-repositories/README.md
View File

@ -11,7 +11,12 @@ from git and pbr from a tarball would be
*File : elements/custom-element/source-repository-pbr* *File : elements/custom-element/source-repository-pbr*
pbr tar /usr/local/pbr http://tarballs.openstack.org/pbr/pbr-master.tar.gz # <ref> is defined as "*" by default, although this behavior is deprecated.
# A value of "." extracts the entire contents of the tarball.
# A value of "*" extracts the contents within all its subdirectories.
# A value of a subdirectory path may be used to extract only its contents.
# A value of a specific file path within the archive is not supported.
pbr tar /usr/local/pbr http://tarballs.openstack.org/pbr/pbr-master.tar.gz .
diskimage-builder will then retrieve the sources specified and place them diskimage-builder will then retrieve the sources specified and place them
at the directory \<destination\> at the directory \<destination\>
@ -49,9 +54,7 @@ may contain environment variables.
Git sources will be cloned to \<destination\> Git sources will be cloned to \<destination\>
Tarballs will be extracted to \<destination\>. Tarballs should contain a Tarballs will be extracted to \<destination\>.
single topleval directory, regardless of the name of this top level directory
it will be renamed to \<destination\>
The package type indicates the element should install from packages onto the The package type indicates the element should install from packages onto the
root filesystem of the image build during the install.d phase. root filesystem of the image build during the install.d phase.

29
elements/source-repositories/extra-data.d/98-source-repositories
View File

@ -29,9 +29,15 @@ function get_repos_for_element(){
local REGEX="^([^ ]+) (git|tar|file|package) ?(/[^ ]+)? ?([^ ]+)? ?([^ ]*)$" local REGEX="^([^ ]+) (git|tar|file|package) ?(/[^ ]+)? ?([^ ]+)? ?([^ ]*)$"
while read line; do while read line; do
# temporarily turn off globbing '*' (since it may be used as the REPOREF for tarballs)
set -f
# expand variables # expand variables
line=$(eval echo $line) line=$(eval echo $line)
# restore globbing
set +f
# ignore blank lines and lines beginning in '#' # ignore blank lines and lines beginning in '#'
[[ "$line" == \#* ]] || [[ -z "$line" ]] && continue [[ "$line" == \#* ]] || [[ -z "$line" ]] && continue
@ -41,7 +47,17 @@ function get_repos_for_element(){
local REPOPATH=${BASH_REMATCH[3]} local REPOPATH=${BASH_REMATCH[3]}
local REPOLOCATION=${BASH_REMATCH[4]} local REPOLOCATION=${BASH_REMATCH[4]}
local REPO_ORIG_LOCATION=$REPOLOCATION local REPO_ORIG_LOCATION=$REPOLOCATION
local REPOREF=${BASH_REMATCH[5]:-master}
if [ $REPONAME = "tar" -a -z "${BASH_REMATCH[5]:-}" ] ; then
echo "Warning: Default tarball REPOREF of '*' is deprecated; do not rely on it."
fi
# Default of '*' for tar repositories is deprecated; do not rely on it.
local REPOREF_DEFAULT_TAR=*
local REPOREF_DEFAULT_GIT=master
local REPOREF_LOOKUP_DEFAULT=REPOREF_DEFAULT_${REPOTYPE^^}
local REPOREF=${BASH_REMATCH[5]:-${!REPOREF_LOOKUP_DEFAULT:-}}
local REPO_DEST=$TMP_MOUNT_PATH$REPOPATH local REPO_DEST=$TMP_MOUNT_PATH$REPOPATH
local REPO_SUB_DIRECTORY=$(dirname $REPO_DEST) local REPO_SUB_DIRECTORY=$(dirname $REPO_DEST)
@ -129,8 +145,7 @@ function get_repos_for_element(){
tar) tar)
# The top level directory of the tarball mightn't have a fixed name i.e. # The top level directory of the tarball mightn't have a fixed name i.e.
# it could contain version numbers etc... so we write it to a tmpdir # it could contain version numbers etc... so we write it to a tmpdir
# the then move the contents into the directory we want it in, this does # for inspection before transferring the contents into the target directory
# assume the tarball only contains a single top level directory
local tmpdir=$(mktemp --tmpdir=$TMP_MOUNT_PATH/tmp -d) local tmpdir=$(mktemp --tmpdir=$TMP_MOUNT_PATH/tmp -d)
if [ -n "$CACHE_PATH" ] ; then if [ -n "$CACHE_PATH" ] ; then
echo "Caching $REPONAME tarball from $REPOLOCATION in $CACHE_PATH" echo "Caching $REPONAME tarball from $REPOLOCATION in $CACHE_PATH"
@ -142,9 +157,15 @@ function get_repos_for_element(){
echo "Fetching $REPONAME tarball from $REPOLOCATION" echo "Fetching $REPONAME tarball from $REPOLOCATION"
curl $REPOLOCATION | tar -C $tmpdir -xzf - curl $REPOLOCATION | tar -C $tmpdir -xzf -
fi fi
sudo mkdir -p $REPO_DEST sudo mkdir -p $REPO_DEST
sudo mv $tmpdir/*/* $REPO_DEST
# A REPOREF of '.' will select the entire contents of the tarball,
# while '*' will select only the contents of its subdirectories.
sudo rsync -a --remove-source-files $tmpdir/$REPOREF/. $REPO_DEST
rm -rf $tmpdir rm -rf $tmpdir
;; ;;
file) file)
sudo mkdir -p $REPO_SUB_DIRECTORY sudo mkdir -p $REPO_SUB_DIRECTORY