sig_cloud/diskimage-builder
2
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Improve source-repositories git caching.

Browse Source 
We now unconditionally cache the repositories being fetched with
source-repositories.

Additionally, by improving the way we handle repository refs, we are now
able to build images with code taken directly from gerrit reviews.

Change-Id: Ifd4c5691f6761eb5551663e6d0aa1c0e42afced3
This commit is contained in:
Chris Jones 2013-12-20 14:35:34 +00:00
parent c1eb44c9b2
commit 4c7a3bf79f
2 changed files with 27 additions and 22 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
elements/source-repositories/README.md
View File

@ -29,6 +29,13 @@ mirror you could set DIB_REPOLOCATION_ironic=git://localgitserver/ironic.git
Alternatively if you would like to use the keystone element and build an image with Alternatively if you would like to use the keystone element and build an image with
keystone from a stable branch then you would set DIB_REPOREF_keystone=stable/grizzly keystone from a stable branch then you would set DIB_REPOREF_keystone=stable/grizzly
If you wish to build an image using code from a gerrit review, you can set
DIB_REPOLOCATION_<name> and DIB_REPOREF_<name> to the values given by gerrit in the
fetch/pull section of a review. For example:
DIB_REPOLOCATION_nova=https://review.openstack.org/openstack/nova
DIB_REPOREF_nova=refs/changes/72/61972/8
Git sources will be cloned to \<destination\> Git sources will be cloned to \<destination\>
Tarballs will be extracted to \<destination\>. Tarballs should contain a Tarballs will be extracted to \<destination\>. Tarballs should contain a

34
elements/source-repositories/extra-data.d/98-source-repositories
View File

@ -35,38 +35,36 @@ function get_repos_for_element(){
# REPOLOCATION can be overridden with DIB_REPOLOCATION_{name} # REPOLOCATION can be overridden with DIB_REPOLOCATION_{name}
local REPOLOCATION_OVERRIDE=DIB_REPOLOCATION_${REPONAME//-/_} local REPOLOCATION_OVERRIDE=DIB_REPOLOCATION_${REPONAME//-/_}
REPOLOCATION=${!REPOLOCATION_OVERRIDE:-$REPOLOCATION} REPOLOCATION=${!REPOLOCATION_OVERRIDE:-$REPOLOCATION}
if [ "$REPOLOCATION" = "$REPO_ORIG_LOCATION" ] ; then
# We're pulling from the element specified repo, do caching for
# the user.
CACHE_NAME=$(echo $REPOLOCATION | sha1sum | awk '{ print $1 }' )
CACHE_PATH=~/.cache/image-create/repository-sources/$CACHE_NAME
else
CACHE_PATH=''
fi
# REPOREF can be overridden with DIB_REPOREF_{name} # REPOREF can be overridden with DIB_REPOREF_{name}
local REPOREF_OVERRIDE=DIB_REPOREF_${REPONAME//-/_} local REPOREF_OVERRIDE=DIB_REPOREF_${REPONAME//-/_}
REPOREF=${!REPOREF_OVERRIDE:-$REPOREF} REPOREF=${!REPOREF_OVERRIDE:-$REPOREF}
# Determine a unique cache path for this repo
CACHE_NAME=$(echo "${REPOTYPE}_${REPOLOCATION}" | sha1sum | awk '{ print $1 }' )
CACHE_PATH=~/.cache/image-create/repository-sources/$CACHE_NAME
case $REPOTYPE in case $REPOTYPE in
git) git)
echo "Fetching $REPONAME git repository from $REPOLOCATION($REPOREF)"
sudo mkdir -p $REPO_SUB_DIRECTORY sudo mkdir -p $REPO_SUB_DIRECTORY
if [ -n "$CACHE_PATH" ] ; then
echo "Caching $REPONAME source repository in $CACHE_PATH"
if [ ! -e "$CACHE_PATH" ] ; then if [ ! -e "$CACHE_PATH" ] ; then
echo "Caching $REPONAME from $REPOLOCATION in $CACHE_PATH"
git clone $REPOLOCATION $CACHE_PATH.tmp git clone $REPOLOCATION $CACHE_PATH.tmp
mv ${CACHE_PATH}{.tmp,} mv ${CACHE_PATH}{.tmp,}
elif [ -z "$DIB_OFFLINE" ] ; then
git --git-dir=$CACHE_PATH/.git fetch
git --git-dir=$CACHE_PATH/.git --work-tree=$CACHE_PATH reset --hard origin/master
fi fi
HAS_REF=$(git --git-dir=$CACHE_PATH/.git show-ref $REPOREF || true)
if [ -z "$DIB_OFFLINE" -o -z "$HAS_REF" ] ; then
echo "Updating cache of $REPOLOCATION in $CACHE_PATH with ref $REPOREF"
git --git-dir=$CACHE_PATH/.git fetch --update-head-ok $REPOLOCATION ${REPOREF}:${REPOREF}
fi
echo "Cloning from $REPONAME cache and applying ref $REPOREF"
sudo git clone $CACHE_PATH $REPO_DEST sudo git clone $CACHE_PATH $REPO_DEST
else
sudo git clone $REPOLOCATION $REPO_DEST
fi
pushd $REPO_DEST pushd $REPO_DEST
sudo git reset --hard $REPOREF sudo git fetch $CACHE_PATH $REPOREF
sudo git reset --hard FETCH_HEAD
popd popd
;; ;;
tar) tar)