diskimage-builder/elements/pypi/extra-data.d/00-mount-pypi-mirror

#!/bin/bash

if [ ${DIB_DEBUG_TRACE:-0} -gt 0 ]; then
    set -x
fi
set -eu
set -o pipefail

MIRROR_SOURCE=$DIB_IMAGE_CACHE/pypi/mirror/
if [ -d "$MIRROR_SOURCE" ]; then
    MIRROR_TARGET=$TMP_MOUNT_PATH/tmp/pypi

    sudo mkdir -p $MIRROR_SOURCE $MIRROR_TARGET
    sudo mount --bind $MIRROR_SOURCE $MIRROR_TARGET
fi
Allow using a pypi mirror to install via pip. Using a custom pypi mirror can be very convenient, making image builds substantially faster - because we create multiple virtual environments we benefit more than single-virtualenv users would. Change-Id: I997daf1f9477c447e1fb30818aea9e80a49b31a6 2013-08-09 13:13:15 +00:00			`#!/bin/bash`

Standarise tracing for scripts There is a wide variety of tracing options through the various shell scripts. Some use "set -eux", others explicity set xtrace and others do nothing. There is a "-x" option to bin/disk-image-create but it doesn't flow down to the many scripts it calls. This adds a global integer variable set by disk-image-create DIB_DEBUG_TRACE. All scripts have a stanza added to detect this and turn on tracing. Any other tracing methods are rolled into this. So the standard header is --- if [ "${DIB_DEBUG_TRACE:-0}" -gt 0 ]; then set -x fi set -eu set -o pipefail --- Multiple -x options can be specified to dib-create-image, which increases the value of DIB_DEBUG_TRACE. If script authors feel their script should only trace at higher levels, they should modify the "-gt" value. If they feel it should trace by default, they can modify the default value also. Changes to pachset 16 : scripts which currently trace themselves by default have retained this behaviour with DIB_DEBUG_TRACE defaulting to "1". This was done by running [1] on patch set 15. See the thread beginning at [2] dib-lint is also updated to look for the variable being matched. [1] https://gist.github.com/ianw/71bbda9e6acc74ccd0fd [2] http://lists.openstack.org/pipermail/openstack-dev/2014-November/051575.html Change-Id: I6c5a962260741dcf6f89da9a33b96372a719b7b0 2014-09-04 04:56:29 +00:00			`if [ ${DIB_DEBUG_TRACE:-0} -gt 0 ]; then`
			`set -x`
			`fi`
Allow using a pypi mirror to install via pip. Using a custom pypi mirror can be very convenient, making image builds substantially faster - because we create multiple virtual environments we benefit more than single-virtualenv users would. Change-Id: I997daf1f9477c447e1fb30818aea9e80a49b31a6 2013-08-09 13:13:15 +00:00			`set -eu`
set -u and -o pipefail everywhere As with the previous similar changes, this is intended to catch problems as they happen instead of ignoring them and continuing on to potentially fail later. Setting this on all existing scripts will allow us to enforce use via Jenkins. Change-Id: Iad2d490c86dceab148ea9ab08f457c49a5d5352e 2014-04-03 02:24:15 +00:00			`set -o pipefail`
Allow using a pypi mirror to install via pip. Using a custom pypi mirror can be very convenient, making image builds substantially faster - because we create multiple virtual environments we benefit more than single-virtualenv users would. Change-Id: I997daf1f9477c447e1fb30818aea9e80a49b31a6 2013-08-09 13:13:15 +00:00
Use DIB_IMAGE_CACHE everywhere disk-image-create processes a DIB_IMAGE_CACHE variable and exports it, but there are several elements that ignore the value and wrote out the base location themselves. Use the variable everywhere so that it will get overridden everywhere. Change-Id: I00fff354d6c931ad67cf3052d055f0e4604dfdc8 2014-10-05 20:55:34 +00:00			`MIRROR_SOURCE=$DIB_IMAGE_CACHE/pypi/mirror/`
Always bind-mount pypi mirror if dir exists It may be desirable to specify file://tmp/pypi as one of the list of mirrors specified by PYPI_MIRROR_URL, so the local pypi mirror should be mounted as long as the directory exists. Even though the path is mounted, it is only used by pip if there is no specified PYPI_MIRROR_URL, or if one PYPI_MIRROR_URL entry specifies file://tmp/pypi. The pypi element previously assumed that a local mirror would be a complete offline mirror. This change allows for a mixture of local mirror plus remote mirrors. This means that the local mirror could (for example) contain a few locally built pip packages from arbitrary git checkouts for gate testing of those projects. In this scenario, most packages would be downloaded from the configured remote mirrors. This change is required for synchronous gating on the os-*-config tools. Change-Id: I0f6d153c06993ffd0d90bd8ca1717462e8b6b541 2014-05-26 02:20:50 +00:00			`if [ -d "$MIRROR_SOURCE" ]; then`
			`MIRROR_TARGET=$TMP_MOUNT_PATH/tmp/pypi`
Allow using a pypi mirror to install via pip. Using a custom pypi mirror can be very convenient, making image builds substantially faster - because we create multiple virtual environments we benefit more than single-virtualenv users would. Change-Id: I997daf1f9477c447e1fb30818aea9e80a49b31a6 2013-08-09 13:13:15 +00:00
Always bind-mount pypi mirror if dir exists It may be desirable to specify file://tmp/pypi as one of the list of mirrors specified by PYPI_MIRROR_URL, so the local pypi mirror should be mounted as long as the directory exists. Even though the path is mounted, it is only used by pip if there is no specified PYPI_MIRROR_URL, or if one PYPI_MIRROR_URL entry specifies file://tmp/pypi. The pypi element previously assumed that a local mirror would be a complete offline mirror. This change allows for a mixture of local mirror plus remote mirrors. This means that the local mirror could (for example) contain a few locally built pip packages from arbitrary git checkouts for gate testing of those projects. In this scenario, most packages would be downloaded from the configured remote mirrors. This change is required for synchronous gating on the os-*-config tools. Change-Id: I0f6d153c06993ffd0d90bd8ca1717462e8b6b541 2014-05-26 02:20:50 +00:00			`sudo mkdir -p $MIRROR_SOURCE $MIRROR_TARGET`
			`sudo mount --bind $MIRROR_SOURCE $MIRROR_TARGET`
			`fi`