sig_cloud/diskimage-builder
2
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
ee18fe41da
diskimage-builder/elements/pypi
History
Robert Collins ee18fe41da Permit use of wheel mirrors in pypi element. 
pypi-mirror creates a separate mirror index for wheels (one per OS
that mirrors are built on). To be able to use it one then needs to be
able to export multiple mirrors for inclusion in pip.conf. As a drive
by I made it possible to disable the use of the pypi.python.org index
without using --offline (as --offline has larger impact).

Change-Id: I3e85a8069b18cafd7eae4cd0591821acc3b5a739
2014-03-25 16:48:18 +13:00
..
extra-data.d Permit using arbitrary PyPI urls. 2014-02-19 14:05:31 +13:00
post-install.d Allow using a pypi mirror to install via pip. 2013-08-10 22:07:30 +12:00
pre-install.d Permit use of wheel mirrors in pypi element. 2014-03-25 16:48:18 +13:00
README.md Permit use of wheel mirrors in pypi element. 2014-03-25 16:48:18 +13:00

README.md

Inject a PyPI mirror

Use a custom PyPI mirror to build images. The default is to bind mount one from ~/.cache/image-create/pypi/mirror into the build environment. The element temporarily overwrites /root/.pip.conf and .pydistutils.cfg to use it.

When online, the official pypi.python.org pypi index is supplied as an extra-url, so uncached dependencies will still be available. When offline, only the mirror is used - be warned that a stale mirror will cause build failures. To disable the pypi.python.org index without using --offline (e.g. when working behind a corporate firewall that prohibits pypi.python.org) set DIB_NO_PYPI_PIP to any non-empty value.

To use an arbitrary mirror set PYPI_MIRROR_URL=http[s]://somevalue/

Additional mirrors can be added by exporting PYPI_MIRROR_URL_1=... etc. Only the one mirror can be used by easy-install, but since wheels need to be in the first mirror to be used, the last listed mirror is used as the pydistutils index. NB: The sort order for these variables is a simple string sort - if you have more than 9 additional mirrors, some care will be needed.

A typical use of this element is thus: export PYPI_MIRROR_URL=http://site/pypi/Ubuntu-13.10 export PYPI_MIRROR_URL_1=http://site/pypi/

pypi-mirror can be useful in making a partial PyPI mirror suitable for building images. For instance:

  • sudo apt-get install libxml2-dev libxslt-dev libmysqlclient-dev libpq-dev
    libnspr4-dev pkg-config libsqlite3-dev libzmq-dev libffi-dev libldap2-dev
    libsasl2-dev

  • pip install git+https://git.openstack.org/openstack-infra/pypi-mirror

  • cat << EOF > mirror.yaml cache-root: /home/USER/.cache/image-create/pypi/download

    mirrors:

  • mkdir -p /home/USER/.cache/image-create/pypi/{download,mirror}

  • run-mirror -b remotes/origin/master --verbose -c mirror.yaml

    This creates and updates the mirror.

If you have additional packages that are not identified in the global openstack requirements project, you can include them:

  • pip install -d ~/.cache/image-create/pypi/download/pip/openstack
    heat-cfntools distribute os-apply-config run-mirror -b remotes/origin/master --verbose -c mirror.yaml --no-download