diskimage-builder

History

Ian Wienand 672705831f Add a best-effort sudo safety check As motivation for this; we have had two breakouts of dib in recent memory. One was a failure to unmount through symlinks in the core code (I335316019ef948758392b03e91f9869102a472b9) and the other was removing host keys on the build-system (Ib01d71ff9415a0ae04d963f6e380aab9ac2260ce). For the most part, dib runs unprivileged. Bits of the core code are hopefully well tested (modulo bugs like the first one!). We give free reign inside the chroot (although there is still some potential there for adverse external affects via bind mounts). Where we could be a bit safer (and could have prevented at least the second of these breakouts) is with some better checking that the "sudo" calls outside the chroot at least looked sane. This adds a basic check that we're using chroot or image paths when calling sudo in those parts of elements that run outside the chroot. Various files are updated to accomodate this check; mostly by just ignoring it for existing code (I have not audited these calls). Nobody is pretending this type of checking makes dib magically safe, or removes the issues with it needing to do things as root during the build. But this can help find egregious errors like the key removal. Change-Id: I161a5aea1d29dcdc7236f70d372c53246ec73749		2016-05-09 15:41:38 +10:00
..
extra-data.d	Add a best-effort sudo safety check	2016-05-09 15:41:38 +10:00
post-install.d	Standarise tracing for scripts	2015-02-12 10:41:32 +11:00
pre-install.d	Allow setting DIB_PIP_RETRIES	2015-02-19 15:29:36 -08:00
README.rst	Allow setting DIB_PIP_RETRIES	2015-02-19 15:29:36 -08:00

README.rst

====
pypi
====
Inject a PyPI mirror
====================

Use a custom PyPI mirror to build images. The default is to bind mount one from
~/.cache/image-create/pypi/mirror into the build environment as mirror URL
file:///tmp/pypi. The element temporarily overwrites /root/.pip.conf
and .pydistutils.cfg to use it.

When online, the official pypi.python.org pypi index is supplied as an
extra-url, so uncached dependencies will still be available. When offline, only
the mirror is used - be warned that a stale mirror will cause build failures.
To disable the pypi.python.org index without using --offline (e.g. when working
behind a corporate firewall that prohibits pypi.python.org) set
DIB\_NO\_PYPI\_PIP to any non-empty value.

To use an arbitrary mirror set DIB\_PYPI\_MIRROR\_URL=http[s]://somevalue/

Additional mirrors can be added by exporting DIB\_PYPI\_MIRROR\_URL\_1=... etc.
Only the one mirror can be used by easy-install, but since wheels need to be in
the first mirror to be used, the last listed mirror is used as the pydistutils
index. NB: The sort order for these variables is a simple string sort - if you
have more than 9 additional mirrors, some care will be needed.

You can also set the number of retries that occur on failure by setting the
DIB\_PIP\_RETRIES environment variable. If setting fallback pip mirrors you
typically want to set this to 0 to prevent the need to fail multiple times
before falling back.

A typical use of this element is thus:
export DIB\_PYPI\_MIRROR\_URL=http://site/pypi/Ubuntu-13.10
export DIB\_PYPI\_MIRROR\_URL\_1=http://site/pypi/
export DIB\_PYPI\_MIRROR\_URL\_2=file:///tmp/pypi
export DIB\_PIP\_RETRIES=0

[devpi-server](https://git.openstack.org/cgit/openstack-infra/pypi-mirro://pypi.python.org/pypi/devpi-server)
can be useful in making a partial PyPI mirror suitable for building images. For
instance:

 * pip install -U devpi

 * devpi-server quickstart

 * devpi use http://machinename:3141

* Re-export your variables to point at the new mirror:

    export DIB\_PYPI\_MIRROR\_URL=http://machinename:3141/
    unset DIB\_PYPI\__MIRROR\_URL\_1
    unset DIB\_PYPI\__MIRROR\_URL\_2

The next time packages are installed, they'll be cached on the local devpi
server; subsequent runs pointed at the same mirror will use the local cache if
the upstream can't be contacted.

Note that this process only has the server running temporarily; see
[Quickstart: Permanent install on
server/laptop](http://doc.devpi.net/latest/quickstart-server.html) guide from
the devpi developers for more information on a more permanent setup.