diskimage-builder/diskimage_builder/elements/pip-and-virtualenv/README.rst
Ian Wienand 18215274d8 pip-and-virtualenv : deprecate source for CentOS 8, new variables
As described inline, deprecate the "source" install for CentOS 8.
Overwriting the packaged tools has long been a pain-point in our
images, and the best outcome is just not to play the game [1].

However, the landscape remains complicated.  For example, RHEL/CentOS
8 introduces the separate "platform-python" binary, which seems like
the right tool to install platform tools like "glean" (simple-init)
with.  However, platform-python doesn't have virtualenv (only the
inbuilt venv).

So that every element doesn't have to hard-code in workarounds for
these various layouts, create two new variables DIB_PYTHON_PIP and
DIB_PYTHON_VIRTUALENV to just "do the right thing".  If you need is
"install a pip package" or "create a virtualenv" this should work on
all the platforms we support.  If you know more specifically what you
want (e.g. must be a python3 virtualenv) then nothing stops elements
calling that directly (e.g. python3 -m virtualenv create); these are
just helper wrappers for base elements that need to be broadly
compatible.

[1] http://lists.openstack.org/pipermail/openstack-infra/2019-September/006483.html

Change-Id: Ia267a60eecfa8f4071dd477d86daebe07e9a7e38
2019-10-03 00:22:18 +00:00

112 lines
4.1 KiB
ReStructuredText

==================
pip-and-virtualenv
==================
This element installs pip and virtualenv in the image.
Package install
===============
If the package installtype is used then these programs are installed
from distribution packages. In this case, ``pip`` and ``virtualenv``
will be installed *only* for the python version identified by
``dib-python`` (i.e. the default python for the platform).
Namespacing of the tools will be up to your distribution. Some
distribution packages have worked out name-spacing such that only
python2 or python3 owns common scripts like ``/usr/bin/pip`` (on most
platforms, ``pip`` refers to python2 pip, and ``pip3`` refers to
python3 pip, although some may choose the reverse). Other platforms
have avoided making a decision and require explicit version suffixes.
To install pip and virtualenv from package::
export DIB_INSTALLTYPE_pip_and_virtualenv=package
Source install
==============
.. note:: For source installs this element setups and Python 2 and
Python 3 environments. This means it will bring in python2
packages, so isn't appropriate if you want a python3 only
environment.
.. note:: Source install is considered deprecated for several reasons.
Because it makes for a hetrogenous environment between
distro packaged tools and upstream it means the final images
create bespoke environments that make standarised testing
difficult. The tricks used around holding packages to
overwrite them cause difficulty for users of images. This
also brings in Python 2 unconditonally, something not wanted
on modern Python 3 only distributions.
Source install is the default on most platforms for historical
purposes. The current exception(s) are RHEL8 and CentOS 8.
If the source installtype is used, ``pip`` and ``virtualenv`` are
installed from the latest upstream releases.
Source installs from upstream releases are not name-spaced. It is
inconsistent across platforms if the first or last install will own
common scripts like ``/usr/bin/pip`` and ``virtualenv``.
To avoid inconsistency, we firstly install the packaged python 2
**and** 3 versions of ``pip`` and ``virtualenv``. This prevents a
later install of these distribution packages conflicting with the
source install. We then overwrite ``pip`` and ``virtualenv`` via
``get-pip.py`` and ``pip`` respectively.
The system will be left in the following state:
* ``/usr/bin/pip`` : python2 pip
* ``/usr/bin/pip2`` : python2 pip (same as prior)
* ``/usr/bin/pip3`` : python3 pip
* ``/usr/bin/virtualenv`` : python2 virtualenv
(note python3 ``virtualenv`` script is *not* installed, see below)
Source install is supported on limited platforms. See the code, but
this includes Ubuntu and RedHat platforms.
Environment Variables
=====================
To simplify the common-case of "install a package" or "create a
virtualenv" with the default system Python, the following variables
are exported by this element:
* ``DIB_PYTHON_PIP``
* ``DIB_PYTHON_VIRTUALENV``
This will create/install using the ``dib-python`` version for the
platform (i.e. python2 for older distros, python3 for modern distros).
Note that on Python 3 platforms it will use the inbuilt ``venv``
(rather than the ``virtualenv`` package -- if you absolutely need
features only ``virtualenv`` provides you should call it directly in
your element; see below).
Explicit use of the tools
=========================
Due to the essentially unsolvable problem of "who owns the script", it
is recommended to *not* call ``pip`` or ``virtualenv`` directly. You
can directly call them with the ``-m`` argument to the python
interpreter you wish to install with.
For example, to create a python3 environment do::
# python3 -m virtualenv myenv
# myenv/bin/pip install mytool
To install a python2 tool from pip::
# python2 -m pip install mytool
In this way, you can always know which interpreter is being used (and
affected by) the call.
Ordering
========
Any element that uses these commands must be designated as
05-* or higher to ensure that they are first installed.