Commit Graph

6 Commits

Author SHA1 Message Date
Ian Wienand
18215274d8 pip-and-virtualenv : deprecate source for CentOS 8, new variables
As described inline, deprecate the "source" install for CentOS 8.
Overwriting the packaged tools has long been a pain-point in our
images, and the best outcome is just not to play the game [1].

However, the landscape remains complicated.  For example, RHEL/CentOS
8 introduces the separate "platform-python" binary, which seems like
the right tool to install platform tools like "glean" (simple-init)
with.  However, platform-python doesn't have virtualenv (only the
inbuilt venv).

So that every element doesn't have to hard-code in workarounds for
these various layouts, create two new variables DIB_PYTHON_PIP and
DIB_PYTHON_VIRTUALENV to just "do the right thing".  If you need is
"install a pip package" or "create a virtualenv" this should work on
all the platforms we support.  If you know more specifically what you
want (e.g. must be a python3 virtualenv) then nothing stops elements
calling that directly (e.g. python3 -m virtualenv create); these are
just helper wrappers for base elements that need to be broadly
compatible.

[1] http://lists.openstack.org/pipermail/openstack-infra/2019-September/006483.html

Change-Id: Ia267a60eecfa8f4071dd477d86daebe07e9a7e38
2019-10-03 00:22:18 +00:00
Ian Wienand
e9df83b2b3 Revert "Dont install python-pip for py3k"
This reverts commit ab89c7d69c.

This commit checked for DIB_PYTHON_VERSION and only installed the v3
packages.  This is unfortunately backwards-incompatible, as consumers
such as the openstack gate are relying on this package installing pip
& virtualenv packages for python2 AND python3.

This was sort-of expressed in the docs, where it discusses what the
resulting setup of the system will be, but I've added a note to make
it clearer.

If we want to change this, I think we'll need either a new element, or
a non-defaulting flag.

Change-Id: I419dbdf4682394db68974944af1e5c432f3e0565
2018-01-10 15:48:03 +11:00
Ian Wienand
7a962e9d1c Skip python3-virtualenv on <= trusty
If DIB_PYTHON_VERSION is < 3 on the !redhat path, that means we're on
an older platform that may not have python3-virtualenv packages.  Skip
install.

Ensure the order of operations happens by forcing the installs

Also add a note about limited platform support (patches welcome :)

Change-Id: I18412767f0ebf946d557a0a126285369e96af159
2017-04-12 06:36:20 +10:00
Ian Wienand
79d4113cbe pip-and-virtualenv : install python2 & 3, and default to 2
Recent changes in project-config have shown that we leave the system
in an inconsistent state when installing from source.  On fedora, we
will have installed the python2 packages, but then used $DIB_PYTHON to
install python3 pip from source!

This tries to clarify the situation.  As described in the document,
with package installs, we just install the $DIB_PYTHON packaged
versions.

Source installs want to take over the global namespace.  This is the
price you pay for running the latest versions outside package managers
:) The only sane thing seems to be for us to normalise python2 &
python3 versions of pip, setuptools and virtualenv and then hacking
things such that "/usr/bin/pip" and "/usr/bin/virtalenv" remain
defaulted to python2 versions.

Documentation is added

Change-Id: Ibc6572b89e256d1f48b7fe7c672b8b9524dc704f
2017-04-11 18:59:11 +10:00
Ian Wienand
bfca36c772 Release 1.25.2
-----BEGIN PGP SIGNATURE-----
 
 iQEcBAABAgAGBQJYV1yqAAoJEBty/58O8cX8hLwIAKP66w6MdPN8PDgUOteui/Sx
 N0UFKJ9yR4GQOAP0NffPLjch5/g0iJLs3eFKOhtGC1LjbDjpVgjX8vW18ib8wBZK
 GemOZPF3uxg8FROrZF1vpoDy/cHgL1YV10hCnwdjN/r9rb8zOuSabqjW+Dennj2n
 fZ0SJfa8Owfudn3YxGuOymVb/wMtEloDmVGBEI1Y+h7osELCCDi3OXmwsA8qMsdl
 cTwbeugBs4PlOVbZUK/JKGuwIHKgPnDYzYu5KpXw77/MdjGT0fo5Tlq5AOBDI2sC
 9JOFEBDli4Ro05VwvI58ADMpvvOax+9EvOhLbB1dRPdZl21Iyb6gOdy2PUbFO0c=
 =aKxq
 -----END PGP SIGNATURE-----

Merge tag '1.25.2' into merge-branch

Release 1.25.2

Change-Id: I698bcf2e82117bd81649cd065a7af5cac85990c7
2017-02-02 11:20:00 +11:00
Ian Wienand
97c01e48ed Move elements & lib relative to diskimage_builder package
Currently we have all our elements and library files in a top-level
directory and install them into
<root>/share/diskimage-builder/[elements|lib] (where root is either /
or the root of a virtualenv).

The problem with this is that editable/development installs (pip -e)
do *not* install data_files.  Thus we have no canonical location to
look for elements -- leading to the various odd things we do such as a
whole bunch of guessing at the top of disk-image-create and having a
special test-loader in tests/test_elements.py so we can run python
unit tests on those elements that have it.

data_files is really the wrong thing to use for what are essentially
assets of the program.  data_files install works well for things like
config-files, init.d files or dropping documentation files.

By moving the elements under the diskimage_builder package, we always
know where they are relative to where we import from.  In fact,
pkg_resources has an api for this which we wrap in the new
diskimage_builder/paths.py helper [1].

We use this helper to find the correct path in the couple of places we
need to find the base-elements dir, and for the paths to import the
library shell functions.

Elements such as svc-map and pkg-map include python unit-tests, which
we do not need tests/test_elements.py to special-case load any more.
They just get found automatically by the normal subunit loader.

I have a follow-on change (I69ca3d26fede0506a6353c077c69f735c8d84d28)
to move disk-image-create to a regular python entry-point.

Unfortunately, this has to move to work with setuptools.  You'd think
a symlink under diskimage_builder/[elements|lib] would work, but it
doesn't.

[1] this API handles stuff like getting files out of .zip archive
modules, which we don't do.  Essentially for us it's returning
__file__.

Change-Id: I5e3e3c97f385b1a4ff2031a161a55b231895df5b
2016-11-01 17:27:41 -07:00