When merging feature/v2 we have re-enabled the centos-minimal test
which has a blank line in the element-deps. The modified get_elements
functions added in If97bcd21e45de1b5ed91023fdc441a4617051a6b were not
handling this.
Fixing this, we can extract this into some common code for
element-deps and element-provides.
Also, I noticed while debugging that the element list is an iterator
from reverse(), so printing it doesn't help. Fix up the debug output.
Change-Id: I8e9b079f586e8a36dba3797a0fc03ca1b2041a04
Change ec7f56c1b2 add added unit tests
in diskimage_builder/tests/functional (this is probably misnamed).
These are found and run by testr just as part of the normal "setup.py
test" run. Don't run them as part of the functional tests (this
breaks "-h"/"-l" because it installs a virtualenv and runs tests, and
also is incorrect in the gate where it's creating a nested virtualenv
underneath the testing virtualenv).
Change-Id: I9908e080042d3026a198ba89eb653c6eff376d22
During the creation of a disk image (e.g. for a VM), there is the need
to create, setup, configure and afterwards detach some kind of storage
where the newly installed OS can be copied to or directly installed
in.
This patch implements partitioning handling.
Change-Id: I0ca6a4ae3a2684d473b44e5f332ee4225ee30f8c
Signed-off-by: Andreas Florath <andreas@florath.net>
We landed the fix for this in
Icdb769541eee9793f261b4b8ec563be76ee13fe2.
This reverts commit 2978ff885b.
Change-Id: Iecfc41ab2aad57bc4f6f86a13810b534d19a8fd5
debian ships a modified site.py which has some interesting behavior when
VIRTUAL_ENV is set. In this case it will add
/usr/lib/pythonx.x/site-packages to the start of sys.path. This causes
pip to install packages to this location (rather than /usr/local). As a
result, later on when booting where VIRTUAL_ENV is not set this branch
is not hit and the path where python packages were installed is not part
of sys.path.
Change-Id: Icdb769541eee9793f261b4b8ec563be76ee13fe2
There are issues with pip packages and a python3 only Xenial systems.
This is occuring after Ie609de51cc5fcde701296c9474e315981d9778a2.
We believe the issue is with VIRTUAL_ENV being set within the chroot
and messing up pip installs
(Icdb769541eee9793f261b4b8ec563be76ee13fe2) but a full solution is not
yet clear.
For now, set Xenial to ensure we use python2. Install the package for
the ubuntu element (75-debian-minimal-baseinstall will install python2
for the minimal elements).
Change-Id: Id403919b0af93b375a900186c01a0d3a3bdfafea
Because we run this image in openstack-infra, we want to increase our
test coverage to help avoid potential breaks to our CI systems.
Change-Id: I26405e3f7465654075278ec35b5e0da1338bb45e
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Since we still run these 3 version of ubuntu-minimal elements in
openstack-infra, also run functional testing for them.
Trusty and xenial will be in voting gate, precise added as skipped for
non-voting.
Add the default skip/run status to the "-l" output just to confirm
this too.
Change-Id: Icfbfd0cb7d9acae824972474b77e2fe0486c4f69
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Set the grub timeout to 5 seconds by default, and add notes on how to
update this. This will stop infra having to carry an element that
goes and rewrites the grub configuration.
Change-Id: I556b3f48eff1b67ee8c4b9b64f749af95100fb99
expand_dependencies() was a public interface so we should try and
preserve backwards compat. However, since the interface is really
broken, add a new exported function "get_elements" that instack can
switch to. This returns the canonical list of elements without
duplicates, and gives the path to each element too.
This highlighted that the unit tests were really a bit wrong. They're
testing inner functions when we have an "API" in the get_elements()
function. Convert all unit-tests to use this function instead. Since
this is a library call, convert the sys.exit() calls to raised
exceptions.
Refactor the variable output into a separate function so we can do a
sanity check on it.
The added flake8 ignores are for the "over-indented for ... indent"
which happens a lot with these new longer lines. Most other projects
ignore them.
This is an alternative proposal to
I15609389c18adf3017220fc94552514d195b323a
Change-Id: If97bcd21e45de1b5ed91023fdc441a4617051a6b
Our setuptools action classifiers are woefully out of date, notably: we
are no longer alpha and we support python3.
Change-Id: I2425152129406e22073936275761bd5d850903fb
The squashfs format brings a couple of advantages over the other
formats. Image is often an order of magnitude smaller and it can
be used natively, either as an initrd, either with loop mount.
Change-Id: If72940b0c4dafb2504c52dd0429a8eb3f8305751
We now support tgz (tar.gz) as an output format.
Change-Id: Iadec92f2f96c3f904f28bd49f87ffc7d48ef7bd7
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
dracut has a "hostonly" mode where it builds an initramfs that is
suitable for booting the system it is building on. This is on by
default, but obviously in our nested multi-platform chroot situation
this is fraught with danger.
As highlighted by [1] our builds were inadvertently turning off
"hostonly" mode when the mountpoints in the chroot were not found.
The CentOS 7.3 behaviour change broke this and we ended up with an
initramfs with no file-system modules.
Iaf2a1e8470f642bfaaaad3f9b7f26cfc8cc445c9 introduced a regeneration of
the initramfs, which I think does work as described because it runs in
the loopback device.
However, dracut includes a package that installs configuration
overrides to build a generic initramfs. This is really what we want,
and should solve the problem no matter where the initramfs is created.
Add this package into yum-minimal and remove the extra re-create call
which should not be necessary.
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1405238
Change-Id: I5d203f2abe743cb23a44d449850e692a948e7871
openSUSE 13.1 was discontinued on Feb 3rd, 2016, so defaulting
to it doesn't make sense (see https://en.opensuse.org/Lifetime).
Leap 42.2 is the most current release that is supported by
disk-image-builder and being tested in a 3rd party ci.
Enable functests for it to ensure we're not regressing again.
Moved to non-voting gate first.
Depends-On: Iff495b3cd0b6c3558c44cf4883651eca67b572d6
Change-Id: Iae6cd34a5853f1e309861c554d94d8595cbd9993
For some reason [1] introduced -m option without ever checking that the
mapping exists. Because there is no grub-ieee1275 mapping anywhere (not
in base, not in bootloader), pkg-map fails. So stop using the mapping in
package-install of grub-ieee1275 on ppc.
There is another patch that tries to solve the same bug by adding the
mapping [2]. I think it is better to undo the breakage introduced in [1]
first, and then, if various distributions have differing names for the
package, introduce various mappings. My reasoning is that at the moment
this element is broken for all ppc64 distributions. This patch would
fix it for some (namely, Ubuntu). Then we can add mappings as tests
are done for other distributions.
[1] Ibca43173c30c2a74a73a2e2d9dd6d6d832c62694
[2] Id2b0f63a7015f883070fd59b79fd96a1c024858a
Change-Id: I8425876c26e9e416c8ce2f53a4e38d26b4208633
Closes-Bug: #1624021
dracut has a loop [1] where it probes top-level directories, tries to
find what block device they are on, then determines the file-system of
that block device. It then puts those file-system modules into the
initramfs for boot.
Since we install the kernel package during the chroot phase, / there
is not a block device and thus this loop matches nothing and we end up
with no file-system modules in the initramfs. This results in a very
annoying silent boot hang.
By moving re-generation of dracut into finalise.d phase, we run inside
the final image where / is the loop-device; the root file-system gets
detected correctly and the ext4 module is included correctly.
[1] http://git.kernel.org/cgit/boot/dracut/dracut.git/tree/dracut.sh?h=RHEL-7#n1041
Change-Id: Iaf2a1e8470f642bfaaaad3f9b7f26cfc8cc445c9
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Tripleo-image-elements have an install.d file '05-heat-cfntools' that runs
the following command:
virtualenv --setuptools $VENV
With the recent change to diskimage-builder (moving the install of pip
and virtualenv to the 10- range) virtualenv is no longer available for
this elementr; as a side-effect, the trove kick-start command is now
broken and gate jobs are failing.
The solutions is to move the (now) 10-install-pip to 04-install-pip.
This should still alleviate the race condition that
https://review.openstack.org/#/c/408277/ attempted to fix, as all
*-package-installs files are 00-, 01- or 02-.
Change-Id: Ia4e01f00c4c5e9a2087df1e2a91d9154480a0422
Closes-Bug: #1650008
Commit 6278371eaa13("Make dib-python use the default python for distro")
added default python version for various distros but it missed openSUSE
which leads to build failures since the openSUSE elements are pulling
python2 packages. Add openSUSE to the list of python2 distributions
until python3 support for the openSUSE elements is in place.
Change-Id: I95f1fa849a22607c430387a2a915f9d19c9c209f
We are explicitly calling python in this element which does not work on
systems which only have python3.
Change-Id: Ia730850a48e2478fd5461710a9d2619408725cd8
Now that we are explicit about what python version we intend to use
for dib we can have package installs optionally install packages
depending on this. Add a new dib_python_version that matches on the
DIB_PYTHON_VERSION string set by dib-python.
Co-Authored-By: Adam Harwell <flux.adam@gmail.com>
Change-Id: I70659aab7d12924bdb9bc0489a7f02d5fd0dbb39
