This came in with edc06a20e5 although
it's not totally clear why.
We found that we were sourcing devstack scripts during build in infra,
and inadvertently relying on this package being around on the host.
This was incorrect but can this type of problem is hidden if the
package is installed unnecessarily.
I do not think this actually required now; remove from bindep to
minimise chance for such false or hidden dependencies.
Change-Id: I7895d4c5019511eb2169915cd8092e707e8ee732
Story: #2007407
Task: #39068
putting it in requirements means it's installed for anything that
installs diskimage-builder. Not just for tests, but in actual prod
installations. Because of the way flake8 plugins work, this means
then that hacking is injected into all flake8 invocations in that
environment.
This is currently causing issues in nodepool where we do not
actually use hacking but it's still spewing spurious messages to
the screen because it is incorrect finding our apache header to be
incorrect.
Just depend on flake8, since that's what dib-lint depends on.
Change-Id: I8d2b2076b604d596906a3b12a5518e2a56c0fe88
All the platforms we care about now have python3 with venv (even
centos7 now) packaged somehow. Add an ensure-venv element to make
sure that "python3 -m venv" works. Any other elements that wish to
install non-distribution-packaged Python utilities can use this to
keep them separate from the main system installs.
Port glean to use this, and drop its dependency on pip-and-virtualenv.
Change-Id: Ic16f134fe34293bb68e7c632dd320f523366320d
This causes problems for other projects incorporating dib; we don't
have a specific need for a cap.
Fix a few issues, mostly spacing or regex matches. No functional
changes.
W503 and W504 relate to leaving artithmetic operators at the start or
end of lines, and are mutually exclusive and, due to "ignore"
overriding the defaults both get enabled. It seems everyone gets this
wrong (https://gitlab.com/pycqa/flake8/issues/466). Don't take a
position on this and ignore both.
Use double # around comments including YAML snippets using "# type: "
which now gets detected as PEP484/mypy type hints.
Change-Id: I8b7ce6dee02dcce31c82427a2441c931d136ef57
* Add "centos" element, a CentOS version-independent element. This is in
line with the same work done for RHEL in Stein cycle.
* Deprecate the centos7 element. CentOS 7 support itself it not
deprecated though. The new "centos" element provides the same support
level as the "centos7" element.
* Add functional testing
The default CentOS version is 8. You can adjust it using the DIB_RELEASE
environment variable.
Change-Id: I373ba2296c4613765676e59aabd9c651345298d1
in CentOS build case building an image with "iscsi-boot" and "dracut-regenerate" will exit building because of statement "[ "$found" = 0 ]"
Change-Id: I1a6d60e9ec5f5cb508866c8376465c3e73551a30
On IPA we are using efivar and efibootmgr, we already added the
packages on ipa-builder.
Adding the pacakges on diskimage-builder, so that people who use
it to build the images won't get into trouble.
Change-Id: I9ab6588f20302b4808b09dc060aced5fd267a3d2
Debian default Python interpreter version is 2.7, but it's
possible to install a Python 3 interpreter from the base
repository.
With this change, if we set DIB_PYTHON_VERSION to 3, we install
the python3 package from base, with python3-libs, python3-pip and
python3-setuptools, and redefine python_path, effectively allowing
Python 3 interpreter to be used in Debian.
See a result of the job for building the ipa image here:
https://review.opendev.org/705773
Change-Id: Idabfa94c2bff6e0de6daa0866084d5db14d7dcb0
When there is a hashsum mismatch diskimage-builder forces downloads
with the -f switch of cache-url. This is currently broken because bash
escapes the quotes in curl_opts. This tricks curl trying to download
'no-cache' instead of the url. This can be fixed by using an array for
curl_opts which does the correct thing here.
Change-Id: Id9f1579dda9a3e0a2b08dd5faaeef0e2e580d419
Ironic is now using diskimage-builder for producing and publishing
official production-ready IPA ramdisks. These jobs test ironic
against changes in diskimage-builder to ensure they still work:
* Two check-image jobs merely build an IPA image with DIB:
* The CentOS 8 job is our primary now and should eventually
become voting.
* The CentOS 7 one is used on stable/train and will be removed
once DIB stops supporting this version.
* The two other jobs build and use the IPA image to deploy a node
in two different ways (pushed via iSCSI and pulled via HTTP).
Since they're likely to be affected by factors outside of DIB
control, they're added non-voting.
Depends-On: https://review.opendev.org/703585
Change-Id: Ibce594657bb90db1bac63fa9f965ee88586f8246
Add a basic test to ensure that all elements have a README.rst file.
This way they will be exhaustively listed in the Sphinx documentation.
Add dummy README.rst for 'disable-selinux' and 'rpm-distro' elements.
Change-Id: Ia5252ddd89b5ae5c6e9a12a66ef10f912fd54da5
OpenDev is removing trusty from the infrastructure, remove the jobs
using ubuntu-trusty.
Depends-On: https://review.opendev.org/702817
Change-Id: I0d328f4d1125be7e3801a9dd2c9374086024b903
CentOS 8.1 split repositories and GPG keys out into subpackages. This
broke DIB support for CentOS 8.
7e41cef41826a0d73ced
Change-Id: If3de6efa6074e059dc9fdd47c7bdc19d26d4d7f2
The hook inside extra-data.d runs outside the chroot when
building the image which means that we need to prefix paths
inside the hook to avoid running things on the host.
We also run it with sudo because if we're running DIB not
as root, /etc is uid 0 and we'll get a permission denied.
Change-Id: I1838890fe124c84c879285a471bcc78fe47d6c23
Make sure rngd, a hardware RNG entropy gatherer daemon, is installed on
all DIB-built Red Hat family distro images. rngd comes installed by
default in a typical base installation as it's proven to help speed
things up.
Nova attaches the virtio-rng-pci device to VMs. virtio-rng-pci is a
device that provides feed random data. However, it is of little to no
use if the virtual machine is not configured to make use of given
device. That is where rngd can help by facilitating entropy to the pool
from virtio-rng-pci.
$ openstack image set --property hw_rng_model=virtio [...]
$ openstack flavor set --property hw_rng:allowed=True [...]
DIB-built minimal images do not come with rngd installed. This patch
makes sure the daemon is installed. Its systemd service comes already
enabled.
Change-Id: I34a989dbfc57d4c98113ac25c81dfb500945ff0a
Install policycoreutils-python in RHEL/CentOS 7.
Install policycoreutils-python-utils in Fedora and RHEL/CentOS 8.
This patch also drops python3-setuptools from bindep as Fedora 28 is
EOL and RPM dependency is fixed in >= Fedora 29.
Change-Id: I0ecd29e0f113005d7e993add84d2c4fb90fd16e3
Some phases of diskimage-builder run outside the chroot environment,
such as the extra-data.d scripts, and don't have access to dib-python.
This means these scripts may choose the wrong python version by using
"#!/usr/bin/env python" to execute. The svc-map element is an example.
This patch creates a temporary directory and symbolic link for the
correct version of python, then manipulates the environment PATH
to preference the symbolic link "python" command.
This will allow elements with these scripts to work correctly with
the version of python diskimage-builder is running under.
Change-Id: I289d621e1bfbba0eb174dff977d1a5c92c04e4fa
Co-Authored-By: Ian Wienand <iwienand@redhat.com>
As described inline, Bionic hosts will build invalid Trusty images.
Hack around this by disabling metadata_csum in the ext4 mkfs.
Change-Id: Ibd67d58ca830a9e60605d0700ee2b17906c804e6
Devstack dropped Xenial support with
Iefcca99904dde76b34efbbfc0e04515dfa5a09e5.
I have ported the required debootstrap in the openstackci PPA to
Bionic, so these builds should work there now.
Unfortunatley, there's no current solution for Suse builds as there is
no zypper for the minimal builds on bionic. These will either have to
fix that, or figure out how to pin devstack to Train branch for the
jobs. Since it causes retries in the gate as non-voting, put it in
experimental for now until we figure something out.
Trusty also fails due to issues with ext4 versions using bionic hosts.
This is fixed in a follow-on
Ibd67d58ca830a9e60605d0700ee2b17906c804e6; the job is disabled for now.
Change-Id: I71823236731583e28fddcceb71f44d09b58664a4
The base URL of EPEL repository installed by the epel-release package in
CentOS 8 at least now defaults to https.
The error seen when building an CentOS 8 image was:
"Error: Cannot find a valid baseurl for repo: epel"
This patch fixes it so that it will always match regardless of being
http or https.
Change-Id: I9ec5536ee72047c929a1ef6950ff4e9092842a4c
The ndisc6 package is not yet available in EPEL 8.
See: https://bugzilla.redhat.com/show_bug.cgi?id=1779134
Until the package is available set the pkg-map to "" for
the ndisc6 package when distro is redhat and install the
package using || true in the element script instead so
that CentOS 8 build's do not fail because of the missing
package.
Once the package is in EPEL 8 this change can be reverted.
Related-Bug: #1754219
Change-Id: Icd4bad8852ce5ba40fb0e7b0d335191efbe88c67
After the introduction of 'Add output for mis-configured element
scripts' we started seeing CI failures in tripleo where
instack-undercloud is being used (rocky/queens):
/usr/lib/python2.7/site-packages/diskimage_builder/lib/dib-run-parts: line 108: DIB_DEBUG_TRACE: unbound variable
INFO: 2019-12-02 16:24:33,423 -- ############### End stdout/stderr logging ###############
ERROR: 2019-12-02 16:24:33,423 -- Hook FAILED.
Let's make sure that by default the env variable is set
to 0.
Change-Id: I38c76c0edee436f1e7dd0c9a868cea1e6ee3271d
Closes-Bug: #1854904
Without this change, operating system elements that use the
zypper-minimal element always must use download.opensuse.org as their
repository source. This change makes ZYPPER_REPOS overrideable, which
allows the user to create custom operating system elements that can use
private repositories as their source for base packages. For example,
with only this change, it is possible to create a sles-minimal element
that generates a SLE 15 SP1 image just by overriding DIB_ZYPPER_REPOS
and DIB_OPENSUSE_PATTERNS.
Change-Id: I46e40fbe4408d4204056a27b182b21213f1176ff
On openSUSE Tumbleweed, the login.defs config file was moved under
/usr[1]. This change allows the login.defs config change to work for
both old and new locations.
[1] https://build.opensuse.org/request/show/736424
Change-Id: Ia5eff5e7b0709836278361b1b8daa788619eff75
