Commit Graph

4156 Commits

Author SHA1 Message Date
Zuul
8ad2956911 Merge "Do not include efibootmgr and efivars for ppc architectures" 2020-02-26 04:02:42 +00:00
Bob Fournier
14a3776885 Do not include efibootmgr and efivars for ppc architectures
These don't exist and are not needed for ppc.

Change-Id: I9ba53e22583b148b43f74cd9b8ec79402796d09b
2020-02-25 13:27:13 -05:00
Chandan Kumar (raukadah)
31a32ff6ee Change tgt pkg-map to target-restore CentOS/RHEL-8
In ironic-agent tgt pkg-map, scsi-target-utils was used for
redhat family which is not used in CentOS/RHEL-8.

As per https://opendev.org/openstack/ironic-python-agent-builder/src/branch/master/dib/ironic-python-agent-ramdisk/pkg-map#L5
targetcli got removed and switch to target-restore. It syncs
the same element for RHEL-8 and adds the same for CentOS-8.

Closes-Bug: #1864427

Change-Id: I95a0c09a6739af23cd1e8c88dded198bd69cc53e
Signed-off-by: Chandan Kumar (raukadah) <chkumar@redhat.com>
2020-02-24 14:17:24 +05:30
Ian Wienand
28ebd24844 Uncap hacking
This causes problems for other projects incorporating dib; we don't
have a specific need for a cap.

Fix a few issues, mostly spacing or regex matches.  No functional
changes.

W503 and W504 relate to leaving artithmetic operators at the start or
end of lines, and are mutually exclusive and, due to "ignore"
overriding the defaults both get enabled.  It seems everyone gets this
wrong (https://gitlab.com/pycqa/flake8/issues/466).  Don't take a
position on this and ignore both.

Use double # around comments including YAML snippets using "# type: "
which now gets detected as PEP484/mypy type hints.

Change-Id: I8b7ce6dee02dcce31c82427a2441c931d136ef57
2020-02-24 10:34:46 +11:00
Carlos Goncalves
8226384cf0 Add CentOS 8 support
* Add "centos" element, a CentOS version-independent element. This is in
  line with the same work done for RHEL in Stein cycle.
* Deprecate the centos7 element. CentOS 7 support itself it not
  deprecated though. The new "centos" element provides the same support
  level as the "centos7" element.
* Add functional testing

The default CentOS version is 8. You can adjust it using the DIB_RELEASE
environment variable.

Change-Id: I373ba2296c4613765676e59aabd9c651345298d1
2020-02-19 10:44:56 +01:00
Zuul
9d08848f25 Merge "Fix cache-url -f" 2020-02-18 22:55:23 +00:00
Zuul
0ec74b948c Merge "Allow python3 to be used in Debian" 2020-02-17 18:18:07 +00:00
Noam Angel
48eac8b899 fix iscsi-boot element exiting build even if dracut-regenerate used
in CentOS build case building an image with "iscsi-boot" and "dracut-regenerate" will exit building because of statement "[ "$found" = 0 ]"

Change-Id: I1a6d60e9ec5f5cb508866c8376465c3e73551a30
2020-02-12 09:09:20 +00:00
Iury Gregory Melo Ferreira
6986441e2c Add efi packages for ironic-agent
On IPA we are using efivar and efibootmgr, we already added the
packages on ipa-builder.
Adding the pacakges on diskimage-builder, so that people who use
it to build the images won't get into trouble.

Change-Id: I9ab6588f20302b4808b09dc060aced5fd267a3d2
2020-02-11 17:26:59 +01:00
Riccardo Pittau
cf7d39e4cd Allow python3 to be used in Debian
Debian default Python interpreter version is 2.7, but it's
possible to install a Python 3 interpreter from the base
repository.
With this change, if we set DIB_PYTHON_VERSION to 3, we install
the python3 package from base, with python3-libs, python3-pip and
python3-setuptools, and redefine python_path, effectively allowing
Python 3 interpreter to be used in Debian.
See a result of the job for building the ipa image here:
https://review.opendev.org/705773

Change-Id: Idabfa94c2bff6e0de6daa0866084d5db14d7dcb0
2020-02-07 18:10:19 +01:00
Tobias Henkel
bad433fa92
Fix cache-url -f
When there is a hashsum mismatch diskimage-builder forces downloads
with the -f switch of cache-url. This is currently broken because bash
escapes the quotes in curl_opts. This tricks curl trying to download
'no-cache' instead of the url. This can be fixed by using an array for
curl_opts which does the correct thing here.

Change-Id: Id9f1579dda9a3e0a2b08dd5faaeef0e2e580d419
2020-02-05 10:19:12 +01:00
Zuul
500e60dbf4 Merge "Enable possibility to select HWE kernel for Ubuntu minimal" 2020-02-04 09:00:47 +00:00
Zuul
0668cd37fe Merge "Add ironic jobs to the CI" 2020-02-04 06:23:52 +00:00
Dmitry Tantsur
3b27ae1578 Add ironic jobs to the CI
Ironic is now using diskimage-builder for producing and publishing
official production-ready IPA ramdisks. These jobs test ironic
against changes in diskimage-builder to ensure they still work:
* Two check-image jobs merely build an IPA image with DIB:
  * The CentOS 8 job is our primary now and should eventually
    become voting.
  * The CentOS 7 one is used on stable/train and will be removed
    once DIB stops supporting this version.
* The two other jobs build and use the IPA image to deploy a node
  in two different ways (pushed via iSCSI and pulled via HTTP).
  Since they're likely to be affected by factors outside of DIB
  control, they're added non-voting.

Depends-On: https://review.opendev.org/703585
Change-Id: Ibce594657bb90db1bac63fa9f965ee88586f8246
2020-01-21 12:48:32 +01:00
Antoine Musso
168127b60a dib-lint: test elements have README.rst file
Add a basic test to ensure that all elements have a README.rst file.
This way they will be exhaustively listed in the Sphinx documentation.

Add dummy README.rst for 'disable-selinux' and 'rpm-distro' elements.

Change-Id: Ia5252ddd89b5ae5c6e9a12a66ef10f912fd54da5
2020-01-20 11:43:43 +01:00
Fatih Degirmenci
e2af38e3a5 Enable possibility to select HWE kernel for Ubuntu minimal
Change-Id: I206a77590f575e472e31b4f867fd5fd35475542d
2020-01-17 10:47:37 +00:00
Andreas Jaeger
71ef0a8b3c Remove trusty jobs
OpenDev is removing trusty from the infrastructure, remove the jobs
using ubuntu-trusty.

Depends-On: https://review.opendev.org/702817
Change-Id: I0d328f4d1125be7e3801a9dd2c9374086024b903
2020-01-17 09:23:05 +01:00
Carlos Goncalves
ae2be0b464 Fix Yum repositories and GPG keys for CentOS 8.1
CentOS 8.1 split repositories and GPG keys out into subpackages. This
broke DIB support for CentOS 8.

7e41cef418
26a0d73ced

Change-Id: If3de6efa6074e059dc9fdd47c7bdc19d26d4d7f2
2020-01-15 19:39:00 +01:00
Matthew Thode
87e7f7b869
allow building of Gentoo images for non-systemd profiles
Change-Id: I61e3ba391eaaf9b300f88b082c03116388401ba3
2019-12-20 17:51:55 -06:00
Mohammed Naser
6f1b51627f modprobe.d: use $TMP_MOUNT_PATH
The hook inside extra-data.d runs outside the chroot when
building the image which means that we need to prefix paths
inside the hook to avoid running things on the host.

We also run it with sudo because if we're running DIB not
as root, /etc is uid 0 and we'll get a permission denied.

Change-Id: I1838890fe124c84c879285a471bcc78fe47d6c23
2019-12-18 11:42:02 -05:00
Zuul
71e1bcbdb8 Merge "Install rng-tools in Red Hat family distro images" 2019-12-17 06:25:32 +00:00
Zuul
f6d32a684f Merge "Add arm64 based functional test" 2019-12-16 21:31:51 +00:00
Ian Wienand
082397a86a Add arm64 based functional test
An initial functional test for bionic/arm64 builds, put it in the new
arm64 check queue.

Change-Id: I5f8a047f41c6555da7211b758c55f7a87b3aa5d1
2019-12-16 10:51:01 +11:00
Zuul
db9501cab2 Merge "Fix wrong URL in ironic-agent element" 2019-12-15 23:43:33 +00:00
Carlos Goncalves
3ff7365ee8 Install rng-tools in Red Hat family distro images
Make sure rngd, a hardware RNG entropy gatherer daemon, is installed on
all DIB-built Red Hat family distro images. rngd comes installed by
default in a typical base installation as it's proven to help speed
things up.

Nova attaches the virtio-rng-pci device to VMs. virtio-rng-pci is a
device that provides feed random data. However, it is of little to no
use if the virtual machine is not configured to make use of given
device. That is where rngd can help by facilitating entropy to the pool
from virtio-rng-pci.

$ openstack image set --property hw_rng_model=virtio [...]
$ openstack flavor set --property hw_rng:allowed=True [...]

DIB-built minimal images do not come with rngd installed. This patch
makes sure the daemon is installed. Its systemd service comes already
enabled.

Change-Id: I34a989dbfc57d4c98113ac25c81dfb500945ff0a
2019-12-12 20:26:06 +00:00
Carlos Goncalves
45d9917c00 Update bindep for RHEL/CentOS 8
Install policycoreutils-python in RHEL/CentOS 7.
Install policycoreutils-python-utils in Fedora and RHEL/CentOS 8.

This patch also drops python3-setuptools from bindep as Fedora 28 is
EOL and RPM dependency is fixed in >= Fedora 29.

Change-Id: I0ecd29e0f113005d7e993add84d2c4fb90fd16e3
2019-12-12 18:28:56 +01:00
Zuul
74dff74907 Merge "Set correct python version for non-chroot scripts" 2019-12-12 16:48:17 +00:00
Madhuri Kumari
46491eb1b5 Fix wrong URL in ironic-agent element
Change-Id: Iaa107094b1409f93b101b23cea2e874c60ea26f5
2019-12-12 16:39:48 +05:30
Michael Johnson
9c7b8d1714 Set correct python version for non-chroot scripts
Some phases of diskimage-builder run outside the chroot environment,
such as the extra-data.d scripts, and don't have access to dib-python.
This means these scripts may choose the wrong python version by using
"#!/usr/bin/env python" to execute. The svc-map element is an example.

This patch creates a temporary directory and symbolic link for the
correct version of python, then manipulates the environment PATH
to preference the symbolic link "python" command.
This will allow elements with these scripts to work correctly with
the version of python diskimage-builder is running under.

Change-Id: I289d621e1bfbba0eb174dff977d1a5c92c04e4fa
Co-Authored-By: Ian Wienand <iwienand@redhat.com>
2019-12-11 22:52:28 +00:00
Ian Wienand
801fa77a16 Work around Trusty ext4 metadata_csum errors on Bionic
As described inline, Bionic hosts will build invalid Trusty images.
Hack around this by disabling metadata_csum in the ext4 mkfs.

Change-Id: Ibd67d58ca830a9e60605d0700ee2b17906c804e6
2019-12-11 16:02:13 +11:00
Ian Wienand
4ca306cc0e Drop Xenial from functional test
Devstack dropped Xenial support with
Iefcca99904dde76b34efbbfc0e04515dfa5a09e5.

I have ported the required debootstrap in the openstackci PPA to
Bionic, so these builds should work there now.

Unfortunatley, there's no current solution for Suse builds as there is
no zypper for the minimal builds on bionic.  These will either have to
fix that, or figure out how to pin devstack to Train branch for the
jobs.  Since it causes retries in the gate as non-voting, put it in
experimental for now until we figure something out.

Trusty also fails due to issues with ext4 versions using bionic hosts.
This is fixed in a follow-on
Ibd67d58ca830a9e60605d0700ee2b17906c804e6; the job is disabled for now.

Change-Id: I71823236731583e28fddcceb71f44d09b58664a4
2019-12-11 16:01:52 +11:00
Zuul
361a751d36 Merge "Allow zypper repos to be overrideable" 2019-12-04 06:14:55 +00:00
Zuul
68b41854b2 Merge "Install ndisc6 package in element script" 2019-12-03 22:41:09 +00:00
Zuul
9b00377f31 Merge "Fix regex for mirror URL substitution" 2019-12-03 21:56:19 +00:00
Zuul
0b1eba25d5 Merge "Break retry loop on success in dhcp-all-interfaces" 2019-12-03 21:56:18 +00:00
Zuul
e3a904d4a6 Merge "Fix login.defs config for tumbleweed" 2019-12-03 21:56:16 +00:00
Carlos Goncalves
670df3326d Fix regex for mirror URL substitution
The base URL of EPEL repository installed by the epel-release package in
CentOS 8 at least now defaults to https.

The error seen when building an CentOS 8 image was:
"Error: Cannot find a valid baseurl for repo: epel"

This patch fixes it so that it will always match regardless of being
http or https.

Change-Id: I9ec5536ee72047c929a1ef6950ff4e9092842a4c
2019-12-03 19:13:18 +01:00
Harald Jensås
7948fee7e2 Install ndisc6 package in element script
The ndisc6 package is not yet available in EPEL 8.
See: https://bugzilla.redhat.com/show_bug.cgi?id=1779134

Until the package is available set the pkg-map to "" for
the ndisc6 package when distro is redhat and install the
package using || true in the element script instead so
that CentOS 8 build's do not fail because of the missing
package.

Once the package is in EPEL 8 this change can be reverted.

Related-Bug: #1754219
Change-Id: Icd4bad8852ce5ba40fb0e7b0d335191efbe88c67
2019-12-03 14:57:33 +01:00
Michele Baldessari
f9dcbd30cc Make sure DIB_DEBUG_TRACE has a default value
After the introduction of 'Add output for mis-configured element
scripts' we started seeing CI failures in tripleo where
instack-undercloud is being used (rocky/queens):

  /usr/lib/python2.7/site-packages/diskimage_builder/lib/dib-run-parts: line 108: DIB_DEBUG_TRACE: unbound variable
  INFO: 2019-12-02 16:24:33,423 -- ############### End stdout/stderr logging ###############
  ERROR: 2019-12-02 16:24:33,423 -- Hook FAILED.

Let's make sure that by default the env variable is set
to 0.

Change-Id: I38c76c0edee436f1e7dd0c9a868cea1e6ee3271d
Closes-Bug: #1854904
2019-12-03 08:26:12 +01:00
Colleen Murphy
ad67aa213b Allow zypper repos to be overrideable
Without this change, operating system elements that use the
zypper-minimal element always must use download.opensuse.org as their
repository source. This change makes ZYPPER_REPOS overrideable, which
allows the user to create custom operating system elements that can use
private repositories as their source for base packages. For example,
with only this change, it is possible to create a sles-minimal element
that generates a SLE 15 SP1 image just by overriding DIB_ZYPPER_REPOS
and DIB_OPENSUSE_PATTERNS.

Change-Id: I46e40fbe4408d4204056a27b182b21213f1176ff
2019-12-02 16:25:53 -08:00
Colleen Murphy
4fb5a57b8a Fix login.defs config for tumbleweed
On openSUSE Tumbleweed, the login.defs config file was moved under
/usr[1]. This change allows the login.defs config change to work for
both old and new locations.

[1] https://build.opensuse.org/request/show/736424

Change-Id: Ia5eff5e7b0709836278361b1b8daa788619eff75
2019-12-02 16:21:45 -08:00
Hervé Rousseau
b91e212434 Break retry loop on success in dhcp-all-interfaces
If rdisc6 is available, a node using this element will loops until
DIB_DHCP_TIMEOUT is reached because of a missing 'break' when rdisc6
return code is 0.
This will mark the dhcp-interface@.service unit as failed (because it
has the same timeout) and not bring any network interface online.

Change-Id: I034dcda94d765f236950ebcbee36789f5bdc515f
Closes-Bug: #1854717
Signed-off-by: Hervé Rousseau <hroussea@cern.ch>
2019-12-02 15:50:20 +01:00
Zuul
a231bc6b9f Merge "Add IPv6 support in dhcp-all-interfaces" 2019-11-26 23:54:54 +00:00
Zuul
20e468c604 Merge "Stop installing pydistutils.cfg" 2019-11-26 02:12:41 +00:00
Monty Taylor
75341292f9 Stop installing pydistutils.cfg
Support for easy_install codepaths is increasingly broken, and now
putting allow-hosts in this file breaks most recent pip. Just stop
installing the file - people should be using pip anyway.

Change-Id: I0a6b2432f81d80fbcbb336403fe555003880fa9f
2019-11-25 11:46:42 +11:00
Zuul
4248858925 Merge "Update bindep.txt for some missing dependencies" 2019-11-22 07:43:05 +00:00
Zuul
fdb6225acc Merge "Add output for mis-configured element scripts" 2019-11-22 06:57:40 +00:00
Zuul
9dd161471a Merge "Ensure nouveau is blacklisted in initramfs too" 2019-11-22 06:57:39 +00:00
Zuul
f268c72ed9 Merge "Adds support for GPG keyring" 2019-11-22 06:57:38 +00:00
Zuul
05a6f898fd Merge "Only wait for checksum processes" 2019-11-22 06:57:37 +00:00