Commit Graph

576 Commits

Author SHA1 Message Date
Ian Wienand
df3ad26f58 yum-minimal: strip env vars in chroot calls
This showed up with dnf in containers when TMPDIR was set; dnf started
trying to write to this directory while in the chroot.

We already do stripping like this in run_in_target -- but this is a
bit of a unique place because it's actually setting up the initial
chroot so the target doesn't actually exist yet; so we just hard-code
it in place here.

Change-Id: If7310cb820846da903bf60daa4486c8bf7cb0136
2020-04-24 12:38:42 +10:00
Ian Wienand
3f823488c5 Add centos aarch64 tests
Change-Id: I2fffeeffa82c505a34c5c9972724b557f3b58975
2020-04-16 14:15:50 +10:00
Dirk Mueller
68bb43535e opensuse: fix python 2.x install
openSUSE Tumbleweed is dropping python2-* packages so we need
to stop intalling them. We can also stop installing those
for Leap 15. which avoids a pip uninstall issue (as python2-six
was still built with distribute).

Change-Id: Ie93c8addb26aab3d0154c4b5b52423799abede91
2020-04-03 18:22:33 +02:00
Zuul
ac250e5ab0 Merge "centos 8 image build: fix mirror" 2020-03-31 05:03:06 +00:00
Zuul
dc67ccfe12 Merge "Mellanox element: removed ibutils,libibcm,libmlx4-dev" 2020-03-30 22:10:13 +00:00
Ian Wienand
56dc0a2c31 centos 8 image build: fix mirror
We're ending up with "centoscentos" in the mirror location and the
build fails; strip out the $contentdir from the original too.

Change-Id: If09dbbd8028ea510d2ab0d3d8afe484cea611df5
2020-03-31 08:57:15 +11:00
Zuul
21f9ea4f78 Merge "Add Fedora 31 support and test jobs" 2020-03-30 01:26:32 +00:00
Lee Yarwood
ee9ad32b6f Add Fedora 31 support and test jobs
Change-Id: Iad0261ac8db001ffa4d3a4dd6df05fe923402a69
2020-03-30 08:47:00 +11:00
Zuul
56d9956cbd Merge "Add python-stow-versions element" 2020-03-27 13:49:21 +00:00
Lenny Verkhovsky
2f4e1886e5 Mellanox element: removed ibutils,libibcm,libmlx4-dev
Those packages will be installed as part of dependencies

Change-Id: I640dc55c818dd8538bfcd199c33f9af93aae7608
2020-03-23 00:06:53 +02:00
Harald Jensås
1ac31afd62 Use rpm -e instead of dnf for cleaning old kernels
If the running kernel of the system building the image
matches the kernel that is to be removed dnf will fail.

Repalce use use of dnf with rpm -e.

Closes-Bug: #1623409
Change-Id: Ie2481ea8a02b7b0720e46fa179f24badf4aa25c5
2020-03-19 22:35:23 +01:00
Dmitriy Rabotyagov
1de6fe4ba7 Add python-stow-versions element
This element is designed to install latest minor versions of different
python releases, like py27, py35, py36, py37, py38
into stow directory, and later easily enable them with stow.

Change-Id: Iab6d20e7643e549b53c629fb430e58b1c5e72991
2020-03-19 21:21:01 +02:00
Monty Taylor
c113703050 Add support for build-only packages
Sometimes an element needs packages installed so that it can
perform tasks but those package are not appropriate for the
final image content. Add a "build-only" flag to package-install-squash
which will cause package to be installed at the beginning of the
phase and then uninstalled at the end of the phase.

Change-Id: Ie01b795991710c93f6b669c8f14b57eb4412c1d5
2020-03-18 14:40:44 -05:00
Zuul
61b72ca2c2 Merge "Add ensure-venv element, install glean with it" 2020-03-10 05:08:54 +00:00
Zuul
991586c20a Merge "Uncap hacking" 2020-03-10 01:16:02 +00:00
Ian Wienand
82cfcfe551 Add ensure-venv element, install glean with it
All the platforms we care about now have python3 with venv (even
centos7 now) packaged somehow.  Add an ensure-venv element to make
sure that "python3 -m venv" works.  Any other elements that wish to
install non-distribution-packaged Python utilities can use this to
keep them separate from the main system installs.

Port glean to use this, and drop its dependency on pip-and-virtualenv.

Change-Id: Ic16f134fe34293bb68e7c632dd320f523366320d
2020-03-10 11:57:43 +11:00
Zuul
18f46bde30 Merge "fix iscsi-boot element exiting build even if dracut-regenerate used" 2020-02-26 04:05:57 +00:00
Zuul
8deca122f9 Merge "Change tgt pkg-map to target-restore CentOS/RHEL-8" 2020-02-26 04:05:56 +00:00
Zuul
8ad2956911 Merge "Do not include efibootmgr and efivars for ppc architectures" 2020-02-26 04:02:42 +00:00
Bob Fournier
14a3776885 Do not include efibootmgr and efivars for ppc architectures
These don't exist and are not needed for ppc.

Change-Id: I9ba53e22583b148b43f74cd9b8ec79402796d09b
2020-02-25 13:27:13 -05:00
Chandan Kumar (raukadah)
31a32ff6ee Change tgt pkg-map to target-restore CentOS/RHEL-8
In ironic-agent tgt pkg-map, scsi-target-utils was used for
redhat family which is not used in CentOS/RHEL-8.

As per https://opendev.org/openstack/ironic-python-agent-builder/src/branch/master/dib/ironic-python-agent-ramdisk/pkg-map#L5
targetcli got removed and switch to target-restore. It syncs
the same element for RHEL-8 and adds the same for CentOS-8.

Closes-Bug: #1864427

Change-Id: I95a0c09a6739af23cd1e8c88dded198bd69cc53e
Signed-off-by: Chandan Kumar (raukadah) <chkumar@redhat.com>
2020-02-24 14:17:24 +05:30
Ian Wienand
28ebd24844 Uncap hacking
This causes problems for other projects incorporating dib; we don't
have a specific need for a cap.

Fix a few issues, mostly spacing or regex matches.  No functional
changes.

W503 and W504 relate to leaving artithmetic operators at the start or
end of lines, and are mutually exclusive and, due to "ignore"
overriding the defaults both get enabled.  It seems everyone gets this
wrong (https://gitlab.com/pycqa/flake8/issues/466).  Don't take a
position on this and ignore both.

Use double # around comments including YAML snippets using "# type: "
which now gets detected as PEP484/mypy type hints.

Change-Id: I8b7ce6dee02dcce31c82427a2441c931d136ef57
2020-02-24 10:34:46 +11:00
Carlos Goncalves
8226384cf0 Add CentOS 8 support
* Add "centos" element, a CentOS version-independent element. This is in
  line with the same work done for RHEL in Stein cycle.
* Deprecate the centos7 element. CentOS 7 support itself it not
  deprecated though. The new "centos" element provides the same support
  level as the "centos7" element.
* Add functional testing

The default CentOS version is 8. You can adjust it using the DIB_RELEASE
environment variable.

Change-Id: I373ba2296c4613765676e59aabd9c651345298d1
2020-02-19 10:44:56 +01:00
Zuul
9d08848f25 Merge "Fix cache-url -f" 2020-02-18 22:55:23 +00:00
Zuul
0ec74b948c Merge "Allow python3 to be used in Debian" 2020-02-17 18:18:07 +00:00
Noam Angel
48eac8b899 fix iscsi-boot element exiting build even if dracut-regenerate used
in CentOS build case building an image with "iscsi-boot" and "dracut-regenerate" will exit building because of statement "[ "$found" = 0 ]"

Change-Id: I1a6d60e9ec5f5cb508866c8376465c3e73551a30
2020-02-12 09:09:20 +00:00
Iury Gregory Melo Ferreira
6986441e2c Add efi packages for ironic-agent
On IPA we are using efivar and efibootmgr, we already added the
packages on ipa-builder.
Adding the pacakges on diskimage-builder, so that people who use
it to build the images won't get into trouble.

Change-Id: I9ab6588f20302b4808b09dc060aced5fd267a3d2
2020-02-11 17:26:59 +01:00
Riccardo Pittau
cf7d39e4cd Allow python3 to be used in Debian
Debian default Python interpreter version is 2.7, but it's
possible to install a Python 3 interpreter from the base
repository.
With this change, if we set DIB_PYTHON_VERSION to 3, we install
the python3 package from base, with python3-libs, python3-pip and
python3-setuptools, and redefine python_path, effectively allowing
Python 3 interpreter to be used in Debian.
See a result of the job for building the ipa image here:
https://review.opendev.org/705773

Change-Id: Idabfa94c2bff6e0de6daa0866084d5db14d7dcb0
2020-02-07 18:10:19 +01:00
Tobias Henkel
bad433fa92
Fix cache-url -f
When there is a hashsum mismatch diskimage-builder forces downloads
with the -f switch of cache-url. This is currently broken because bash
escapes the quotes in curl_opts. This tricks curl trying to download
'no-cache' instead of the url. This can be fixed by using an array for
curl_opts which does the correct thing here.

Change-Id: Id9f1579dda9a3e0a2b08dd5faaeef0e2e580d419
2020-02-05 10:19:12 +01:00
Zuul
500e60dbf4 Merge "Enable possibility to select HWE kernel for Ubuntu minimal" 2020-02-04 09:00:47 +00:00
Antoine Musso
168127b60a dib-lint: test elements have README.rst file
Add a basic test to ensure that all elements have a README.rst file.
This way they will be exhaustively listed in the Sphinx documentation.

Add dummy README.rst for 'disable-selinux' and 'rpm-distro' elements.

Change-Id: Ia5252ddd89b5ae5c6e9a12a66ef10f912fd54da5
2020-01-20 11:43:43 +01:00
Fatih Degirmenci
e2af38e3a5 Enable possibility to select HWE kernel for Ubuntu minimal
Change-Id: I206a77590f575e472e31b4f867fd5fd35475542d
2020-01-17 10:47:37 +00:00
Carlos Goncalves
ae2be0b464 Fix Yum repositories and GPG keys for CentOS 8.1
CentOS 8.1 split repositories and GPG keys out into subpackages. This
broke DIB support for CentOS 8.

7e41cef418
26a0d73ced

Change-Id: If3de6efa6074e059dc9fdd47c7bdc19d26d4d7f2
2020-01-15 19:39:00 +01:00
Matthew Thode
87e7f7b869
allow building of Gentoo images for non-systemd profiles
Change-Id: I61e3ba391eaaf9b300f88b082c03116388401ba3
2019-12-20 17:51:55 -06:00
Mohammed Naser
6f1b51627f modprobe.d: use $TMP_MOUNT_PATH
The hook inside extra-data.d runs outside the chroot when
building the image which means that we need to prefix paths
inside the hook to avoid running things on the host.

We also run it with sudo because if we're running DIB not
as root, /etc is uid 0 and we'll get a permission denied.

Change-Id: I1838890fe124c84c879285a471bcc78fe47d6c23
2019-12-18 11:42:02 -05:00
Zuul
71e1bcbdb8 Merge "Install rng-tools in Red Hat family distro images" 2019-12-17 06:25:32 +00:00
Zuul
f6d32a684f Merge "Add arm64 based functional test" 2019-12-16 21:31:51 +00:00
Ian Wienand
082397a86a Add arm64 based functional test
An initial functional test for bionic/arm64 builds, put it in the new
arm64 check queue.

Change-Id: I5f8a047f41c6555da7211b758c55f7a87b3aa5d1
2019-12-16 10:51:01 +11:00
Carlos Goncalves
3ff7365ee8 Install rng-tools in Red Hat family distro images
Make sure rngd, a hardware RNG entropy gatherer daemon, is installed on
all DIB-built Red Hat family distro images. rngd comes installed by
default in a typical base installation as it's proven to help speed
things up.

Nova attaches the virtio-rng-pci device to VMs. virtio-rng-pci is a
device that provides feed random data. However, it is of little to no
use if the virtual machine is not configured to make use of given
device. That is where rngd can help by facilitating entropy to the pool
from virtio-rng-pci.

$ openstack image set --property hw_rng_model=virtio [...]
$ openstack flavor set --property hw_rng:allowed=True [...]

DIB-built minimal images do not come with rngd installed. This patch
makes sure the daemon is installed. Its systemd service comes already
enabled.

Change-Id: I34a989dbfc57d4c98113ac25c81dfb500945ff0a
2019-12-12 20:26:06 +00:00
Madhuri Kumari
46491eb1b5 Fix wrong URL in ironic-agent element
Change-Id: Iaa107094b1409f93b101b23cea2e874c60ea26f5
2019-12-12 16:39:48 +05:30
Zuul
361a751d36 Merge "Allow zypper repos to be overrideable" 2019-12-04 06:14:55 +00:00
Zuul
68b41854b2 Merge "Install ndisc6 package in element script" 2019-12-03 22:41:09 +00:00
Zuul
9b00377f31 Merge "Fix regex for mirror URL substitution" 2019-12-03 21:56:19 +00:00
Zuul
0b1eba25d5 Merge "Break retry loop on success in dhcp-all-interfaces" 2019-12-03 21:56:18 +00:00
Zuul
e3a904d4a6 Merge "Fix login.defs config for tumbleweed" 2019-12-03 21:56:16 +00:00
Carlos Goncalves
670df3326d Fix regex for mirror URL substitution
The base URL of EPEL repository installed by the epel-release package in
CentOS 8 at least now defaults to https.

The error seen when building an CentOS 8 image was:
"Error: Cannot find a valid baseurl for repo: epel"

This patch fixes it so that it will always match regardless of being
http or https.

Change-Id: I9ec5536ee72047c929a1ef6950ff4e9092842a4c
2019-12-03 19:13:18 +01:00
Harald Jensås
7948fee7e2 Install ndisc6 package in element script
The ndisc6 package is not yet available in EPEL 8.
See: https://bugzilla.redhat.com/show_bug.cgi?id=1779134

Until the package is available set the pkg-map to "" for
the ndisc6 package when distro is redhat and install the
package using || true in the element script instead so
that CentOS 8 build's do not fail because of the missing
package.

Once the package is in EPEL 8 this change can be reverted.

Related-Bug: #1754219
Change-Id: Icd4bad8852ce5ba40fb0e7b0d335191efbe88c67
2019-12-03 14:57:33 +01:00
Colleen Murphy
ad67aa213b Allow zypper repos to be overrideable
Without this change, operating system elements that use the
zypper-minimal element always must use download.opensuse.org as their
repository source. This change makes ZYPPER_REPOS overrideable, which
allows the user to create custom operating system elements that can use
private repositories as their source for base packages. For example,
with only this change, it is possible to create a sles-minimal element
that generates a SLE 15 SP1 image just by overriding DIB_ZYPPER_REPOS
and DIB_OPENSUSE_PATTERNS.

Change-Id: I46e40fbe4408d4204056a27b182b21213f1176ff
2019-12-02 16:25:53 -08:00
Colleen Murphy
4fb5a57b8a Fix login.defs config for tumbleweed
On openSUSE Tumbleweed, the login.defs config file was moved under
/usr[1]. This change allows the login.defs config change to work for
both old and new locations.

[1] https://build.opensuse.org/request/show/736424

Change-Id: Ia5eff5e7b0709836278361b1b8daa788619eff75
2019-12-02 16:21:45 -08:00
Hervé Rousseau
b91e212434 Break retry loop on success in dhcp-all-interfaces
If rdisc6 is available, a node using this element will loops until
DIB_DHCP_TIMEOUT is reached because of a missing 'break' when rdisc6
return code is 0.
This will mark the dhcp-interface@.service unit as failed (because it
has the same timeout) and not bring any network interface online.

Change-Id: I034dcda94d765f236950ebcbee36789f5bdc515f
Closes-Bug: #1854717
Signed-off-by: Hervé Rousseau <hroussea@cern.ch>
2019-12-02 15:50:20 +01:00