Commit Graph

4055 Commits

Author SHA1 Message Date
Ian Wienand
3833c2e59c containerfile: fix tar extraction
Ic68e8c5b839cbc2852326747c68ef89f630f26a3 removed the sudo from the
tar extraction here, meaning that production is failing to create the
chroot.  This is hidden in testing because
DIB_CONTAINERFILE_PODMAN_ROOT is set.  Make the sudo here
unconditional.

Change-Id: I6e36e3fc65981f85fad12ea2cd10780fde9c37da
2021-11-10 11:42:49 +11:00
Ian Wienand
a1debc726d centos 9-stream: make non-voting for mirror issues
Change-Id: I9dc5015e4f3f6a710329e024913abe822cac3ee1
2021-11-10 11:42:40 +11:00
Zuul
eb7782b78f Merge "tests: remove debootstrap install" 2021-11-08 04:59:04 +00:00
Ian Wienand
f75fed076e fedora-container: update to Fedora 35
Support Fedora 35 in this element, and update the default

Depends-On: https://review.opendev.org/c/zuul/nodepool/+/815766
Change-Id: I2a9a8648158e1e4b1a071e20266fb8d125da67ac
2021-11-03 06:26:56 +11:00
Zuul
311621a094 Merge "Add support for CentOS Stream 9 in DIB" 2021-10-29 09:55:30 +00:00
Alfredo Moralejo
cdff9045c0 Add support for CentOS Stream 9 in DIB
CentOS Stream 9 is close to be released, and official mirrors are
already poplated. This patch is adding support to centos-minimal in CS9.

Also enable centos-minimal/[8,9]-stream-build-succeeds tests.

This patch is being tested together with [1] to apply following list of elements:

 vm centos-minimal simple-init growroot nodepool-base openstack-repos infra-package-needs

[1] https://review.opendev.org/c/openstack/project-config/+/811442

Change-Id: Iecf7f7e4c992bb23437b6461cdd04cdca96aafa6
2021-10-27 13:38:14 +02:00
Ian Wienand
57b7d2bceb dracut-regenerate: drop Python 2 packages
We don't support any Python 2 guest systems, drop the Python 2 bits
from package-installs.

Change-Id: Ica5c71c2926968d438ac1d0b0f16981cadf0bab1
2021-10-27 11:11:50 +11:00
Ian Wienand
0f987ca4b6 tests: remove debootstrap install
This is not needed on the host any more; it's part of the
nodepool-builder container.

Change-Id: I56564d58fe6193c21294aacbf9f769bba2142ec7
2021-10-27 11:01:55 +11:00
Zuul
6a42cad8d5 Merge "Switch ARM64 testing to bullseye" 2021-10-26 08:54:55 +00:00
Dr. Jens Harbott
4bd8258fbf Fix bootloader installation for gentoo
The if/elif block added in [0] doesn't work for gentoo, let's hope
that we can get along with an easy fix.

[0] https://review.opendev.org/c/openstack/diskimage-builder/+/804000
Signed-off-by: Dr. Jens Harbott <harbott@osism.tech>
Change-Id: I543e04d2d7efea3e718bae31aa1cc4767bd359f8
2021-10-21 10:20:58 +02:00
Ian Wienand
edbf1fa45b Switch ARM64 testing to bullseye
This more closely matches the nodepool-builder container, which is
Bullseye based.

Refactor to remove unnecessary abstract job.

Change-Id: I34822608f19e1ce9ef781034ff831d6359ed8e15
2021-10-21 16:00:05 +11:00
Sagi Shnaidman
d5a01519c6 Update centos element for 9-stream
This adds 9-stream support to the centos element.

See https://review.opendev.org/q/topic:cs9 for related patches.

Change-Id: Ib80fbd21edb77c25764eff2c0d66e55bde7a90af
2021-10-20 09:39:27 +11:00
Ian Wienand
1a5c563920 Run functional tests on Debian Bullseye
We need to update the base reference platform we perform the
functional tests on.  Debian bullseye seems like the best choice -- it
is recent enough to last for a while, and will match the
nodepool-builder container environment.

Depends-On: https://review.opendev.org/c/zuul/zuul-jobs/+/814088
Change-Id: Ic68e8c5b839cbc2852326747c68ef89f630f26a3
2021-10-20 09:39:27 +11:00
Ian Wienand
c41a2b7f84 Simplify functests job
At one point we had an array of functest jobs; we were testing
building on trusty, bionic, maybe centos, with python2 and python3.
The only thing left of all these combos these days is "bionic-python3"
(that's where this naming comes from).

We can remove the levels of abstraction now and just have the one job
to avoid confusion.

Change-Id: Id37d62a17f9b7f6dc6dc35585c29eddd435ce913
2021-10-20 09:39:27 +11:00
Ian Wienand
1ea71b3d69 Remove extras job, put gentoo job in gate
The only job left in the "extras" functional test is for gentoo.  We
might as well just drop this and put the gentoo devstack test in the
gate -- in terms of resources it's a 1:1 swap now.

Cleanup the ordering of the check queue list

Change-Id: I2af8f9235131cd0cce33e67c8d0d05c3b357320d
2021-10-20 09:39:27 +11:00
Ian Wienand
a6da91f753 functests: drop minimal tests in the gate
We test the minimal builds in the devstack based end-to-end tests.
There is no need to also run these functional tests; they test *less*
than the full build/upload/boot testing done in the devstack test.

We consolidate the separate "image" based job

Change-Id: I73ca83bedb4b0c40af5209f9c93a0e657c152591
2021-10-20 09:39:27 +11:00
Ian Wienand
55efeb575e centos7 : drop functional testing
This element has been deprecated since Sep 2019.  At this point we
don't need to keep gate testing it.

Change-Id: I2851b9be93effac30ca7c97ef8fd17e8641eeffa
2021-10-20 09:39:27 +11:00
Ian Wienand
f1f717ed07 functests: drop apt-sources
This builds ubuntu-minimal to test apt-sources, an element that
appears unused.  We don't need this as a gate test.

Change-Id: I5310c0a38a19a7c00764c1057b0a26aafdbacccc
2021-10-20 09:39:27 +11:00
Ian Wienand
f3a3153390 ubuntu: add Focal test
Switch this to testing bionic and focal, dropping xenial

Change-Id: Ifba6800ad84e504d248de8389e49a31f22c4554c
2021-10-20 09:39:27 +11:00
Ian Wienand
2f1e9320bd ubuntu-systemd-container: deprecate and remove jobs
I'm not aware this element is used/was ever used.  It hasn't ever been
updated to Focal.  To reduce our testing footprint remove this test,
and note in the element its probably broken.

Change-Id: I17cd3b13948287fe78990cfbe16a22919a329ba9
2021-10-20 09:39:27 +11:00
Ian Wienand
d5fa64325f Revert "Allowing ubuntu element use local image"
This reverts commit 1f4fb1d7a5.

This unfortunately wasn't actually tested.  Because the image-based
tests run sequentially, a prior failure in the centos-8 job meant the
ubuntu job never ran.

This is failing with

 10-cache-ubuntu-tarball: line 28: DIB_LOCAL_IMAGE: unbound variable

There is also a seemingly unused variable DIB_IMAGE_LOCAL_FILE; I'm
not sure what this is doing.

For now revert, and it can be re-proposed with appropriate testing.

Change-Id: I0f3897c90dc863ee04c3295b9cb094f02d8658e3
2021-10-20 09:39:27 +11:00
Ian Wienand
2d5981d673 epel: match replacement better
It looks like upstream have changed this line to "download.example",
breaking our subsitution.  Let's do a generic match.

Change-Id: I8e443022a5f239b98ccefe73a9abf8cf259dc8e9
2021-10-20 09:39:27 +11:00
Ian Wienand
2b85ddc669 Remove py35 tox jobs
At this point py35 practically means Xenial, and we are not interested
in having that as a build platform.  Drop the py35 jobs.

Update python setup.cfg metadata; we are building on 3.9 so add that.

Change-Id: I981f0f67a6fd809af1ab70934358dc3404890f35
2021-10-20 09:38:47 +11:00
Zuul
6e83df4d40 Merge "Fix cron not installed in debian" 2021-10-06 13:23:00 +00:00
yatinkarel
8b4ad6dae6 Drop lower version requirement for networkx
With [1] to fix lower constraints job, networkx
requirement was bumped to 2.4.0, but since diskimage-builder
is branchless and to get it work with stable/train upper-constraints
for networkx i.e 2.3.0 can't bump to 2.4.0.
Also networkx-2.3.0 has dep on decorator>=4.3.0
so actual decorator issue with 3.4.0 shouldn't be there.

[1] https://review.opendev.org/c/openstack/diskimage-builder/+/811048
[2] https://github.com/networkx/networkx/blob/networkx-2.3/requirements/default.txt

Closes-Bug: #1945926
Change-Id: I93a2826019d9bb0d004f7daa661fdd1ca4269331
2021-10-05 12:55:43 +05:30
Zuul
82aa8c516d Merge "Add DIB_YUM_REPO_PACKAGE as an alternative to DIB_YUM_REPO_CONF" 2021-10-04 06:51:31 +00:00
Zuul
70378de688 Merge "Add policycoreutils package mappings for RHEL/Centos 9" 2021-10-04 06:51:29 +00:00
Zuul
0e93b853bf Merge "RHEL/Centos 9 does not have package grub2-efi-x64-modules" 2021-10-04 06:51:27 +00:00
Zuul
0a9c0b6ab8 Merge "Support grubby and the Bootloader Spec" 2021-10-04 06:46:29 +00:00
Zuul
a4615a2977 Merge "Move grubenv to EFI dir" 2021-10-04 06:46:28 +00:00
Zuul
175c5c4da6 Merge "Allowing ubuntu element use local image" 2021-10-01 15:31:07 +00:00
Marco Vaschetto
1f4fb1d7a5 Allowing ubuntu element use local image
Patch allow to set path for local image source,
instead download latest or use the cached image.
This permit to build image also in environment without internet access.

Change-Id: I9422e21c5d0445e31d5a7258aa7310b20e39b929
2021-09-27 20:56:06 +02:00
Steve Baker
517e7eca95 Fix lower constraints
decorator-3.4.0 now errors on setup due to using use_2to3, and
networkx-1.1.0 attempts to import fractions.gcd, which is math.gcd
from python-3.5 onwards.

Change-Id: I4f4ddebb079a95708be3dd76086b039db18d5948
2021-09-27 16:03:22 +13:00
Zuul
0aa6e24286 Merge "Fix debian-minimal security repos" 2021-09-14 23:16:28 +00:00
Steve Baker
296c81b9ca Add DIB_YUM_REPO_PACKAGE as an alternative to DIB_YUM_REPO_CONF
A custom yum repository can now be configured by defining
`DIB_YUM_REPO_PACKAGE` as a yum available package or a URL to an rpm file.
This package can install repo files with any associated keys and
certificates.

A good example of such a package upstream is rdo-release[1] which
includes multiple repo files, the repo keys, and a root certificate.
This makes these repos impractical to install via DIB_YUM_REPO_CONF.

Downstream, repo packages like this a frequently used to bootstrap
development builds of RHEL with development repos.

[1] https://www.rdoproject.org/repos/rdo-release.rpm

Change-Id: I2832e723998c9bd7635cdf7541a4c20eff6294d2
2021-09-13 09:32:53 +12:00
Steve Baker
017ba12eb0 Add policycoreutils package mappings for RHEL/Centos 9
Change-Id: I69f0bc39b3ff4759bb66cea76112901ea8afff68
2021-09-13 09:32:53 +12:00
Steve Baker
fd63fe6999 RHEL/Centos 9 does not have package grub2-efi-x64-modules
Change-Id: Ida37eb500b7f331fd35f1b76365940c3b95ec83e
2021-09-13 09:32:53 +12:00
Steve Baker
97f940ace5 Support grubby and the Bootloader Spec
Fedora 30 and RHEL-8.2 onwards support the Bootloader Spec and use grubby
to manage kernel menu entries and kernel arguments.
https://fedoraproject.org/wiki/Changes/BootLoaderSpecByDefault

This change detects if this is a BLS enabled environment, and uses
grubby to set kernel arguments on all kernel entries if it is.

Change-Id: I2701260d54cf6bc79f1ac765b512d99d799e8c43
2021-09-13 09:32:53 +12:00
Steve Baker
b01aac9715 Move grubenv to EFI dir
If the grubenv is regenerated, its changes won't be available to UEFI
boot systems unless the changed grubenv is copied to the EFI
directory.

This change copies the grubenv to the EFI directory when the grub.cfg
is copied.

Change-Id: I512502117a6bf1e6122fdfd8965ca488b4a5bae4
2021-09-13 09:32:29 +12:00
Zuul
3c3ef6e32b Merge "Check and remove existing image interface configurations" 2021-09-10 15:01:28 +00:00
Zuul
48ef1c4d96 Merge "simple-init: allow disabling DHCP fallback" 2021-09-10 14:54:03 +00:00
Zuul
3eae88795d Merge "simple-init: support installing Glean from packages" 2021-09-10 14:54:00 +00:00
Riccardo Pittau
6ed7f3b139 Fix debian-minimal security repos
Debian stable security repos is now stable-security, as well as other
versions.

Move the Debian bullseye job from experimental to non-voting check.

Change-Id: I451cacda6573727de9448b5857bed5181850b4ad
2021-09-10 11:40:48 +02:00
Zuul
f03c50b4a1 Merge "Fix doc typo" 2021-09-09 00:24:00 +00:00
Zuul
8250b33d68 Merge "yum-minimal: use DNF tools on host" 2021-09-09 00:09:56 +00:00
Michal Arbet
db1409f550 Fix cron not installed in debian
This patch adds cron to debian package-installs
as cron should be included in OS.

Change-Id: I50140dc98c240911e769acf2b3c11958ffad5494
2021-09-01 23:49:27 +02:00
Piotr Parczewski
169f755165 Fix doc typo
Change-Id: I5b2729fa874e8e19a9f0dc7b1aed2ce7f5090c97
2021-09-01 19:58:52 +02:00
Ian Wienand
5f47584196 yum-minimal: use DNF tools on host
The latest Debian bullseye release doesn't provide yum any more, only
DNF.  This breaks the minimal builds that are using on-host yum tools
to start the chroot.  Probe for yumdownloader, and if it's not there,
use DNF.

Note this requires "dnf download" which may not be packaged.  See
I21cfbd3935e48be4b92591ea36c7eed301230753 for a sample work-around
that installs this plugin in the nodepool-builder container.

Change-Id: Ia7f1e4d115cc67c378d865d91af94a07b8cdc6cc
2021-08-30 14:14:32 +10:00
Steve Baker
59420be677 Use non-greedy modifier for SUBRELEASE grep
Some fedora mirrors are serving image icon html, and this grep is
over-matching on those mirrors.

Change-Id: Ibd737f44a8d2eee1902b6c5363c61d591feb75c9
2021-08-23 14:12:17 +12:00
Xinliang Liu
a6ee4d0c21 Introduce openEuler distro
Add openeuler-minimal element and add CI functional tests for both
x86_64 and arm64.

OpenEuler is an open source community driven YUM/DNF distro like
Fedora. It references Fedora and CentOS a lot for the rpm packages
building. So somewhat it can be treated as a redhat family distro
and reuse the YUM/DNF related elements to help build openEuler images.

For more info about openEuler, see: https://openeuler.org/en

Depends-On: https://review.opendev.org/c/zuul/zuul-jobs/+/803413
Change-Id: I3e06e49b524364c3a4edeba8bce7a8c06b9c7b76
2021-08-04 03:06:55 +00:00