Commit Graph

918 Commits

Author SHA1 Message Date
Zuul
25f15f8965 Merge "Make centos reset-bls-entries behave the same as rhel" 2022-05-25 16:04:56 +00:00
Pavlo Shchelokovskyy
62626521ad Fix openssl example command in dynamic-login
the command had one error in it (missing one backslash)
and was rendered wrong, w/o any backslashes at all.

Change-Id: If187f645b818f47d10b602ccee12c29892a8d88d
2022-05-24 14:10:01 +03:00
Zuul
2f06cbc4eb Merge "Add Jammy functesting to dib" 2022-05-13 06:31:17 +00:00
Denys Mischenko
abc2524a5c Adopted dkms element to work on Ubuntu Jammy and nvidia drivers
Previously a module version was splitted from the module name:

  nvidia, 510.47.03, 5.4.0-109-generic, x86_64: installed

In Jammy it is now a part of the name:

  nvidia/510.47.03, 5.15.0-27-generic, x86_64: installed

Assuming the fact that it would be threatted as a path this change
doesn't brake anything which was working before. But at the same
time it allows to pass last step where dkms is requested to build all
modules.

Change-Id: Ic1bb2b45f9db906b64ca03ae5c4e05b2114f2a74
2022-05-12 10:20:25 +00:00
Cédric Jeanneret
6645f47db4 Ensure cloud-init is configured to generated host keys
It may happen a base image has an edited version of cloud-init
"cloud.cfg" that prevents the host keys to be generated.

While it didn't represent an issue with older releases of cloud-init,
starting cloud-init-22 this isn't true anymore.

Before that release, an sshd-keygen@.service was present and called by
sshd-keygen.target (which was called by sshd.service), and we ended up
with ssh host keys in any cases - either generated from cloud-init, or
generated by sshd-keygen.service.

But cloud-init-22 introduced an edition to the sshd-keygen.service,
making it check for the presence of cloud-init service, and preventing
this sshd-keygen to kick in this case.

So we'd better ensure cloud-init is able to generate the keys, else
we'll be in a bad state, since it's instructed to remove the ones
present.

Closes-Bug: #1971751
Change-Id: I37b2f3e9d57a86544ef14e74a4a927309c18bbf0
2022-05-06 11:10:54 +02:00
Clark Boylan
7b6247554e Add Jammy functesting to dib
This adds arm64 ubuntu-minimal Jammy functests and x86 ubuntu image
based Jammy functests. To make this happen we have to install
debootstrap from debian unstable on the functest nodes in order to get
access to a debootstrap that knows what jammy is.

As we ramp up Jammy support in our tools having good testing will be
helpful.

Change-Id: I1d1dc752ce176457d0656cbd50e27a2721ca9856
2022-05-04 13:03:50 -07:00
Zuul
d3ce32016a Merge "containerfile: update test to jammy" 2022-05-03 22:56:35 +00:00
Zuul
a055b59f8d Merge "centos: avoid head pipe failure" 2022-05-03 21:04:39 +00:00
Ian Wienand
999da87243 yum-minimal: workaround missing $releasedir variable
As described inline, work around missing $releasedir variables seen in
production builds.

Change-Id: Ia114f41d7a2fa86957235996f10a74adeecf767c
2022-05-03 15:47:55 +00:00
Zuul
45abd8ef95 Merge "Revert "Fallback to persistent netifs names with systemd"" 2022-05-03 00:26:20 +00:00
Steve Baker
8059a59583 Make centos reset-bls-entries behave the same as rhel
This makes 03-reset-bls-entries consistent with rhel so that the glob
match is *.conf, and a check is added to ensure that a rename is
actually required.

Change-Id: I4adff43cf7d4f31d939e6ddf37ac8d162ccd0db7
2022-04-29 13:12:24 +12:00
Zuul
555cecb670 Merge "Set machine-id to uninitialized to trigger first boot" 2022-04-28 01:35:54 +00:00
Zuul
737c9a8ff3 Merge "Fix dhcp-all-interfaces on debuntu systems" 2022-04-28 00:05:10 +00:00
Maksim Malchuk
b97dfb8fbd Revert "Fallback to persistent netifs names with systemd"
This reverts commit 8401290976.

We are reverting this because some users may want to use predictable
device names and may not even use Debian. However, after some
investigation we have found a couple of bugs in dhcp-all-interfaces on
Debuntu distros. The parent change corrects those bugs. Additionally new
Linux kernels emit "move" events to udev when interfaces are renamed to
their predictable name. Support this "move" in the dhcp-all-interfaces
udev rules. Making these changes appaers to produce functional images
for Debian users using predictable device names. If predictable device
names are not desired turning them off is straightforward and release
notes are updated to give users the info they need to do that outside of
this element.

Change-Id: I125f1a0c78a103b51bda961528c3e66c345bf604
Co-Authored-By: Clark Boylan <clark.boylan@gmail.com>
Signed-off-by: Maksim Malchuk <maksim.malchuk@gmail.com>
2022-04-27 16:29:58 +00:00
Clark Boylan
7b41f8160b Fix dhcp-all-interfaces on debuntu systems
There are two issues with dhcp-all-interfaces on debuntu interfaces
addressed here. First is the path to dhclient lease files is
/var/lib/dhcp not /var/lib/dhclient. Second there is a missing newline
in the ENI interface file which causes a parse error.

Change-Id: Ice83e0d49a4234301dc12daf828ba80fef414cdb
2022-04-26 13:37:13 -07:00
Zuul
638a94d8c9 Merge "Move reset-bls-entries to post-install" 2022-04-26 10:02:48 +00:00
Zuul
fb7db1b736 Merge "yum-minimal: clean up release package installs" 2022-04-26 09:54:34 +00:00
Ian Wienand
5bc9e87da1 centos: avoid head pipe failure
I just saw in the trace output of a failure

  > grep -o 'CentOS-.[^>]*GenericCloud-.[^>]*.qcow2'
  > sort -r
  > head -1
  sort: fflush failed: 'standard output': Broken pipe
  sort: write error

i.e. the "head -1" has exited after reading one line, but "sort -r"
still wants to write and thus has hit a pipe failure, and because we
run with "-o pipefail" this has halted the script.

This seems like it has been there more or less forever, maybe we just
got lucky hitting it now?  Anyway, we can work around this by using a
process substitution and passing the output of this into head, this
way we won't hit a pipe failure.

I also updated the fedora path as it does the same thing.

Change-Id: I44d97e5bb31702aacf396e0229329a2ef9c64f2f
2022-04-26 17:28:00 +10:00
Ian Wienand
28821fd283 containerfile: update test to jammy
We've not really been using the Focal containerfile, as we move
forward jammy is a better choice for keeping stable as we might find
some new users for it.

Also add binutils to bindep for native bullseye builds (see
Icb0e40827c9f8ac583fa143545e6bed9641bf613)

Change-Id: I22ebe2bbccaec34180e58996b21e47bfc4f36055
2022-04-22 14:48:18 +10:00
Zuul
adc40db9e9 Merge "Update gentoo python version to 3.9" 2022-04-21 13:56:25 +00:00
Steve Baker
e129fad7f8 Move reset-bls-entries to post-install
03-reset-bls-entries was previously a pre-install script to run after
the machine-id was set, but a new kernel may be installed during the
install phase, which will install another bls entry file with a
filename which differs from the machine-id.

This means this package installed bls file won't be updated when
grub2-mkconfig is called, resulting in incorrect kernel args and boot
device in the entry file that will get booted by default.

By fixing the filenames after the new kernel is installed,
grub2-mkconfig will update the bls file that actually gets used on
boot.

Change-Id: I653bef9638e38ded68458fd40d90e30e5206caad
2022-04-21 10:13:10 +12:00
Steve Baker
147641fc3e Set machine-id to uninitialized to trigger first boot
According to the systemd documentation[1], if /etc/machine-id is empty
it will be populated with a unique value, but not in a way which
triggers an actual first boot event (running units with
ConditionFirstBoot=yes set)

This change writes "uninitialized" to /etc/machine-id to ensure that
systemd-firstboot.service actually runs, and other units can use
first-boot-complete.target as a dependency to trigger on first boot.

Since /var/lib/dbus/machine-id is sometimes a symlink to
/etc/machine-id, it is truncated before writing to /etc/machine-id.

On older versions of systemd before first boot semantics were
formalised, any non-uuid value will trigger a new machine-id to be
generated, so "uninitialized" also works.

[1] https://www.freedesktop.org/software/systemd/man/machine-id.html#First%20Boot%20Semantics

Change-Id: I77c35e51a3da2e8a6b5a2c80d033a159b303c9af
2022-04-21 09:39:42 +12:00
Zuul
65569921bf Merge "Move grub-install to the end, and skip for partition images" 2022-04-20 15:14:05 +00:00
Zuul
ab75f1d8a2 Merge "Add interpolation note for dynamic-login password" 2022-04-20 06:04:38 +00:00
Ian Wienand
27903f90e2 yum-minimal: clean up release package installs
This started a long way from here, when I noticed that "top" on centos
9-stream images wasn't working because ncurses-base wasn't installed.

This led me to the extant install of bash/glibc/ncurses-libs from
Iecf7f7e4c992bb23437b6461cdd04cdca96aafa6.  However it didn't really
explain why these are brought in here.

Reading further it became clearer that over the years of distribution
additions, Fedora updates, etc. this has grown into a bit of a mess.

Refactor the release package installs into a more logical flow,
pulling out checks/comments for Fedora's of ancient history, etc.
Remove the 9-stream package installs; this isn't the place for them,
and the should be brought in by the base packages.

Ultimately, this is intendend to a be a no-op refactor.

Change-Id: Ie7d9a6497d0d20a3303ec0da3d0668c74efa2c3d
2022-04-20 09:11:16 +10:00
Ian Wienand
4cb3346fec source-repositories : use explicit sudo/-C args when in REPO_DEST
The recent git ownership-checking changes (see related bug for full
details) mean we can not run git in non-owned directories.

We have a couple of cases here where we have done a "pushd" to work in
the REPO_DEST context; this is the destination directory that is
inside the chroot so needs to be operated on as "root" (via sudo
calls).  This certainly makes sense -- but given the new way of things
it can hide what context each call is working in, which is now very
important.  Previously this worked because you could read it; now it's
doing the UID check too, calls in here without sudo now fail.

Remvoe the pushd's and make every call that works in REPO_DEST
explicit with -C, and add sudo calls around it.

Change-Id: Id1f6bd94c9c77ef6ab2b562a7e0bc48f749c58ac
Related-Bug: https://bugs.launchpad.net/devstack/+bug/1968798
2022-04-14 16:53:37 +00:00
Dr. Jens Harbott
4b565f1ddb Update gentoo python version to 3.9
Seems gentoo has updated their defaults so builds are failing with
py3.8.

Change-Id: I01747e96215cfb2e8648e168f823344bcdbcb4e7
2022-04-12 09:26:28 +02:00
Zuul
709d18fd42 Merge "Update fedora element testing to F35" 2022-04-11 05:15:52 +00:00
Marcin Juszkiewicz
64bdd24e4e CentOS Stream 9 has EPEL now
Change-Id: I13131f8f4b668d01661a57d918da1bfdb69a0584
2022-04-07 09:01:13 +02:00
Pawel Konczalski
7f5d6538c1 Add interpolation note for dynamic-login password
Change-Id: I416c60347679dc648914e7c1b02761882bae8d4a
2022-03-25 11:59:44 +01:00
Steve Baker
9987d0911a Move grub-install to the end, and skip for partition images
The bootloader element installs the grub bootloader for whole-disk
images, but it also correctly sets values in /etc/default/grub and BLS
entries.

This value setting is useful even if the bootloader isn't installed.
For example, the overcloud-full partition image benefits from a
correct /etc/default/grub and BLS entries which ironic-python-agent
will use when it installs grub on the disk during baremetal deploy.

This change moves the actual grub install to the end of the script,
and if there is no $DIB_BLOCK_DEVICE set then install is skipped.

This allows overcloud-full to use the bootloader element instead of
the grub2 element, so the correct grub defaults are set on centos9,
including the correct root device on centos9.

Change-Id: I8cb34914bbbfa05521bbb71cc6637368b980358f
2022-03-21 08:43:48 +13:00
Benedikt Loeffler
94fab70e58 Use https for downloading ubuntu images
Change-Id: I8d317582d454df2c418a7d3a37aea38dc7570391
2022-03-16 13:43:30 +01:00
Michal Nasiadka
7d782ae1c9 containerfile: Add support for setting network driver
Change-Id: Ia885237406bf4c7b9d49b349f374558ae746401f
2022-03-15 13:18:11 +00:00
Zuul
bce7af6a29 Merge "Handle btrfs root subvolume for fedora extract-image" 2022-03-14 18:54:12 +00:00
Ian Wienand
4f089b33c5 Update fedora element testing to F35
This reverts the mirror removal in
I817b412b7f06523df635e8b16111bc1081b40f66 and updates the test to F35,
which is mirrored.

Change-Id: I00d24690f57bedd3fc5ebbc18de0ed874ad1e4ef
2022-03-11 14:53:00 +11:00
Michal Nasiadka
a02cb9ff74 containerfile: add support for Docker
In some build environments Docker is already installed - and adding
podman is not an option.  Add a new variable to toggle this, and
rename the now incorrectly titled DIB_CONTAINERFILE_PODMAN_ROOT to
just ...RUNTIME_ROOT to match.

Change-Id: I677e4f491b40360dceabdf4f2a9e64c7cb493dc7
2022-03-11 14:04:52 +11:00
Steve Baker
7de5bc6fa3 Handle btrfs root subvolume for fedora extract-image
This adds a check for the root device having filesystem type btrfs,
and when it is assume there is a subvolume called "root". This fixes
extract-image when using Fedora-Cloud-Base btrfs images.

This should be sufficient until there is another btrfs base image with
a different subvolume layout.

Change-Id: Ib18979090585ba92566e523951b521b9d902fcb7
2022-03-11 15:48:03 +13:00
Steve Baker
41c21e91db Revert "Revert "Detect boot and EFI partitions in extract-image""
This change is proposed again, avoiding lsblk features missing from
older distros:
- lsblk is avoided entirely for a whole-disk image with a single
  partition, which would be the majority of old image building jobs
- Field PARTTYPENAME not available on the lsblk in CentOS-8, instead
  rely on the GUID being correct for EFI partitions
- Argument --output-all not available on the lsblk in CentOS-7, this
  is just for logging debug, so can be removed

This reverts commit b06bac734c.

Change-Id: Ib0d4e7751fd968511fc7f672d524e58d1488ae11
2022-02-25 14:52:45 +13:00
Zuul
0e5986e9fb Merge "Revert "Detect boot and EFI partitions in extract-image"" 2022-02-24 16:45:31 +00:00
Riccardo Pittau
b06bac734c Revert "Detect boot and EFI partitions in extract-image"
This reverts commit 0630b3cb69.

Reason for revert: breaks compatibility with CentOS Stream 8, lsblk does not have PARTTYPENAME until version 2.35 and CS8 has version 2.32.1 installed

Change-Id: I7fc0e76f0eeb8594d8a0d57629b2c67526b961ad
2022-02-24 13:42:50 +00:00
Zuul
6a3b8d9482 Merge "Detect boot and EFI partitions in extract-image" 2022-02-24 10:04:49 +00:00
Zuul
6382c693a1 Merge "rhel: work around RHEL-9 BLS issues" 2022-02-24 10:04:47 +00:00
Clark Boylan
95c3c54fed Force use of NetworkManager with glean on Rocky Linux
Rocky Linux is very similar to CentOS 8. CentOS 8 required and forced
NetworkManager with glean so we update dib to do the same for Rocky.

Change-Id: I145e57d61059c2f34dc2d4810e83809b71c6aade
2022-02-23 12:43:06 -08:00
Steve Baker
0630b3cb69 Detect boot and EFI partitions in extract-image
RHEL-9 base images are whole-disk images with the /boot/efi partition
correctly set up for EFI Secure Boot. This doesn't work with
extract-image because it only mounts the root partition, leaving
/boot/efi empty even though grub2-efi & shim packages are "installed".

This change mounts discovered partitions to mnt/boot, mnt/boot/efi so
all content can be extracted from the image.

Partition detection is done by reading block device attributes and
matching on Boot Loader Specification[1] UIDs or labels as observed in
supported base images.

[1] https://systemd.io/BOOT_LOADER_SPECIFICATION/
Change-Id: I8487002a18ae6ca98609ab68d92ae9173a2b864f
2022-02-23 15:28:32 +13:00
Zuul
4a2b63e7f9 Merge "bootloader: clean up EFI checking" 2022-02-22 09:33:07 +00:00
Zuul
3e4e66de16 Merge "Add rocky support to the epel element" 2022-02-22 07:40:13 +00:00
Zuul
6b3af23854 Merge "bootloader: fix arm64 install path" 2022-02-22 03:17:20 +00:00
Steve Baker
4376f66407 rhel: work around RHEL-9 BLS issues
Similar to the CentOS-9-Stream fix [1] this change renames the default
BLS entry to match the current machine-id so that grub2-mkconfig calls
will refresh the kernel options.

However there is an additional issue with the rhel-9 base image. It is
unique in having a dedicated boot partition, so the path to the kernel
and initramfs don't include /boot. This results in an unbootable image
when /boot is a directory of the root partition.

These paths do not get corrected by calling grub2-mkconfig, so this
change performs a sed on the paths to fix them for a root partition
/boot.

[1] I327f5e7a95e47905c01138c8c4483f3f03e8efff
Change-Id: I37a1d310e1854f4a49725e355d484e456ea4fc7a
2022-02-22 13:43:18 +13:00
Ian Wienand
7f98cfbcf7 bootloader: clean up EFI checking
The check removed here came in with
I4481b43e4a8fe4144be9c7eb9d9c618bbb2df21e a long time ago.  At that
time we were not building EFI images, and were building i386 images;
both of which are now untrue.

We can simplify this now by merging it into the gpt/mbr path.  If we
are in there we know that we should set --target=i386-pc for BIOS
boot.  For sanity check that we are x86 in this path -- PPC is handled
separately (although it's probably bit-rotted) and ARM64 is EFI.

Change-Id: Ie9839c9adc642b0dd688bced3faa46e9314e9799
Co-Authored-By: Clark Boylan <clark.boylan@gmail.com>
2022-02-22 11:03:14 +11:00
Clark Boylan
7d77f4fab2 Add rocky support to the epel element
OpenDev relies on the epel role to configure the epel repository for our
image builds. Specifically we need epel to pull in haveged. Update the
epel role to recognize rocky and configure it properly.

Change-Id: I968d4702ef39590e972b782a09e18a5db40703ad
2022-02-21 14:38:30 -08:00