Commit Graph

979 Commits

Author SHA1 Message Date
Zuul
174089a6a5 Merge "Reduce thin pool by one more extent" 2023-01-27 02:34:56 +00:00
Ian Wienand
2c4d230d7a
cache-url: Give up on curl install for Redhat platforms
The RedHat platforms vary if they come pre-installed with curl or
curl-minimal.  For example, Fedora 37 container images have curl, and
centos 9-stream and Rocky images have curl-minimal.  If you try and
install curl when curl-minimal is installed, you get an error, and
vice-versa.

Unfortunately package-installs can't really sort this out; we're just
passing a package list to the system package manager.  We don't have a
way to say "the curl OR curl-minimal package is fine".  As this breaks
builds and is such a common dependency that it's already there, let's
just add a note that curl is required and blank out the package-map.

Change-Id: I9ccebe2dbf3a8682dab60c2070c5f78849e01446
2023-01-20 17:29:13 +11:00
Steve Baker
7e2a2aa027 Reduce thin pool by one more extent
The previous commit was tested on 2TB without issue, but testing on a
very small volume (80GB) resulted in the thin pool lvextend failing
for being one extent too large.

This change reduces the pool size by one extent.

Change-Id: I7ca002783f8f15946bc84af95eecaa097e70aaf1
Related: rhbz#2149586
2022-12-22 17:24:03 +13:00
Zuul
4a161d0cde Merge "Add variable for check installing python3 in yum element" 2022-12-21 05:16:35 +00:00
Zuul
9d0ba50494 Merge "Grow thin pool metadata by 1GiB" 2022-12-21 04:20:44 +00:00
Steve Baker
00ca126287 Grow thin pool metadata by 1GiB
An LVM thin pool has an associated metadata volume, and it can be
assumed that the size of this volume on the image is minimized for
distribution.

This change grows the metadata volume by 1GiB, which is recommended[1] as
a reasonable default. This fixes a specific issue with the metadata
volume being exausted when growing into a 2TB drive.

Other minor changes include:
- Human readable printed values have switched to GiB, MiB, KiB, B
- Growth percentage volumes are adjusted down to not over-provision
  the thin volume

[1] https://access.redhat.com/solutions/6318131

Change-Id: I1dd6dd932bb5f5d9adac9b78a026569165bd4ea9
Resolves: rhbz#2149586
2022-12-19 13:40:04 +13:00
Frikin Evgenii
a9edef6548 Add variable for check installing python3 in yum element
If client have not internet or have some limitation, such as firewall/proxy/etc. this step will stop build image with error. Client must have possible override of URL for pass this step.

Change-Id: Iafe3283665a437d0a9cf83a93ff66c0613310b69
2022-12-16 03:51:10 +00:00
Zuul
b706802a1c Merge "Fix issue in extract image" 2022-12-16 03:43:14 +00:00
Ian Wienand
41aa936fa2
tox jobs: pin to correct nodesets; use host networking for containerfile
These must have broken when we switched the base nodes to Jammy.
Update to use compatible versions of distros.

We need to squish another gate-breaking change in here to update the
containerfile "podman build" calls to use "--network host".  We added
this with Ia885237406bf4c7b9d49b349f374558ae746401f and the only
external user I can find is kayobe, which is setting this anyway.

I honestly haven't 100% root-caused what changed to require this; the
last time our containerfile jobs ran and worked has unfortunately been
purged so I can't compare versions to try and pinpoint something;
i.e. this may be a podman bug or feature.  At first I thought it
related to the networking plugin package from the Depends-On (which is
still useful for the right packages) but that didn't help get the
bridge networking working.

Depends-On: https://review.opendev.org/c/zuul/nodepool/+/867590
Change-Id: I23f091654cb212e8bdd908664b262de9bfe98cef
2022-12-16 09:52:46 +11:00
jgupta
f2e7cd1307 Fix issue in extract image
The problem lays with the 'extract-image' script as
it is using lsblk commands to extract image's partition
(find out root/efi/boot, lines:100-102) but the output
is empty inside a container.

lsblk gives empty output for FSTYPE, LABEL, GUID..
the fix is to use blkid.

Closes-Bug: 1974350
Change-Id: I3b460c6dd9caa519c55327c5bd4b7e4585a8bd22
2022-11-02 14:21:16 +00:00
Rafal Lewandowski
4ff8721a66 Added example configuration
Change-Id: Icebe0e2f43f4a96d3c1fa464c6f2dc3da3be65c5
2022-10-20 10:00:18 +02:00
Zuul
ea7de37ccb Merge "Added cloud-init growpart element" 2022-10-13 20:41:55 +00:00
Rafal Lewandowski
b73048f98d Added cloud-init growpart element
Added growpart element. It allows for growing specific partitions
during the deployment, which will result in less post deploy actions
needed for the server to be ready for use.

Change-Id: I6519fba3e8f1d078b99d3c03f2ac85f7b6e37d8a
2022-10-11 16:58:26 +02:00
Zuul
3e72fdbf77 Merge "Fix wrong yum.conf name of CentOS 9 Stream" 2022-10-05 14:50:07 +00:00
Ian Wienand
d321b08f4d
Add Rocky 9 ARM64 functional test
This will test if we can get a Rocky 9 ARM64 image

Needed-By: https://review.opendev.org/c/openstack/project-config/+/858554
Change-Id: I3e589e4838cea489e11719b5f7fce82bf0d978e8
2022-09-21 15:08:20 +10:00
Ian Wienand
2a25db9ee6
Start running dib-lint again
A recent change that didn't fail with hard-tabs made me realise we're
not running tox -e pep8 ... which means we're not running dib-lint
which should find this (and other things).

I couldn't pinpoint when this happened; maybe job config was never in
this repo.

Anyway, move the pylint and dib-lint/flake8 testing to the now
standard "linters" and update the linting job to
openstack-tox-linters.

It looks like pylint is very lightly used (came in with
I7e24d8348db3aef79e1395d12692199a1f80161a and we've never expanded any
testing).  Leave this alone for now, but probably it is not important
any more.

This revealed some issues; updated flake8
(Iaa19c36f8cab8482a01f764c588375db8e7d8be3) found some spacing issues
with keywords and an update to elrepo to match our standard bash
flags.

Change-Id: I45bf108c467f7c8190ca252e6c48450c2622aaf8
2022-09-21 07:56:05 +10:00
Clark Boylan
793e16b76c Install Fedora ifcfg NM compat package
Starting with Fedora 36 the NetworkManager package no longer includes
ifcfg support by default. You need an additional package
"NetworkManager-initscripts-ifcfg-rh" to pull in the compatibility
plugin. Glean's support for Fedora relies on this compatibility system
so we install this package via the simple-init element package deps.

Change-Id: I76ac39b8dedcb1c5bc4595aedc0a732c99c8721e
2022-09-20 08:58:37 -07:00
Zuul
9ce3358fde Merge "changed release check logic in lvm element" 2022-09-16 12:30:03 +00:00
Zuul
0c323755bc Merge "added elrepo element" 2022-09-16 09:50:21 +00:00
Rafal Lewandowski
35c0c2c6db added elrepo element
Change-Id: I7bc144afa96f45122857ff634c8c19c1fd759450
2022-09-02 15:09:08 +02:00
Rafal Lewandowski
f9287fe2ae changed release check logic in lvm element
Change-Id: I83b3c2dad3b0d6006ae23307ae7a8a83b12806e7
2022-09-02 12:37:15 +02:00
Zuul
4a1f684e5f Merge "Add subscription-manager repo names for RHEL-9" 2022-09-02 07:35:17 +00:00
Steve Baker
f61548d863 Add thin provisioning support to growvols
This change enhances the growvols script to support all volumes being
backed by one thin provisioning pool.

If a pool is detected, the following occurs:
- validation to confirm every volume is backed by the pool
- only the pool is extended into the new partition
- volumes are extended by the same amount as the non thin-provisioned
  case

This results in no volumes being over-provisioned, so
out-of-space behaviour will be the same as the non thin-provisioned
case.

This change also switches to using /dev/mapper device mapper paths for
volume block devices, since that is the only path the thin pool is
mapped to.

Change-Id: I96085fc889e72c942cfef7e3acb6f6cd73f606dd
2022-08-24 10:14:26 +12:00
Ian Wienand
467bbf36dd rocky : create machine-id in 9
It turns out we do need to create the machine-id for the same reason
as on 8.  This was being hidden by the bootloader choosing the root
disk label from the host (see the dependent change).

Change I3b518802d681b888916a5cc6a3dcf7e1b537da1e has modified the
testing to use a different root-disk label, which should help catch
this in the fututure.

Depends-On: https://review.opendev.org/c/zuul/nodepool/+/853574
Change-Id: I64de66cac25fd2e051780fb4812e075c647eb76e
2022-08-19 18:54:43 +10:00
Vanou Ishii
e8c7787ec8 Fix wrong yum.conf name of CentOS 9 Stream
This commit fixes wrong name of yum.conf files
on CentOS 9 Stream in centos element to correct ones.

Change-Id: I25f0661fa79b7bc8ac1b8e3b2831a413c4161d1d
2022-08-08 22:53:21 +09:00
Neil Hanlon
6b42cb23ee
Add Rockylinux 9 build configuration and update jobs for 8 and 9
* Add note in Rocky Linux 9 containerfile to explain not needing to run
  systemd-machine-setup-id

Change-Id: I01233ce27cc4558270b2f41fae36b46a17f56e52
2022-08-02 14:07:05 -04:00
Zuul
6745279243 Merge "Upgrade openEuler to 22.03 LTS" 2022-08-02 11:21:59 +00:00
Zuul
b918e7188b Merge "rockylinux : create machine-id early" 2022-08-02 09:27:37 +00:00
annumishra_17
2424943e11 Add subscription-manager repo names for RHEL-9
This commit adds subscription-manager repo names for RHEL-9 qcow2 images

Change-Id: I2f6418a70b6afe543c225d5b2d383f8809842393
2022-08-02 08:53:13 +00:00
Zuul
ac5823cab1 Merge "opensuse: better report checksum errors" 2022-08-01 14:44:36 +00:00
wangxiyuan
934a65bc34 Upgrade openEuler to 22.03 LTS
openEuler 20.03-LTS-SP2 was out of date in May 2022. 22.03 LTS
is the newest LTS version. It was release in March 2022 and
will be maintained for 2 years. This patch upgrades the LTS
version. It'll be used in Devstack, Kolla-ansible and so on
in CI jobs.

This patch also enables the YUM mirror to speed up the package
download.

Change-Id: Iba38570d96374226b924db3aca305f7571643823
2022-08-01 19:22:35 +08:00
Ian Wienand
99f10f9380 rockylinux : create machine-id early
Somewhere between the upstream container
rockylinux/rockylinux:8.6.20220515 and the latest release, systemd
started to be pre-installed in the container.

With <= 20220515 installing the kernel-core package would end up
pulling in systemd.  As part of the systemd package installation, the
/etc/machine-id file is created and populated.

The kernel package post-install steps install the kernel with
/bin/kernel-install; this is responsible for copying the kernel
binaries into /boot.  It does this based on the machine-id, and it
seems its failure case with a blank machine-id is to simply skip
copying the kernels into /boot.  To compound this problem, it seems
our bootloader installation doesn't notice that we don't have a kernel
installed, so we end up building an unbootable image.

Testing is/was showing us this; but as rocky is non-voting and this
occured at a random time (rather than in response to a dib change) I
think it slipped by us.

To work around this, create the machine-id early in the container.  We
already have paths that remove the machine-id from final images.

Change-Id: I07e8262102d4e76c861667a98ded9fc3f4f4b82d
2022-08-01 15:55:11 +10:00
Zuul
50390d047f Merge "Removing old grub removal step" 2022-07-25 04:32:11 +00:00
Zuul
fc0699bec7 Merge "Allow Gentoo to manage python versions by itself" 2022-07-25 02:15:56 +00:00
Zuul
a282d2a2bf Merge "cache-url: turn off -x by default" 2022-07-25 02:04:58 +00:00
Zuul
ba88a12456 Merge "ubuntu: more exact match on squashfs file, containerfile: use focal" 2022-07-12 16:29:16 +00:00
Ian Wienand
4ba0c588dd cache-url: turn off -x by default
I think that generally this is a lot of noise in the logs, as the
internals of cache-url is well tested, so we don't need to trace log
by default.

Change-Id: I25b5a1ec0d8f99691b2b4b62b9fdd537e5a773e4
2022-07-12 17:09:24 +10:00
Ian Wienand
0abf65eaf7 opensuse: better report checksum errors
Report the checksums and size to make it clearer what is failing.

Change-Id: I7c662e13cdbc19b3935f66ed422f15c2de53e525
2022-07-12 14:03:10 +10:00
Ian Wienand
78d389526c ubuntu: more exact match on squashfs file, containerfile: use focal
This is a squash of two changes that have unfortunately simultaneously
broken the gate.

The functests are failing with

 sha256sum: bionic-server-cloudimg-amd64.squashfs.manifest: No such file or directory

I think what has happened here is that the SHA256 sums file being used
has got a new entry "bionic-server-cloudimg-amd64.squashfs.manifest"
which is showing up in a grep for
"bionic-server-cloudimg-amd64.squashfs".  sha256 then tries to also
check this hash, and has started failing.

To avoid this, add an EOL marker to the grep so it only matches the
exact filename.

Change I7fb585bc5ccc52803eea107e76dddf5e9fde8646 updated the
containerfile tests to Jammy and it seems that cgroups v2 prevents
podman running inside docker [1].  While we investigate, move this
testing back to focal.

[1] https://github.com/containers/podman/issues/14884
Change-Id: I1af9f5599168aadc1e7fcdfae281935e6211a597
2022-07-11 19:56:36 +10:00
Matthew Thode
eb162ec6ed
Allow Gentoo to manage python versions by itself
Gentoo can manage python versions itself.  Before this commit users were
forced to set python versions themselves.  Now they have the option to
set it if they wish.

The workaround needed for git is also no longer needed, so it's been
removed.

Change-Id: I06b259ef73a40df6b8ab92a5b424bffcf4ef764d
Signed-off-by: Matthew Thode <mthode@mthode.org>
2022-07-10 22:12:03 -05:00
Zuul
527e75a6de Merge "update default python for gentoo to 3.10" 2022-07-01 00:13:39 +00:00
Zuul
e43597affe Merge "Use internal dhcp client for centos 9-stream and beyond" 2022-06-30 18:20:50 +00:00
Matthew Thode
d85a3421a7
update default python for gentoo to 3.10
Change-Id: Iaf1788342e7418af92dd35714415a979b0ba11db
Signed-off-by: Matthew Thode <mthode@mthode.org>
2022-06-30 09:50:35 -05:00
Christopher Stone
d0a1909e62 Removing old grub removal step
Closes-bug: #1950701
Change-Id: I01f7190db9924baafac4fbc0b45a4c0e158355cc
2022-06-29 15:30:39 -07:00
Julia Kreger
5f01bd5d61 Use internal dhcp client for centos 9-stream and beyond
All indication in CI is that Centos Stream9's use of dhclient
appears to point to compatability issues when interacting with
dnsmasq. However, this doesn't appear to be the issue with the
internal dhcp client. As such, lets constraint the RH default
so that it no longer applies to Centos 9-stream.

I've also added a documentation entry for DIB_DHCP_CLIENT which
was previously undocumented.

As an aside, I've already reached out to RH's NetworkManager team
regarding this, but root cause is not entirely understood at this
point.

Change-Id: I235f75b385a8b0348c8fe064038c51409f8722c4
Story: 2010109
Task: 45677
2022-06-28 11:02:03 -07:00
Steve Baker
f118649738 Fix BLS entries for /boot partitions
Creating a separate /boot partition is desirable in some cases[1].
This change detects if /boot is a partition, and ensures that the
kernel/ramdisk paths are correct in either case. This is applied to
all BLS entries files, whether they were generated by the previous
grub2-mkconfig call or in the source image.

This means the rhel9 specific workaround can be removed since all
paths are now normalised at this stage.

[1] https://review.opendev.org/c/openstack/tripleo-image-elements/+/846807

Change-Id: I62120ec8c65876e451532d2654d37435eb3606a6
Resolves: rhbz#2101514
2022-06-28 02:41:21 +00:00
Zuul
cdf931d190 Merge "Add Fedora 36 support" 2022-06-21 23:42:21 +00:00
Zuul
8bb14b9195 Merge "containerfile: warn if we don't have a Dockerfile" 2022-06-21 23:42:18 +00:00
Ian Wienand
0385857cd9 Add Fedora 36 support
Change-Id: I89ff95604aabf59217382bc784783759191c59a3
2022-06-21 14:47:12 +10:00
Zuul
80c1a519c2 Merge "Add a warning in satellite configuration" 2022-06-21 03:25:58 +00:00