Commit graph

1012 commits

Author SHA1 Message Date
Zuul
9b931852f7 Merge "Update the shebang to python3" 2023-08-29 15:44:45 +00:00
Zuul
04f140535b Merge "Install netplan.io for Debian Bookworm" 2023-08-28 20:17:42 +00:00
Zuul
5f1581fc81 Merge "Fix baseurl for Fedora versions before 36" 2023-08-22 23:52:01 +00:00
Steve Baker
420fb14e8f growvols: reserve space for spare metadata volume
Currently space is reserved for the metadata volume, but there is also a
spare metadata volume which is used for metadata check and restore.

This change reserves space for the spare. It also changes the volume
reference in the lvextend call to vg/lv_thinpool, the path based
reference results in the spare not growing.

Resolves: rhbz#2232632
Change-Id: If78743bb37f24756c049939645db202261df6775
2023-08-23 08:29:20 +12:00
Dmitriy Rabotyagov
bb9e970835 Install netplan.io for Debian Bookworm
Debian Cloud Images are shipped with netplan as a way to
configure networking for Debian. Without netplan being installed,
images built by DIB with cloud-init do not bring networking up,
since systemd-networkd is not enabled after installation, and there
are no other means to configure networking.

Alternative approach could be to enable networkd, though it is
better to be closer to official cloud images.

Change-Id: I115ab83cf374819bc447fc1bd596e71326d13ed9
2023-08-15 07:40:45 +00:00
Dmitriy Rabotyagov
2bfa132264 Fix baseurl for Fedora versions before 36
Fedora has moved their old images (prior to 37) to archives,
so default URIs are adjusted to reflect that.

Change-Id: I81dc6df4e4372149cffe448708ac2416d3528394
2023-08-07 14:38:05 +02:00
Joel Capitao
8dc4c3cd1e Update the shebang to python3
It's needed downstream when packaging. In RDO and Fedora,
we are hitting the issue "ERROR: ambiguous python shebang" during
the build.

Change-Id: Icaf0e62a8c6f0f873016ef1d17c434941e794c2f
2023-07-25 18:55:18 +02:00
Zuul
deb1869c6e Merge "Adds passwd to rocky-container os packages" 2023-07-17 15:15:38 +00:00
waleedm
33c11e1e9c Add nm-dhcp-ib-interfaces element
Currently, NetworkManager can't automatically create default
connection profiles for InfiniBand interfaces.

So, as a workaround, we are adding nm-dhcp-ib-interfaces element to
install NetworkManager-system-connections-infiniband.nmconnection
to NetworkManager to create a wildcard InfiniBand connection profile.

The content of NetworkManager-system-connections-infiniband.nmconnection
is generated by running this command:
`nmcli --offline connection add type infiniband connection.multi-connect multiple`

Closes-Bug: #2016965
Change-Id: Ic972b90e4df9c4aa36cfe3c8631db3e4533045f4
2023-06-04 05:45:12 +00:00
Maksim Malchuk
d8eb624e72 fedora: don't use CI mirrors
OpenDev has removed Fedora from the mirror AFS volumes.  Don't try to
use it during build.

Co-authored-by: Ian Wienand <iwienand@redhat.com>
Change-Id: I0b337c37421de2ff893bcd90d99f569d6551c79e
Signed-off-by: Maksim Malchuk <maksim.malchuk@gmail.com>
2023-06-01 09:28:31 -07:00
wangxiyuan
ed9bdf805d Fix openeuler-minimal mirror problem
TARGET_ROOT is not a valid key during pre-install step. The error
`TARGET_ROOT: unbound variable` raises now.

This PR remove the useless TARGET_ROOT in openeuler-minimal.

Change-Id: I6fae2ca0e6c6e57a3b195367f2626d9e498e0329
2023-04-23 08:24:29 +00:00
Zuul
f8733f729b Merge "Allow custom console=tty0 argument" 2023-04-20 04:27:09 +00:00
Zuul
761fb12682 Merge "fix ifupdown pkg map for dhcp-all-interfaces of redhat family" 2023-04-13 21:46:38 +00:00
Zuul
da03a4d160 Merge "Update satellite_repo labels + add env var" 2023-04-13 08:32:31 +00:00
Zuul
48cbe94ef8 Merge "Check and grow the GPT structure to the end of the volume" 2023-04-11 20:35:05 +00:00
Steve Baker
d56dd56881 Allow custom console=tty0 argument
The bootloader element now has variable
DIB_BOOTLOADER_VIRTUAL_TERMINAL to customize or suppress the
console=tty0 kernel argument.

This is proposed to allow console=tty0 to be removed entirely as it is
causing significant performance degredation in DPDK environments.

Change-Id: Iba2ee5b8a6b4acdd236a770550dffd29c784ce11
Related: rhbz#2179366
2023-04-11 17:15:12 +12:00
waleedm
b50b8489d8 fix ifupdown pkg map for dhcp-all-interfaces of redhat family
Now as network-scripts package is derprecated, and NM should be the default for managing networking in redhat OS family, we need ifup/down scripts for dhcp-all-interfaces element and no one is providing them
So adding the package map for redhat family in the element source

Change-Id: I11e596e17477c39b156617762c4c5b840754ed64
2023-04-05 09:44:40 +00:00
Harald Jensås
dd0ee23989 Update satellite_repo labels + add env var
Red Hat changed the repository names/labels for
Satellite Client repository in Satellite 6.11 and
above, See: https://access.redhat.com/solutions/7004377

This change updates the satellite_repo URL's to use the
new labels.

Also adds environment variable REG_SAT_REPO to allow the
user to override the repository label.

Closes-Bug: #2013451
Change-Id: I6c2a93658213644140caf0e4a8c910b1af22cd1c
2023-03-31 09:59:30 +02:00
wangxiyuan
e5cf2faae3 Rename openeuler mirror script
The file name under pre-install.d should be like "^[0-9A-Za-z_-]+$",
otherwise the script will be skipped by the name check step and raise
"Ignoring non-conforming filenames" warning.

This patch rename the openeuler mirror script to ensure it follows
the regex.

Change-Id: I6d868df7932345547711100519523546bd53360e
2023-03-29 02:29:33 +00:00
Steve Baker
9fa139511e Check and grow the GPT structure to the end of the volume
In the baremetal case this isn't required because it is done by
ironic-python-agent when writing the image to the volume[1].

But when using the image directly (such as in a nova VM) the GPT
structure needs to be extended first. This change does that, along
with the detection for whether extending is required, using the same
approach as [1].

[1] https://github.com/openstack/ironic-lib/blob/master/ironic_lib/disk_utils.py#L670-L674

Co-Authored-By: rminishev@itkey.com
Co-Authored-By: sbaker@redhat.com

Change-Id: I3240eb0ef4dbbb41557985f0129ae4998a846417
2023-03-23 22:14:44 +00:00
Zuul
5d49a95082 Merge "Fix openeuler mirror problem" 2023-03-22 21:39:19 +00:00
Zuul
f8b0468e06 Merge "Add a FIPS element" 2023-03-22 21:39:17 +00:00
wangxiyuan
65507dfe82 Fix openeuler mirror problem
Currently, openeuler mirror doesn't work. It raises the error like:
"Ignoring non-executable files: 00-setup-mirror.bash".

This patch make the file executable to fix the mirror config problem.

Change-Id: I39ec958f864c2ff5193ea6d8f85d672437367ca0
2023-03-22 06:09:23 +00:00
Michael Johnson
45544d5038 Fix ubuntu-minimal to run autoremove
The "ubuntu" target had a post-install 99-autoremove task that removed
unnecessary dependency packages, but the "ubuntu-minimal" target does not.
This patch moves the 99-autoremove post-install task from the "ubuntu" target
to the "ubuntu-common" target so that both will run an autoremove at the end of
the image build.
For the Octavia amphora image, this saved about 1GB in the image by removing
build only package dependencies.

Closes-Bug: #2012406
Change-Id: I4592e3bd502045fa89203c075d3ea8f632e77177
2023-03-21 15:19:37 +00:00
Julia Kreger
c217956079 Add a FIPS element
Adds an element whose purpose is to set the stage
in the resulting image so that a user can generate an
image utilizing DIB which can be used in a FIPS
configuration without doing so with the input image
or after the fact.

Change-Id: Ia8a45584a56f6e06856fc2920c333351935dcd9d
2023-03-21 13:07:02 +00:00
Zuul
9c1ee6dcd8 Merge "Correct boot path to cover FIPS usage cases" 2023-03-21 06:39:00 +00:00
Zuul
a00a68ac7a Merge "Update Fedora to 37" 2023-03-20 00:52:48 +00:00
Zuul
ad87cc76b8 Merge "Fix double-keyed json" 2023-03-16 21:39:14 +00:00
Julia Kreger
4633da7750 Correct boot path to cover FIPS usage cases
When your booting a Linux system using dracut, i.e. with any
redhat style distribution, dracut's internal code looks to validate
the kernel hmac signature in before proceeding to userspace.

It does this by looking at the /boot/ folder file for the kernel
hmac file.

And it normally does this with the root filesystem. Except if the
kernel is not on the root filesystem and is instead on a /boot
filesystem, this breaks horribly. This is compounded because
DIB enables the operator to restructure the OS image/layout
to fit their needs. In order for this to be navigated, as dracut
is written, we need to pass a "boot=" argument to the kernel.

So now we attempt to purge any prior boot entry in the disk image
content, which is good because any filesystem operations invalidate
it, and then we attempt to identify the boot filesystem, and save a
boot kernel command line parameter so the resulting image can
boot properly if FIPS was enabled in the prior image.

Regex developed with https://sed.js.org utilizing stdin:

VAR="quiet boot=UUID=173c759f-1302-48a3-9d51-a17784c21e03 text"
VAR="quiet boot=PARTUUID=173c759f-1302-48a3-9d51-a17784c21e03"
VAR="quiet boot=PARTUUID=173c759f-1302-48a3-9d51-a17784c21e03 reboot=meow"
VAR="quiet boot=UUID=/dev/sda1 text"
VAR="quiet boot=/dev/sda1"
VAR="quiet boot=/dev/sda1 reboot=meow"
VAR="quiet after_boot=1 reboot=meow boot=/dev/sda1"
VAR="quiet after_boot=1 reboot=meow"

Which resulted in stdout:

VAR="quiet text"
VAR="quiet"
VAR="quiet reboot=meow"
VAR="quiet text"
VAR="quiet"
VAR="quiet reboot=meow"
VAR="quiet after_boot=1 reboot=meow"
VAR="quiet after_boot=1 reboot=meow"

Change-Id: I9034c21e84deda2ba2c0ec0d1d6d6595ed10bed4
2023-03-15 11:25:21 -07:00
Ian Wienand
b6cdda4ab0
Update Fedora to 37
This seems like a relatively simple update with nothing breaking
(famous last words).

Change-Id: I5498371ad5af3f0f053af633eb26121ea9f780e6
2023-03-15 16:18:22 +11:00
John L. Villalovos
47dc5a9834 chore: support building Fedora on arm64 AKA aarch64
By default [1] the `aarch64` ARCH value is converted to `arm64`. But
Fedora uses `aarch64` to refer to the architecture.

Convert incoming ARCH values of `arm64` into `aarch64` as is already
done for `amd64` -> `x86_64`

[1] 174089a6a5/diskimage_builder/lib/common-defaults (L29-L30)

Change-Id: I6d9698e45b1183007bac49544da196ec78a7ac6a
2023-03-10 09:32:08 -08:00
Erik Berg
b78afe4c62
Fix double-keyed json
The second release masks the first release, which is probably a
mistake.

Order them from most significant to least;
release > distro > family > default

And fix up the indentation.

Change-Id: I54a6a49d4fe001b1a16ab38637cb55542ce96cdb
2023-03-03 12:04:30 +01:00
Zuul
174089a6a5 Merge "Reduce thin pool by one more extent" 2023-01-27 02:34:56 +00:00
Ian Wienand
2c4d230d7a
cache-url: Give up on curl install for Redhat platforms
The RedHat platforms vary if they come pre-installed with curl or
curl-minimal.  For example, Fedora 37 container images have curl, and
centos 9-stream and Rocky images have curl-minimal.  If you try and
install curl when curl-minimal is installed, you get an error, and
vice-versa.

Unfortunately package-installs can't really sort this out; we're just
passing a package list to the system package manager.  We don't have a
way to say "the curl OR curl-minimal package is fine".  As this breaks
builds and is such a common dependency that it's already there, let's
just add a note that curl is required and blank out the package-map.

Change-Id: I9ccebe2dbf3a8682dab60c2070c5f78849e01446
2023-01-20 17:29:13 +11:00
Steve Baker
7e2a2aa027 Reduce thin pool by one more extent
The previous commit was tested on 2TB without issue, but testing on a
very small volume (80GB) resulted in the thin pool lvextend failing
for being one extent too large.

This change reduces the pool size by one extent.

Change-Id: I7ca002783f8f15946bc84af95eecaa097e70aaf1
Related: rhbz#2149586
2022-12-22 17:24:03 +13:00
Zuul
4a161d0cde Merge "Add variable for check installing python3 in yum element" 2022-12-21 05:16:35 +00:00
Zuul
9d0ba50494 Merge "Grow thin pool metadata by 1GiB" 2022-12-21 04:20:44 +00:00
Steve Baker
00ca126287 Grow thin pool metadata by 1GiB
An LVM thin pool has an associated metadata volume, and it can be
assumed that the size of this volume on the image is minimized for
distribution.

This change grows the metadata volume by 1GiB, which is recommended[1] as
a reasonable default. This fixes a specific issue with the metadata
volume being exausted when growing into a 2TB drive.

Other minor changes include:
- Human readable printed values have switched to GiB, MiB, KiB, B
- Growth percentage volumes are adjusted down to not over-provision
  the thin volume

[1] https://access.redhat.com/solutions/6318131

Change-Id: I1dd6dd932bb5f5d9adac9b78a026569165bd4ea9
Resolves: rhbz#2149586
2022-12-19 13:40:04 +13:00
Frikin Evgenii
a9edef6548 Add variable for check installing python3 in yum element
If client have not internet or have some limitation, such as firewall/proxy/etc. this step will stop build image with error. Client must have possible override of URL for pass this step.

Change-Id: Iafe3283665a437d0a9cf83a93ff66c0613310b69
2022-12-16 03:51:10 +00:00
Zuul
b706802a1c Merge "Fix issue in extract image" 2022-12-16 03:43:14 +00:00
Ian Wienand
41aa936fa2
tox jobs: pin to correct nodesets; use host networking for containerfile
These must have broken when we switched the base nodes to Jammy.
Update to use compatible versions of distros.

We need to squish another gate-breaking change in here to update the
containerfile "podman build" calls to use "--network host".  We added
this with Ia885237406bf4c7b9d49b349f374558ae746401f and the only
external user I can find is kayobe, which is setting this anyway.

I honestly haven't 100% root-caused what changed to require this; the
last time our containerfile jobs ran and worked has unfortunately been
purged so I can't compare versions to try and pinpoint something;
i.e. this may be a podman bug or feature.  At first I thought it
related to the networking plugin package from the Depends-On (which is
still useful for the right packages) but that didn't help get the
bridge networking working.

Depends-On: https://review.opendev.org/c/zuul/nodepool/+/867590
Change-Id: I23f091654cb212e8bdd908664b262de9bfe98cef
2022-12-16 09:52:46 +11:00
jgupta
f2e7cd1307 Fix issue in extract image
The problem lays with the 'extract-image' script as
it is using lsblk commands to extract image's partition
(find out root/efi/boot, lines:100-102) but the output
is empty inside a container.

lsblk gives empty output for FSTYPE, LABEL, GUID..
the fix is to use blkid.

Closes-Bug: 1974350
Change-Id: I3b460c6dd9caa519c55327c5bd4b7e4585a8bd22
2022-11-02 14:21:16 +00:00
Rafal Lewandowski
4ff8721a66 Added example configuration
Change-Id: Icebe0e2f43f4a96d3c1fa464c6f2dc3da3be65c5
2022-10-20 10:00:18 +02:00
Zuul
ea7de37ccb Merge "Added cloud-init growpart element" 2022-10-13 20:41:55 +00:00
Rafal Lewandowski
b73048f98d Added cloud-init growpart element
Added growpart element. It allows for growing specific partitions
during the deployment, which will result in less post deploy actions
needed for the server to be ready for use.

Change-Id: I6519fba3e8f1d078b99d3c03f2ac85f7b6e37d8a
2022-10-11 16:58:26 +02:00
Zuul
3e72fdbf77 Merge "Fix wrong yum.conf name of CentOS 9 Stream" 2022-10-05 14:50:07 +00:00
Ian Wienand
d321b08f4d
Add Rocky 9 ARM64 functional test
This will test if we can get a Rocky 9 ARM64 image

Needed-By: https://review.opendev.org/c/openstack/project-config/+/858554
Change-Id: I3e589e4838cea489e11719b5f7fce82bf0d978e8
2022-09-21 15:08:20 +10:00
Ian Wienand
2a25db9ee6
Start running dib-lint again
A recent change that didn't fail with hard-tabs made me realise we're
not running tox -e pep8 ... which means we're not running dib-lint
which should find this (and other things).

I couldn't pinpoint when this happened; maybe job config was never in
this repo.

Anyway, move the pylint and dib-lint/flake8 testing to the now
standard "linters" and update the linting job to
openstack-tox-linters.

It looks like pylint is very lightly used (came in with
I7e24d8348db3aef79e1395d12692199a1f80161a and we've never expanded any
testing).  Leave this alone for now, but probably it is not important
any more.

This revealed some issues; updated flake8
(Iaa19c36f8cab8482a01f764c588375db8e7d8be3) found some spacing issues
with keywords and an update to elrepo to match our standard bash
flags.

Change-Id: I45bf108c467f7c8190ca252e6c48450c2622aaf8
2022-09-21 07:56:05 +10:00
Clark Boylan
793e16b76c Install Fedora ifcfg NM compat package
Starting with Fedora 36 the NetworkManager package no longer includes
ifcfg support by default. You need an additional package
"NetworkManager-initscripts-ifcfg-rh" to pull in the compatibility
plugin. Glean's support for Fedora relies on this compatibility system
so we install this package via the simple-init element package deps.

Change-Id: I76ac39b8dedcb1c5bc4595aedc0a732c99c8721e
2022-09-20 08:58:37 -07:00
Zuul
9ce3358fde Merge "changed release check logic in lvm element" 2022-09-16 12:30:03 +00:00