The usage of the DIB_CHECKSUM variable is extended to have an
ability generate the only one checksum file, for example only 'sha256'
(by setting an environment variable DIB_CHECKSUM='sha256'), and to
retain the backward compatibility (DIB_CHECKSUM=1 will generate
both 'sha256' and 'md5' supported at this moment). As an additional
feature we have the simple way to completely deprecate 'md5' later,
and add new methods, for example, 'sha512' etc.
Change-Id: I2dd1c60e3bfd9c823a7382b1390b1d40c52a5c97
Signed-off-by: Maksim Malchuk <maksim.malchuk@gmail.com>
The bootloader element now has variable
DIB_BOOTLOADER_VIRTUAL_TERMINAL to customize or suppress the
console=tty0 kernel argument.
This is proposed to allow console=tty0 to be removed entirely as it is
causing significant performance degredation in DPDK environments.
Change-Id: Iba2ee5b8a6b4acdd236a770550dffd29c784ce11
Related: rhbz#2179366
Red Hat changed the repository names/labels for
Satellite Client repository in Satellite 6.11 and
above, See: https://access.redhat.com/solutions/7004377
This change updates the satellite_repo URL's to use the
new labels.
Also adds environment variable REG_SAT_REPO to allow the
user to override the repository label.
Closes-Bug: #2013451
Change-Id: I6c2a93658213644140caf0e4a8c910b1af22cd1c
The "ubuntu" target had a post-install 99-autoremove task that removed
unnecessary dependency packages, but the "ubuntu-minimal" target does not.
This patch moves the 99-autoremove post-install task from the "ubuntu" target
to the "ubuntu-common" target so that both will run an autoremove at the end of
the image build.
For the Octavia amphora image, this saved about 1GB in the image by removing
build only package dependencies.
Closes-Bug: #2012406
Change-Id: I4592e3bd502045fa89203c075d3ea8f632e77177
Adds an element whose purpose is to set the stage
in the resulting image so that a user can generate an
image utilizing DIB which can be used in a FIPS
configuration without doing so with the input image
or after the fact.
Change-Id: Ia8a45584a56f6e06856fc2920c333351935dcd9d
When your booting a Linux system using dracut, i.e. with any
redhat style distribution, dracut's internal code looks to validate
the kernel hmac signature in before proceeding to userspace.
It does this by looking at the /boot/ folder file for the kernel
hmac file.
And it normally does this with the root filesystem. Except if the
kernel is not on the root filesystem and is instead on a /boot
filesystem, this breaks horribly. This is compounded because
DIB enables the operator to restructure the OS image/layout
to fit their needs. In order for this to be navigated, as dracut
is written, we need to pass a "boot=" argument to the kernel.
So now we attempt to purge any prior boot entry in the disk image
content, which is good because any filesystem operations invalidate
it, and then we attempt to identify the boot filesystem, and save a
boot kernel command line parameter so the resulting image can
boot properly if FIPS was enabled in the prior image.
Regex developed with https://sed.js.org utilizing stdin:
VAR="quiet boot=UUID=173c759f-1302-48a3-9d51-a17784c21e03 text"
VAR="quiet boot=PARTUUID=173c759f-1302-48a3-9d51-a17784c21e03"
VAR="quiet boot=PARTUUID=173c759f-1302-48a3-9d51-a17784c21e03 reboot=meow"
VAR="quiet boot=UUID=/dev/sda1 text"
VAR="quiet boot=/dev/sda1"
VAR="quiet boot=/dev/sda1 reboot=meow"
VAR="quiet after_boot=1 reboot=meow boot=/dev/sda1"
VAR="quiet after_boot=1 reboot=meow"
Which resulted in stdout:
VAR="quiet text"
VAR="quiet"
VAR="quiet reboot=meow"
VAR="quiet text"
VAR="quiet"
VAR="quiet reboot=meow"
VAR="quiet after_boot=1 reboot=meow"
VAR="quiet after_boot=1 reboot=meow"
Change-Id: I9034c21e84deda2ba2c0ec0d1d6d6595ed10bed4
The `diskimage-builder` command provides a yaml file based interface
to `disk-image-create` and `ramdisk-image-create`. Every argument to
these scripts has a YAML equivalent. The command has the following
features:
- Environment values can be provided from the calling environment as
well as YAML
- All arguments are validated with jsonschema in the most appropriate
YAML type
- Schema is self-documenting and printed when running with --help
- Multiple YAML files can be specified and each file can have multiple
images defined
- Entries with duplicate image names will be merged into a single
image build, with attributes overwritten, elements appended, and
environment values updated/overwritten. A missing image name implies
the same image name as the previous entry.
- --dry-run and --stop-on-failure flags
A simple YAML defintion would resemble:
- imagename: centos-minimal
checksum: true
install-type: package
elements: [centos, vm]
- imagename: ironic-python-agent
elements:
- ironic-python-agent-ramdisk
- extra-hardware
The TripleO project has managed image build options with YAML files
and it has proved useful having git history and a diff friendly
format, specifically for the following situations:
- Managing differences between distros (centos, rhel)
- Managing changes in major distro releases (centos-8, centos-9-stream)
- Managing the python2 to python3 transition, within and across major
distro releases
Now that the TripleO toolchain is being retired this tool is being
proposed to be used for the image builds of TripleO's successor, as
well as the rest of the community.
Subsequent commits will add documentation and switch some tests to
using `diskimage-builder`.
Change-Id: I95cba3530d1b1c6c52cf547338762e33738f7225
These must have broken when we switched the base nodes to Jammy.
Update to use compatible versions of distros.
We need to squish another gate-breaking change in here to update the
containerfile "podman build" calls to use "--network host". We added
this with Ia885237406bf4c7b9d49b349f374558ae746401f and the only
external user I can find is kayobe, which is setting this anyway.
I honestly haven't 100% root-caused what changed to require this; the
last time our containerfile jobs ran and worked has unfortunately been
purged so I can't compare versions to try and pinpoint something;
i.e. this may be a podman bug or feature. At first I thought it
related to the networking plugin package from the Depends-On (which is
still useful for the right packages) but that didn't help get the
bridge networking working.
Depends-On: https://review.opendev.org/c/zuul/nodepool/+/867590
Change-Id: I23f091654cb212e8bdd908664b262de9bfe98cef
This change extends the block device lvs attributes to allow creating
a volume which represents a thin pool, and to create volumes which are
allocated from this pool.
Change-Id: Ic58f55c36236cc8c6279fbcb708e27dc2982f2d5
openEuler 20.03-LTS-SP2 was out of date in May 2022. 22.03 LTS
is the newest LTS version. It was release in March 2022 and
will be maintained for 2 years. This patch upgrades the LTS
version. It'll be used in Devstack, Kolla-ansible and so on
in CI jobs.
This patch also enables the YUM mirror to speed up the package
download.
Change-Id: Iba38570d96374226b924db3aca305f7571643823
The block device lvm lvs `size` attribute was passed directly to
lvcreate, so using units M, G means base 2. All other block device
size values are parsed with accepted conventions of M, B being base 10
and MiB, GiB being base 2.
lvm lvs `size` attributes are now parsed the same as other size
attributes. This improves consistency and makes it practical to
calculate volume sizes to fill the partition size. This means existing
size values will now create slightly smaller volumes. Previous sizes
can be restored by changing the unit to MiB, GiB, or increasing the
value for a base 10 unit.
The impact on this change should be minimal, the only known uses of lvm
volumes (TripleO, and element block-device-efi-lvm) uses extents
percentage instead of size. The smaller sizes can always be increased
after deployment.
Requested sizes will also be rounded down to align with physical
extents (4MiB). Previously specifying a value which did not align on
4MiB would consume an extra extent which could unexpectedly consume
more than the partition size.
Change-Id: Ia109cc5105071d82cc895d8d9cb85bc47da20a7a
This reverts commit fe0e5324d4.
Reason for revert: Python3.6 is still being used on Centos 8 based
platforms.
This is a partial revert, since the py36 job is currently failing, it
will be restored in a follow-up patch.
Change-Id: Idc0373f9a639cd66925543376fb1e2e3398666da
Although we're not on the OpenStack release schedule as such, Zed
cycle is dropping 3.6/3.7 support. This means it seems like as good a
time as any to also update ourselves to this regime. One important
dependency to think about is nodepool, but that is already >3.8 only
so we will be in sync there.
This also changes dib jobs to run using the zed template and adapts
the bindep file to handle Ubuntu Jammy.
[1] https://governance.openstack.org/tc/reference/runtimes/zed.html
Change-Id: Ibdbcf459608711ac64e7fefb1707f6708d68e750
Co-Authored-By: Jay Faulkner <jay@jvf.cc>
Co-Authored-By: Jens Harbott <frickler@offenerstapel.de>
Co-Authored-By: Ian Wienand <iwienand@redhat.com>
This reverts commit 8401290976.
We are reverting this because some users may want to use predictable
device names and may not even use Debian. However, after some
investigation we have found a couple of bugs in dhcp-all-interfaces on
Debuntu distros. The parent change corrects those bugs. Additionally new
Linux kernels emit "move" events to udev when interfaces are renamed to
their predictable name. Support this "move" in the dhcp-all-interfaces
udev rules. Making these changes appaers to produce functional images
for Debian users using predictable device names. If predictable device
names are not desired turning them off is straightforward and release
notes are updated to give users the info they need to do that outside of
this element.
Change-Id: I125f1a0c78a103b51bda961528c3e66c345bf604
Co-Authored-By: Clark Boylan <clark.boylan@gmail.com>
Signed-off-by: Maksim Malchuk <maksim.malchuk@gmail.com>
GRUB_OPTS has never been documented as externally available, and is
not used. Assume it's value to simplify the code.
Move the grub version check separately, as we only support grub2
Remove references to buliding i386 images. I don't image it works in
any way.
Remove ci.md, which is no longer relevant.
Refactor the test for "building BIOS image on EFI system" consiberably
after these changes.
Change-Id: Ia99687815667c3cf5e82cf21d841d3b1008b8fa9
The dhcp-all-interfaces element does not work with the predictable names
scheme, fallback to the persistent names scheme as workaround.
Bug: 1960301
Change-Id: I117964a60615a5b7e9984f52f02cd018d1a48ed0
Using rpm -e to remove old kernels fails when other packages also
depend on the removed kernel.
This change reverts back to using dnf to remove the kernel, but also
sets the config value protect_running_kernel=False to avoid the issue
where the build host kernel version matches the version of the package
being deleted.
reverts commit 1ac31afd62.
Change-Id: Ie58630c23a34f2db34f3934abbd0c1076ab9d835
SUSE dropped OpenStack Cloud in 2019 [1], and as a result, some
OpenStack-related repositories were removed from openSUSE Download and
root filesystem images stopped being provided. This change deprecates
Leap releases before 15.3 and employs the extract-image script. It also
moves the extract-image script to the sysprep element, since now it's
also used by openSUSE-related elements.
Additionally, revert the "Remove opensuse related funtests" change [2]
so that the opensuse element is tested again and set the default Leap
release to 15.3.
[1] https://www.zdnet.com/article/suse-drops-openstacks/
[2] https://review.opendev.org/c/openstack/diskimage-builder/+/824002
Change-Id: I73d6323aa65cee69a55e54bc53ed682f096dfc89
NetworkManager is quite capable to do automatic
interface configuration. NetworkManager will by default
try to auto-configure any interface with no configuration.
It will use DHCP for IPv4 and Router Advertisements to
decide how to initialize IPv6.
It will most likely do it just as good, or better than the
dhcp-all-interfaces.sh script.
Since dhcp-all-interfaces clean out all ifcfg files in
60-remove-cloud-image-interfaces it means NetworkManager will
by default attempt auto configuration for all interfaces.
This change add's and environment variable:
DIB_DHCP_NETWORK_MANAGER_AUTO (default: false)
When DIB_DHCP_NETWORK_MANAGER_AUTO is set to `true` only the
NetworkManager config will be written. The dhcp-all-interfaces
service will not be installed. Hence dhcp-all-interfaces will
not write any config files, allowing NetworkManager to just do
it's thing.
Change-Id: Id6f8d6aaaf52a78175bb6c065ec88274c364834e
This change:
- adds a note regarding an error when building focal ubuntu-minimal
images on operating systems with older versions of debootstrap
- adds a reference to where the DIB_RELEASE variable definition can be
found
Closes-Bug: #1941831
Change-Id: Ibc1e04dba0562c4f4909a8cb8af041d9b8ac45c4
In the grub2 element the grub2-efi-x64-modules package
is missing in the centos 9 section, this cause a failure
because grub2 cannot find the neccecary files when
installing the bootloader on EFI systems.
It seems grub2-efi-x64-modules was not included in release
9, this is likely why the block was added initially without
this package. Since it is now there, the Centos 9 specific
block is no longer needed.
Removing the rhel 8 block as well, as it is identical to the
family "redhat" block i.e it is redundant.
Closes-Bug: #1957169
Change-Id: Ia6b0ecf0cd15fb23c6740543940ee513a8602afe
Ic68e8c5b839cbc2852326747c68ef89f630f26a3 removed the sudo from the
tar extraction here, meaning that production is failing to create the
chroot. This is hidden in testing because
DIB_CONTAINERFILE_PODMAN_ROOT is set. Make the sudo here
unconditional.
Change-Id: I6e36e3fc65981f85fad12ea2cd10780fde9c37da
A custom yum repository can now be configured by defining
`DIB_YUM_REPO_PACKAGE` as a yum available package or a URL to an rpm file.
This package can install repo files with any associated keys and
certificates.
A good example of such a package upstream is rdo-release[1] which
includes multiple repo files, the repo keys, and a root certificate.
This makes these repos impractical to install via DIB_YUM_REPO_CONF.
Downstream, repo packages like this a frequently used to bootstrap
development builds of RHEL with development repos.
[1] https://www.rdoproject.org/repos/rdo-release.rpm
Change-Id: I2832e723998c9bd7635cdf7541a4c20eff6294d2
The latest Debian bullseye release doesn't provide yum any more, only
DNF. This breaks the minimal builds that are using on-host yum tools
to start the chroot. Probe for yumdownloader, and if it's not there,
use DNF.
Note this requires "dnf download" which may not be packaged. See
I21cfbd3935e48be4b92591ea36c7eed301230753 for a sample work-around
that installs this plugin in the nodepool-builder container.
Change-Id: Ia7f1e4d115cc67c378d865d91af94a07b8cdc6cc
Add openeuler-minimal element and add CI functional tests for both
x86_64 and arm64.
OpenEuler is an open source community driven YUM/DNF distro like
Fedora. It references Fedora and CentOS a lot for the rpm packages
building. So somewhat it can be treated as a redhat family distro
and reuse the YUM/DNF related elements to help build openEuler images.
For more info about openEuler, see: https://openeuler.org/en
Depends-On: https://review.opendev.org/c/zuul/zuul-jobs/+/803413
Change-Id: I3e06e49b524364c3a4edeba8bce7a8c06b9c7b76
This change permits the yum-minimal element to be used in downstream
custom distributions, which may have additional packages containing repo
config or GPG keys needed.
This could also be utilized at a later time to move the
distribution-specific logic in this method to each distribution element
separately.
Change-Id: Ic1434bb2fe7301086cf11ba6bd7f2ee187c5e6c8
As noted inline, this works around potential issues by being a strong
indication you are in a container (e.g. [1]). Since nothing should be
changing anything on the host/build system, this is a generically
safer way to operate.
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1975588
Change-Id: Ic6802c4ffc2e825f129af10717860a2d1770fe80
Element block-device-efi-lvm has been added which is like
block-device-efi but defines an LVM logical group in the root
partition. Three logical volumes are defined in that group, mounted to
/, /var, and /home.
This volume layout will not meet all requirements, but this is more of
an example demonstrating the capability to encourage more usage of
this existing feature.
This is based on the overcloud-partition-uefi element in
tripleo-image-elements, and I believe this capability is too useful to
have the only working example buried in a related project repo.
This change also fixes the element string matching in
_arg_defaults_hack, the 'vm' test was also matching against 'lvm' and
'block-device-efi-lvm' elements. Also the 'block-device-' test now
properly tests for this being the prefix of the block-device element.
This change also makes block-device-efi fsck-passno compliant with the
documentation[1] so that / has value 1 and all other mounts are set to
2.
[1] https://www.man7.org/linux/man-pages/man5/fstab.5.html
Change-Id: If86a0e49186ce5a65cc0084101d31ce59a97b854
Blueprint: whole-disk-default
This is a first pass through the bootloader, that removes the extlinux
and syslinux install/cleanup path.
Change-Id: Ifb107796cdb6748430a124bf13ced93db9689bff
Despite having several issues (like missing firmware), it is still
used by people. It seems that the only way to stop that is to remove it.
Change-Id: I4baed8e8ab663c624dcc8d06ff0293d57b082abb
Provide ability to run dhcp client on VLAN interfaces created on
top of an Ethernet interface. See also
<https://storyboard.openstack.org/#!/story/2008298>`__ for further details.
Change-Id: Ic3ffd7b8e23b1e996cfe6c79ce0ff47e521f30be
