Fedora cloud images have sub-releases in their filename. It is not
exacly clear how this is generated but we do know how we can determine
the greatest programatically.
Change-Id: I7fc56897c681fe037db211c290edcdd23cdd5d5b
This makes the container file element search the active element list
for `containerfiles/${DIB_RELEASE}` for building. This makes it easy
to write wrappers for ubuntu/fedora/etc. containerfile elements.
Change-Id: I68f1d928e54a70bad76985ddd3e156bb5f978b0d
This is a base element which uses a containerfile (Dockerfile) to
build a container image, then the filesystem is extracted from that
image and forms the root of the dib image.
You can add as little or as much to the dockerfile as desired.
Change-Id: I4e821aa2ce7feb8841ef31da56de1a31aa9218b5
With the release of Debian bullseye and later, security updates are
provided in the bullseye-security suite instead of bullseye/updates.
Change-Id: I63580ec96a53e5e8ef8d105e766d838029727917
Currently Debian sets /etc/debian_version to "bullseye/sid" and, due
to a series of issues explained in [1] more fully "lsb_release -c" in
the OpenDev environment doesn't return the distribution code name.
Overriding this to the final release version fixes this.
[1] http://lists.opendev.org/pipermail/service-discuss/2021-April/000222.html
Change-Id: I00c1741dac6ad5f2c4bf855a207f17d8985bc763
Some older distros (like centos8 and xenial) don't support SNI in their
easy_install implementations which are used to install setup_requires
for python packages. PBR is a setup_requires for glean. We work around
this problem when installing glean by preinstalling PBR with pip.
Change-Id: Ie9f5c9ed06954cbe51f23fe8cca0655a931a5201
When a build fails, we can exit and leave ${PROFILE_DIR} behind. Make
sure this is cleaned up with an exit trap.
While we're adding a function, update the syntax of the others for
consistency.
Change-Id: I14499b5ebaaa30126aaa6b3d1bd86ed64f110fda
The rhel-8.4 qcow2 base image already has the grub2-efi-x64 package
installed on its single partition which has files installed to
/boot/efi..., however a partitioned image will have an empty /boot/efi
partition when running 50-bootloader. This means dnf will not install
grub2-efi-x64 when requested and /boot/efi will remain empty.
This commit makes the following changes:
- Refactors redhat bootloader pkg-map for the following:
- Make x86_64/amd64, arm64/aarch64 adjancent so they don't diverge
- Map grub-efi to packages installed to /usr
- Map grub-efi-{arch} to packages installed to /boot/efi
- Removes packages grub-efi-{arch} before installing grub-efi and
grub-efi-{arch}
Change-Id: Ia197feea34f43bd870fed30829b740596e6b2f48
https://review.opendev.org/c/openstack/diskimage-builder/+/785138
adds the support for DIB_DNF_MODULE_STREAMS which is now available
for all Yum based distros.
This patch enhances the docs for using it for all Yum
based distributions.
Signed-off-by: Chandan Kumar (raukadah) <chkumar@redhat.com>
Change-Id: I29e726679c2b675b3c0cd95a3ff48fdad7cd5431
We've noticed that centos8 arm64 images have a root devices of
/dev/mapper/loop7p3 which make sense within a dib image build context
but not at boot time. Dib intends to use labels to set the root device
but when efi is used we end up running grub2-mkconfig against the efi
grub config path before we configure grub to use labels.
Fix this by running grub2-mkconfig after its configuration is set.
This should avoid confusion and complicated paths through the scripts
that configure this for us. We then copy the resulting config to the efi
specific grub.cfg location for platforms that have it.
There is also a small refactoring that is done to try and make the ~3
boot variants more clear:
1) Booting with legacy bios
2) Booting with uefi without a signed shim that directly calls grub
3) Booting with uefi and a signed shim that calls grub
Options 1 and 2 share the /boot/grub*/grub.cfg file. Option 3 needs its
grub.cfg to live alongside distro specific efi target.
Change-Id: Ie9790da9d1bbea58197b37b15a48e77f8a93c1ac
While building cloud images, it is common to set modules
for CentOS and RHEL images. Earlier it was part of rhel-common
which was specific to RHEL OS not for CentOS. Moving it
under yum element as module/stream can be enabled or disabled
via dnf itself.
Signed-off-by: Chandan Kumar (raukadah) <chkumar@redhat.com>
Change-Id: Idc0f277f97e92e4d003f059f01b59f1b5513da34
At this moment the IPA image building with OpenSuse is broken and here, it was failing during the release check for Opensue because etc/SuSE-release is not valid anymore and deprecated for openSuse. Its renamed to /etc/os-release for openSuse rlease 15. This PR will solve the issue to build IPA image with OpenSuse base image. There is another PR opened in ironic-python-agent-builder, which adds all the missing packages, setuptools upgrade and svc mapping to do the build successful.
https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/778726
Bug-Report: https://bugs.launchpad.net/diskimage-builder/+bug/1921510
Change-Id: Id2759be29bfcbf2ecf1ce67e171686924b506b1a
open-iscsi and open-isns need keywording to support gcc-10, move it out
of being keyworded only for musl profiles.
remove unneeded keywords for python-exec and python-exec-conf (marked
stable)
use the full package name for the dev-lang/python-exec-conf package
Change-Id: I44eaf8c2230e9e2089a72fce46954f4336626843
Signed-off-by: Matthew Thode <mthode@mthode.org>
As of grub2 >= 2.02-95 on redhat family distros, calling grub2-install
on an EFI partition will fail with: "this utility cannot be used for
EFI platforms because it does not support UEFI Secure Boot."
This version of grub is now in centos8-stream and non-eus repos of
RHEL-8. It is not currently possible to build whole-disk UEFI images
on these distros, and when this package is promoted this will also
affect centos8 and RHEL-8 eus. The grub maintainers made this change
because the grub2-install generated /boot/efi/EFI/BOOT/BOOTX64.EFI
will never be capable of booting with Secure Boot.
This change defines a $EFI_BOOT_DIR for every distro element. When
directory /boot/efi/$EFI_BOOT_DIR exists a grub.cfg file in will be
generated there. This change also installs the shim package on redhat
family distros, which installs a copy of the shim bootloader to
/boot/efi/EFI/BOOT/BOOTX64.EFI. Using centos as an example, this
allows UEFI to boot the shim /boot/efi/EFI/BOOT/BOOTX64.EFI which
then chains to /boot/efi/EFI/centos/grubx64.efi.
If /boot/efi/$EFI_BOOT_DIR doesn't exist (such as for Ubuntu,
/boot/efi/EFI/ubuntu) the current behaviour of running grub-install to
generate /boot/efi/EFI/BOOT/BOOTX64.EFI will continue. For distros
such as Ubutnu where packaging does not populate /boot/efi/EFI/ubuntu
with .efi files, secure boot can be added in the future by copying
.efi files to /boot/efi/EFI/ubuntu and copying the shim file to
/boot/efi/EFI/BOOT/BOOTX64.EFI.
Change-Id: I90925218ff2aa4c4daffcf86e686b6d98d6b0f21
The path for get-pip.py script in version 3.5 has been changed
with this commit [1].
[1] 2360f025eb
Change-Id: Ifde16e40b4e241c6c4c93df44330c403ee903e6f
As of grub2 >= 2.02-95 calling grub2-install on an EFI partition will
fail with: "this utility cannot be used for EFI platforms because it
does not support UEFI Secure Boot." In the case of ironic deployments,
ironic-python-agent will call efibootmgr to set the local boot for
subsequent boots.
This change adds efibootmgr to the image as well so any other UEFI
menu changes can be made manually.
Change-Id: I765ded15da07d6227d1e337960e54ad0e0d6ca39
The python3/python3-pyyaml packages both are never installed and dnf
itself never updated when $DIB_DISTRIBUTION_MIRROR set and used.
This change fix the order of the operations:
1. yum/dnf configure.
2. *.repo patching.
3. yum/dnf update/install execution.
Change-Id: Ifbbf1f0190fe8c8a77fb3be820e8056447e755f6
Signed-off-by: Maksim Malchuk <maksim.malchuk@gmail.com>
The trvial fix allow override to work in air-gapped envirments where
the command 'curl -s https://cloud.centos.org/...' would fail.
Change-Id: I84296d8816042e4cd4cb02f15746b86d600d13d6
Signed-off-by: Maksim Malchuk <maksim.malchuk@gmail.com>
Use the actual virtualenv to install proliant tools, and make sure the
dependencies needed for sum firmware upgrade are included.
Change-Id: Ie1dbb868450918567b2903cdeccda35af1904417
Closes-Bug: #1916346
This PR updates locations for files used by the bootloader depending on
the target operating system built. The current logic does not take into
account latest versions of operating systems and makes it impossible to
build ISOs against those.
With this change it is possible to correctly build CentOS 8, Ubuntu
18.04 and Ubuntu 20.04 images.
Closes-Bug: #1916913
Change-Id: I3ed0041640f539e82805d03ba26fe46217f3ac3c
The only difference between the rhel and redhat entries is rhel has
the extra grub-efi-x86_64 mapping. All redhat family releases would
benefit from having this too, so this change removes the whole rhel
entry and adds grub-efi-x86_64 to the redhat family.
The assumption is that anything which applies to rhel also applies to
centos-stream, and in this case doesn't harm centos or fedora either.
Change-Id: I0dc44c1f2b57516742f4c3e43cfc8874d6b90fa2
This package doesn't exist in the stream base repo, and neither does
centos-linux-repos.
These are presumably replaced by centos-stream-release and
centos-stream-repos. This change adds an else block to handle the
non-stream base packages.
Change-Id: I32249199c3dfa44fc24fba28d24f314112c2e200
This change will remove any existing interface configurations in the image. They are not necessary
and could interrupt with deployments. In any case they should not exist if we use
dhcp-all-interfaces element.
Change-Id: I35a4b5ea6e2315de3b0d9f8353ac2b6f4b995697
Set eus repositories if REG_RELEASE is set instead of the base repos
as the current behavior is to use the non-EUS repositories for RHEL
8.2 deployment which breaks image building for customers.
Change-Id: I8e687b27922c3f6fc3d69794866795ab89ecc346
The get-pip script does not work well with Python 2 and
it raise error during the installation [1].
[1] https://github.com/pypa/get-pip/issues/83
Change-Id: I3755c34da313ef647547c6ae18b59cc04c2cdd60
portage now generates /etc/python-exec/python-exec.conf based on the
order of PYTHON_TARGETS in /etc/portage/make.conf
fixes an issue where ARCH was being detected as amd64 not x86_64
fixes kernel installs (virtual/dist-kernel)
standardizes simple if statements (note, the 'shorthand' method will
pass the exit code back to shell but the 'longhand' does not).
Change-Id: I74041c232bc6ab4d6e67a4ecfaa759aa4a5feb6c
Signed-off-by: Matthew Thode <mthode@mthode.org>
* Add "DIB_UBUNTU_MIRROR_DISTS":
Default: ``updates,security,backports``
Notes: For some deployment, is may be required
to disable backport|update|etc packages
integration.
Change-Id: Ic7dcd29ea658a66763b4422915149e4d3fe663cc
Despite having several issues (like missing firmware), it is still
used by people. It seems that the only way to stop that is to remove it.
Change-Id: I4baed8e8ab663c624dcc8d06ff0293d57b082abb
This use the same workaround as element bootloader's[1] to fix CentOS
8.3 partition image building error with iscsi-boot:
...
2021-01-12 07:11:02.439 | + grub2-mkconfig -o /boot/grub2/grub.cfg
2021-01-12 07:11:02.654 | /usr/bin/grub2-editenv: error: cannot rename the file /boot/grub2/grubenv.new to /boot/grub2/grubenv: No such file or directory.
2021-01-12 07:11:02.665 | Generating grub configuration file ...
2021-01-12 07:11:03.112 | /usr/bin/grub2-editenv: error: cannot rename the file /boot/grub2/grubenv.new to /boot/grub2/grubenv: No such file or directory.
...
[1]: https://review.opendev.org/c/openstack/diskimage-builder/+/750279
Closes-Bug: 1911120
Change-Id: I2de5444f7e1a145df9abb03fa4c367e8bb914e03
This fixes below building error When build centos with element
dracut-regenerate on ubuntu by specifying the initrd and kernel version.
...
2021-01-12 03:44:15.758 | dracut: Cannot find module directory /lib/modules/5.4.0-58-generic/
2021-01-12 03:44:15.759 | dracut: and --no-kernel was not specified
2021-01-12 03:44:15.765 | Traceback (most recent call last):
2021-01-12 03:44:15.766 | File "/tmp/in_target.d/finalise.d/50-dracut-regenerate", line 102, in <module>
2021-01-12 03:44:15.766 | main()
2021-01-12 03:44:15.766 | File "/tmp/in_target.d/finalise.d/50-dracut-regenerate", line 82, in main
2021-01-12 03:44:15.766 | raise e
2021-01-12 03:44:15.766 | subprocess.CalledProcessError: Command '['dracut', '--list-modules']' returned non-zero exit status 1.
...
Change-Id: I87ae20b3fa2f291bb107e607137fcd7b1c0a4996
While looking at DIB logs, it is very hard to debug dracut
issues due to missing modules, listing dracut modules will
give an insight.
Related-Bug: #1907457
Signed-off-by: Chandan Kumar (raukadah) <chkumar@redhat.com>
Change-Id: I4d277bdfd648adba5a749d22d905c66b807e249a
With Centos 8.3, centos-repos package has been replaced by
other packages [1].
[1] https://lists.centos.org/pipermail/centos-devel/2020-September/056069.html
Also Increase flake8 and pyflakes version in lower-constraints.txt as
this was already broken.
Change-Id: Ife139fcaff0c2d944098ea353259971d2d3f18b8
The git package pulls in heavy dependencies for just a few rarely
used commands. git-core should be enough (and already uses for Suse).
Change-Id: I96b71072c22c26b3b651466053b9e9561527cbe5
Modern distros use more creative interface naming, e.g. CentOS 8
adds both eth0 and ens3 (!). Remove everything.
Change-Id: Ibdebdb09ea790787840cf9b817d4eb549ef18249
Grub2 messes up arguments with double quotes in them, changing from
key="value" to "key=value" [1]. Support this format as well.
[1] https://bugzilla.redhat.com/show_bug.cgi?id=950760
Change-Id: Icf4c9f86009d29a342d6e0e21644af680066b0b2
Autoremove can touch packages, that are assumed to be present
for post-install step.
A good example is dkms, which is dropped in
install.d, but dkms element try to use it in post-install step.
This results in the following log [1]
[1] http://paste.openstack.org/show/797809/
Change-Id: I635af230c6b250fee273039935cf19506e83b3d1
This is never called externally to dib, so doesn't need to be an
entrypoint. Call it from within dib using the running python
executable and from the lib/ directory; this means we do not need to
have the virtualenv activated to run disk-image-create.
Change-Id: Ie9b551824792864402b0c63ccc350dc5c92dcc3f
This is really an internal dib tool. Move it to the lib directory,
and call it with the python we are running under.
This is one less reason to require the virtualenv to be activated when
you run 'disk-image-create'.
Change-Id: Id689683a0b1fdcb446b04ba967284a216133d743
Provide ability to run dhcp client on VLAN interfaces created on
top of an Ethernet interface. See also
<https://storyboard.openstack.org/#!/story/2008298>`__ for further details.
Change-Id: Ic3ffd7b8e23b1e996cfe6c79ce0ff47e521f30be
centos-repos package is not available on centos-8-stream:
2020-11-02 15:35:20.962 | No Match for argument centos-repos
2020-11-02 15:35:20.992 | No Match for argument centos-release-stream
The build fails later on install_pkg_manager sed command:
2020-11-02 15:35:22.903 | sed: can't read
/var/cache/nodepool/dib_tmp/dib_build.iPSHptNW/mnt/etc/yum.repos.d/*repo: No
such file or directory
This change ensure centos-stream-repos package is installed to provide
/etc/yum.repos.d/*repo files
This change replace 'centos-release-stream' package with the new
'centos-stream-release' package [1]
[1] https://lists.centos.org/pipermail/centos-devel/2020-September/056069.html
Change-Id: I6c397bf7b5797a02e5f006c18ee63c9cdf66b38c
We are running yum-config-manager/dnf config-manager in the epel
element. Even though the yum-utils package is declared in the
package-installs.yaml file, the package-installs pre-install.d
script is executed after the one in the epel element, so image
build fails.
This commit ensures yum-utils or dnf-plugins-core are installed
before running the command.
Change-Id: Ib292b0b2b31bd966e0c5e8f2b2ce560bba89c45c
Initial patch for I78d7bcf214a45245e2073428120fcbdd968e1acd
works without the envvar set, however, 'set -eu' causes it to break
if unset.
This makes the module configuration variable not required to be
set, consistent with other DIB scripts.
Change-Id: I5ca80f518d0371a18c107c061dc923876463af57
Signed-off-by: Lon Hohberger <lhh@redhat.com>
This disables growpart module in cloud-init, not resizing / partition
to maximum disk free size by default.
Change-Id: I69984a9141fa8abb12dc5d51bd334f9280deca67
Originally it was added for missing python-cheetah dep
for openstack-nova. Nova has removed usage of it long
ago with [1]. rhel-7-server-rh-common-rpms should be
disabled once it's usage is over as it packages from
it can conflict with other openstack repos.
If some package which is needed by OpenStack Packages
is missing then instead of adding rhel-7-server-rh-common-rpms
repo consideration should be to add it in RDO.
[1] https://review.opendev.org/#/c/40205
This reverts commit c7219a5a60.
Change-Id: Iad3a1c353c10bb35f9c9ef4076b65f5c84b803b2
This file is present in both dpkg and debian-minimal element,
causing a failure to build anything with debian.
Change-Id: I8213d581a79bb432281f31955a44418e4047d9e1
Some OpenStack releases on RHEL require specific modules
in order to function correctly. This adds the ability
to set DIB_DNF_MODULE_STREAMS which then are selected
prior to package installation.
Change-Id: I78d7bcf214a45245e2073428120fcbdd968e1acd
Signed-off-by: Lon Hohberger <lhh@redhat.com>
