Commit Graph

4004 Commits

Author SHA1 Message Date
Clark Boylan
6fc3a4c075 Collect openstack logs
This will aid debugging of nova 500 errors when doing the trivial boots
of cirros to validate the cloud is functional.

Change-Id: Idd6d406e3a5a6ea5dd046c61c391c8f6147a9168
2021-04-28 09:48:22 -07:00
Ian Wienand
9fd4b4bc80 Remove dib-nodepool-functional-openstack-ubuntu-bionic
This test hasn't been running.  However, it doesn't quite do what we
want -- it installs nodepool from the Zuul checkout; this means we are
not testing DIB against the latest released nodepool with this job.
We'll have to rethink it; let's remove this for the moment.

Change-Id: I71814afcc405318f07292bd1cb450369cf9c7b28
2021-04-28 14:57:17 +10:00
Zuul
280d9232c6 Merge "dib-run-parts: stop leaving PROFILE_DIR behind" 2021-04-26 07:00:04 +00:00
Zuul
d03e7f1327 Merge "Ensure redhat efi packages are reinstalled during finalise" 2021-04-26 01:08:04 +00:00
Clark Boylan
da674c4e5b Install pbr before glean to address SNI issues
Some older distros (like centos8 and xenial) don't support SNI in their
easy_install implementations which are used to install setup_requires
for python packages. PBR is a setup_requires for glean. We work around
this problem when installing glean by preinstalling PBR with pip.

Change-Id: Ie9f5c9ed06954cbe51f23fe8cca0655a931a5201
2021-04-23 15:04:26 -07:00
Ian Wienand
7c8e9f8733 dib-run-parts: stop leaving PROFILE_DIR behind
When a build fails, we can exit and leave ${PROFILE_DIR} behind.  Make
sure this is cleaned up with an exit trap.

While we're adding a function, update the syntax of the others for
consistency.

Change-Id: I14499b5ebaaa30126aaa6b3d1bd86ed64f110fda
2021-04-21 10:42:30 +10:00
Steve Baker
5caeba0c68 Ensure redhat efi packages are reinstalled during finalise
The rhel-8.4 qcow2 base image already has the grub2-efi-x64 package
installed on its single partition which has files installed to
/boot/efi..., however a partitioned image will have an empty /boot/efi
partition when running 50-bootloader. This means dnf will not install
grub2-efi-x64 when requested and /boot/efi will remain empty.

This commit makes the following changes:
- Refactors redhat bootloader pkg-map for the following:
  - Make x86_64/amd64, arm64/aarch64 adjancent so they don't diverge
  - Map grub-efi to packages installed to /usr
  - Map grub-efi-{arch} to packages installed to /boot/efi
- Removes packages grub-efi-{arch} before installing grub-efi and
  grub-efi-{arch}

Change-Id: Ia197feea34f43bd870fed30829b740596e6b2f48
2021-04-21 10:56:37 +12:00
Zuul
d7becbeb2b Merge "Add Debian Bullseye Zuul job" 2021-04-12 13:16:23 +00:00
Dmitriy Rabotyagov
cb4e9fc072 Add Debian Bullseye Zuul job
Change-Id: I7d7994565ab7ed62e49efd80766fe19a906499db
2021-04-09 10:20:10 +03:00
Zuul
833d433e92 Merge "Improved the documentation for DIB_DNF_MODULE_STREAMS" 2021-04-08 10:34:30 +00:00
Zuul
cb96117146 Merge "Fix: IPA image buidling with OpenSuse." 2021-04-08 07:05:22 +00:00
Zuul
ffe3aa1610 Merge "Properly set grub2 root device when using efi" 2021-04-08 07:05:17 +00:00
Chandan Kumar (raukadah)
21a752ee4d Improved the documentation for DIB_DNF_MODULE_STREAMS
https://review.opendev.org/c/openstack/diskimage-builder/+/785138
adds the support for DIB_DNF_MODULE_STREAMS which is now available
for all Yum based distros.

This patch enhances the docs for using it for all Yum
based distributions.

Signed-off-by: Chandan Kumar (raukadah) <chkumar@redhat.com>
Change-Id: I29e726679c2b675b3c0cd95a3ff48fdad7cd5431
2021-04-08 11:22:43 +05:30
Clark Boylan
3294aecca2 Properly set grub2 root device when using efi
We've noticed that centos8 arm64 images have a root devices of
/dev/mapper/loop7p3 which make sense within a dib image build context
but not at boot time. Dib intends to use labels to set the root device
but when efi is used we end up running grub2-mkconfig against the efi
grub config path before we configure grub to use labels.

Fix this by running grub2-mkconfig after its configuration is set.
This should avoid confusion and complicated paths through the scripts
that configure this for us. We then copy the resulting config to the efi
specific grub.cfg location for platforms that have it.

There is also a small refactoring that is done to try and make the ~3
boot variants more clear:

 1) Booting with legacy bios
 2) Booting with uefi without a signed shim that directly calls grub
 3) Booting with uefi and a signed shim that calls grub

Options 1 and 2 share the /boot/grub*/grub.cfg file. Option 3 needs its
grub.cfg to live alongside distro specific efi target.

Change-Id: Ie9790da9d1bbea58197b37b15a48e77f8a93c1ac
2021-04-07 15:46:10 -07:00
Chandan Kumar (raukadah)
ced54fea75 Make DIB_DNF_MODULE_STREAMS part of yum element
While building cloud images, it is common to set modules
for CentOS and RHEL images. Earlier it was part of rhel-common
which was specific to RHEL OS not for CentOS. Moving it
under yum element as module/stream can be enabled or disabled
via dnf itself.

Signed-off-by: Chandan Kumar (raukadah) <chkumar@redhat.com>
Change-Id: Idc0f277f97e92e4d003f059f01b59f1b5513da34
2021-04-07 16:06:09 +05:30
Riccardo Pittau
256c798bc4 Convert multi line if statement to case
Having multi-line if-elif statements is not ideal, case improves
readability clarifying the code.

Change-Id: I3383584e09763d4ae8eab2f36a93ee399dae8382
2021-04-07 07:17:19 +00:00
Xinliang Liu
8c86d876e3 Fix centos stream set mirror
This intents to fix job dib-functests-bionic-python3-image.
And no CentOS-Stream-centosplus.repo file[1].

[1]: http://rpm.pbone.net/info_idpl_72967298_distro_centosother_com_centos-stream-repos-8-2.el8.noarch.rpm.html

Change-Id: I9d69413f31d0a9d83e992d05d177f683b7361337
2021-04-02 06:56:45 +00:00
smoshiur1237
93d11a7cf8
Fix: IPA image buidling with OpenSuse.
At this moment the IPA image building with OpenSuse is broken and here, it was failing during the release check for Opensue because etc/SuSE-release is not valid anymore and deprecated for openSuse. Its renamed to /etc/os-release for openSuse rlease 15. This PR will solve the issue to build IPA image with OpenSuse base image. There is another PR opened in ironic-python-agent-builder, which adds all the missing packages, setuptools upgrade and svc mapping to do the build successful.
https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/778726

Bug-Report: https://bugs.launchpad.net/diskimage-builder/+bug/1921510

Change-Id: Id2759be29bfcbf2ecf1ce67e171686924b506b1a
2021-03-29 13:46:04 +03:00
Matthew Thode
b4f768117f
update gentoo keywords to support gcc-10
open-iscsi and open-isns need keywording to support gcc-10, move it out
of being keyworded only for musl profiles.

remove unneeded keywords for python-exec and python-exec-conf (marked
stable)

use the full package name for the dev-lang/python-exec-conf package

Change-Id: I44eaf8c2230e9e2089a72fce46954f4336626843
Signed-off-by: Matthew Thode <mthode@mthode.org>
2021-03-18 23:24:31 -05:00
Zuul
4aae99e64e Merge "Change paths for bootloader files in iso element" 2021-03-16 09:56:07 +00:00
Zuul
0e5922c6b8 Merge "replace the link which is in the 06-hpdsa file" 2021-03-16 06:34:40 +00:00
RotanChen
a9386ba147 replace the link which is in the 06-hpdsa file
The old link does't work,this one does.

Change-Id: I128b5841b2bd4897e2b2c0e82ad31049ce7b0c29
2021-03-12 19:16:50 +08:00
Steve Baker
522113bc6c Only add rhel base repos when REG_REPOS is not set
For offline (satellite based) installs the base repos won't be
available and the base packages will come from a different named repo
in satellite which will be specified by REG_REPOS.

This change will ensure no base repos are added when REG_REPOS are
specified so offline image builds are possible. All required base
repos need to be added to REG_REPOS when it is used. Documentation[1]
already includes base repos, so this should not be disruptive.

[1] https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/16.1/html/director_installation_and_usage/creating-whole-disk-images#disk-image-environment-variables

Change-Id: Iafb81d50dffdac40d3b011670200b8da4c3a58f0
2021-03-11 11:01:29 +13:00
Steve Baker
27a326dafb Support secure-boot bootloader where possible
As of grub2 >= 2.02-95 on redhat family distros, calling grub2-install
on an EFI partition will fail with: "this utility cannot be used for
EFI platforms because it does not support UEFI Secure Boot."

This version of grub is now in centos8-stream and non-eus repos of
RHEL-8. It is not currently possible to build whole-disk UEFI images
on these distros, and when this package is promoted this will also
affect centos8 and RHEL-8 eus. The grub maintainers made this change
because the grub2-install generated /boot/efi/EFI/BOOT/BOOTX64.EFI
will never be capable of booting with Secure Boot.

This change defines a $EFI_BOOT_DIR for every distro element. When
directory /boot/efi/$EFI_BOOT_DIR exists a grub.cfg file in will be
generated there. This change also installs the shim package on redhat
family distros, which installs a copy of the shim bootloader to
/boot/efi/EFI/BOOT/BOOTX64.EFI. Using centos as an example, this
allows UEFI to boot the shim /boot/efi/EFI/BOOT/BOOTX64.EFI which
then chains to /boot/efi/EFI/centos/grubx64.efi.

If /boot/efi/$EFI_BOOT_DIR doesn't exist (such as for Ubuntu,
/boot/efi/EFI/ubuntu) the current behaviour of running grub-install to
generate /boot/efi/EFI/BOOT/BOOTX64.EFI will continue. For distros
such as Ubutnu where packaging does not populate /boot/efi/EFI/ubuntu
with .efi files, secure boot can be added in the future by copying
.efi files to /boot/efi/EFI/ubuntu and copying the shim file to
/boot/efi/EFI/BOOT/BOOTX64.EFI.

Change-Id: I90925218ff2aa4c4daffcf86e686b6d98d6b0f21
2021-03-11 10:27:59 +13:00
Zuul
a0fd571a3c Merge "Add efibootmgr utility for UEFI boot menu management" 2021-03-09 02:11:10 +00:00
Zuul
6a99e314b4 Merge "Use the same bootloader pkg-map for all redhat family" 2021-03-09 02:00:11 +00:00
Zuul
739b529a33 Merge "Add aarch64 support for rhel" 2021-03-09 01:52:31 +00:00
Zuul
b91aae61f7 Merge "Don't use hardcode while override base image file" 2021-03-09 00:03:09 +00:00
Zuul
853e9d7235 Merge "Remove fedora-31 testing" 2021-03-08 23:59:15 +00:00
Zuul
3b5f5b55b1 Merge "Fix hooks order for CentOS/Fedora when mirror used" 2021-03-08 23:51:12 +00:00
Zuul
4c5b1179f3 Merge "Fix installation of proliant tools" 2021-03-08 23:47:32 +00:00
Daniel Pawlik
8b0e2417a6 Change get-pip url
The path for get-pip.py script in version 3.5 has been changed
with this commit [1].

[1] 2360f025eb

Change-Id: Ifde16e40b4e241c6c4c93df44330c403ee903e6f
2021-03-08 09:19:28 +01:00
Xinliang Liu
0c51b49414 Add aarch64 support for rhel
Change-Id: I86ccc56e37b214a45ba620b731b51f58d73471f8
2021-03-08 07:00:15 +00:00
Steve Baker
69bf654eba Add efibootmgr utility for UEFI boot menu management
As of grub2 >= 2.02-95 calling grub2-install on an EFI partition will
fail with: "this utility cannot be used for EFI platforms because it
does not support UEFI Secure Boot." In the case of ironic deployments,
ironic-python-agent will call efibootmgr to set the local boot for
subsequent boots.

This change adds efibootmgr to the image as well so any other UEFI
menu changes can be made manually.

Change-Id: I765ded15da07d6227d1e337960e54ad0e0d6ca39
2021-03-05 10:00:49 +13:00
Maksim Malchuk
c4c21967d8 Fix hooks order for CentOS/Fedora when mirror used
The python3/python3-pyyaml packages both are never installed and dnf
itself never updated when $DIB_DISTRIBUTION_MIRROR set and used.

This change fix the order of the operations:
 1. yum/dnf configure.
 2. *.repo patching.
 3. yum/dnf update/install execution.

Change-Id: Ifbbf1f0190fe8c8a77fb3be820e8056447e755f6
Signed-off-by: Maksim Malchuk <maksim.malchuk@gmail.com>
2021-03-04 10:54:52 +00:00
Maksim Malchuk
ca83a4c3cb Don't use hardcode while override base image file
The trvial fix allow override to work in air-gapped envirments where
the command 'curl -s https://cloud.centos.org/...' would fail.

Change-Id: I84296d8816042e4cd4cb02f15746b86d600d13d6
Signed-off-by: Maksim Malchuk <maksim.malchuk@gmail.com>
2021-03-04 10:54:32 +00:00
Francois RIGAULT
5efb11de0d Fix installation of proliant tools
Use the actual virtualenv to install proliant tools, and make sure the
dependencies needed for sum firmware upgrade are included.

Change-Id: Ie1dbb868450918567b2903cdeccda35af1904417
Closes-Bug: #1916346
2021-03-03 19:04:50 +01:00
Mateusz Kowalski
3b22ee8784
Change paths for bootloader files in iso element
This PR updates locations for files used by the bootloader depending on
the target operating system built. The current logic does not take into
account latest versions of operating systems and makes it impossible to
build ISOs against those.

With this change it is possible to correctly build CentOS 8, Ubuntu
18.04 and Ubuntu 20.04 images.

Closes-Bug: #1916913
Change-Id: I3ed0041640f539e82805d03ba26fe46217f3ac3c
2021-03-03 11:01:14 +01:00
Steve Baker
f7ba7ba1de Use the same bootloader pkg-map for all redhat family
The only difference between the rhel and redhat entries is rhel has
the extra grub-efi-x86_64 mapping. All redhat family releases would
benefit from having this too, so this change removes the whole rhel
entry and adds grub-efi-x86_64 to the redhat family.

The assumption is that anything which applies to rhel also applies to
centos-stream, and in this case doesn't harm centos or fedora either.

Change-Id: I0dc44c1f2b57516742f4c3e43cfc8874d6b90fa2
2021-03-03 13:24:15 +13:00
Steve Baker
5c1f9a3238 Don't install centos-linux-release on 8-stream
This package doesn't exist in the stream base repo, and neither does
centos-linux-repos.

These are presumably replaced by centos-stream-release and
centos-stream-repos. This change adds an else block to handle the
non-stream base packages.

Change-Id: I32249199c3dfa44fc24fba28d24f314112c2e200
2021-02-23 12:57:05 +13:00
Clark Boylan
9ebda01465 Remove fedora-31 testing
Fedora 32 and 33 are current. We can clean up this fedora 31 testing.
A followup change will add fedora-33 jobs.

Change-Id: I84357f9bffc104760fa673035f0d0bd095e9a892
2021-02-18 09:14:40 -08:00
Zuul
cbbcf377d8 Merge "Remove the deprecated ironic-agent element" 2021-02-03 08:45:43 +00:00
Zuul
6cfc94c8bf Merge "Install last stable version of get-pip.py script" 2021-02-03 01:16:13 +00:00
Zuul
d4cbaf12f0 Merge "Fix CentOS Stream 8 base repo in centos element" 2021-02-02 14:10:23 +00:00
Zuul
6ebd8cb974 Merge "simplify updating python versions in gentoo" 2021-02-02 07:28:12 +00:00
Zuul
68c47f91fb Merge "Set eus repositories if REG_RELEASE is set" 2021-02-02 07:24:59 +00:00
David Hill
71449a60ff Set eus repositories if REG_RELEASE is set
Set eus repositories if REG_RELEASE is set instead of the base repos
as the current behavior is to use the non-EUS repositories for RHEL
8.2 deployment which breaks image building for customers.

Change-Id: I8e687b27922c3f6fc3d69794866795ab89ecc346
2021-02-01 09:44:16 -05:00
Daniel Pawlik
383fab59e9 Install last stable version of get-pip.py script
The get-pip script does not work well with Python 2 and
it raise error during the installation [1].

[1] https://github.com/pypa/get-pip/issues/83

Change-Id: I3755c34da313ef647547c6ae18b59cc04c2cdd60
2021-02-01 08:29:04 +01:00
Matthew Thode
4d5ba1497c
simplify updating python versions in gentoo
portage now generates /etc/python-exec/python-exec.conf based on the
order of PYTHON_TARGETS in /etc/portage/make.conf

fixes an issue where ARCH was being detected as amd64 not x86_64
fixes kernel installs (virtual/dist-kernel)

standardizes simple if statements (note, the 'shorthand' method will
pass the exit code back to shell but the 'longhand' does not).

Change-Id: I74041c232bc6ab4d6e67a4ecfaa759aa4a5feb6c
Signed-off-by: Matthew Thode <mthode@mthode.org>
2021-01-27 11:34:09 -06:00
Daniel Pawlik
5da3ec0f06 Install epel-release from URL
The epel-release is not available for RHEL 7, so the
package python3-PyYAML was not installed.

Change-Id: Id81ab9d668fce46349d1abd4fb19cf51a9c1ee17
2021-01-27 12:33:30 +01:00