Commit Graph

3965 Commits

Author SHA1 Message Date
Ian Wienand
eed1be9992 dib-lint: match text/x-script.python
It seems libmagic changed it's Python output to
"text/x-script.python", which I see on Fedora.  Handle this too so we
detect Python files correctly.

[1] eb373e431c

Change-Id: I35992c70523a8f2bc5efff2e5167ed1ac1514d34
2021-05-10 14:29:51 +10:00
Zuul
b536dbba8e Merge "Add containerfile element" 2021-05-10 01:50:40 +00:00
Zuul
c3243be696 Merge "Install epel-release from URL" 2021-05-10 01:28:32 +00:00
James E. Blair
edff632186 Add containerfile element
This is a base element which uses a containerfile (Dockerfile) to
build a container image, then the filesystem is extracted from that
image and forms the root of the dib image.

You can add as little or as much to the dockerfile as desired.

Change-Id: I4e821aa2ce7feb8841ef31da56de1a31aa9218b5
2021-05-07 13:54:59 +10:00
YuehuiLei
661bb85b78 setup.cfg: Replace dashes with underscores
Setuptools v54.1.0 introduces a warning that the use of dash-separated
options in 'setup.cfg' will not be supported in a future version [1].
Get ahead of the issue by replacing the dashes with underscores. Without
this, we see 'UserWarning' messages like the following on new enough
versions of setuptools:

  UserWarning: Usage of dash-separated 'description-file' will not be
  supported in future versions. Please use the underscore name
  'description_file' instead

[1] https://github.com/pypa/setuptools/commit/a2e9ae4cb

Change-Id: I48d280192d3713e09571842d5515da2dcc637d08
2021-05-05 10:11:58 +08:00
Dmitry Tantsur
34de4ebd6e Update the ironic jobs
The iSCSI deploy interface will be removed in Xena. Stop testing it and
use a newer alias for the other job.

Change-Id: Ib50b8fd0aff1da9e51e63c2ad6e9f9361161301a
2021-04-30 17:14:49 +02:00
Zuul
cb0c117659 Merge "debian-minimal: bullseye: /updates -> -security" 2021-04-30 09:25:35 +00:00
Zuul
231770434b Merge "debian-minimal: Set bullseye version" 2021-04-30 05:59:26 +00:00
Zuul
0003de5197 Merge "Collect openstack logs" 2021-04-30 05:57:48 +00:00
Zuul
46fb885ed8 Merge "Fix centos stream set mirror" 2021-04-30 05:21:23 +00:00
Jeremy Stanley
50b1566fa4 debian-minimal: bullseye: /updates -> -security
With the release of Debian bullseye and later, security updates are
provided in the bullseye-security suite instead of bullseye/updates.

Change-Id: I63580ec96a53e5e8ef8d105e766d838029727917
2021-04-28 17:07:22 +00:00
Ian Wienand
3071457355 debian-minimal: Set bullseye version
Currently Debian sets /etc/debian_version to "bullseye/sid" and, due
to a series of issues explained in [1] more fully "lsb_release -c" in
the OpenDev environment doesn't return the distribution code name.
Overriding this to the final release version fixes this.

[1] http://lists.opendev.org/pipermail/service-discuss/2021-April/000222.html

Change-Id: I00c1741dac6ad5f2c4bf855a207f17d8985bc763
2021-04-28 09:48:22 -07:00
Clark Boylan
6fc3a4c075 Collect openstack logs
This will aid debugging of nova 500 errors when doing the trivial boots
of cirros to validate the cloud is functional.

Change-Id: Idd6d406e3a5a6ea5dd046c61c391c8f6147a9168
2021-04-28 09:48:22 -07:00
Ian Wienand
9fd4b4bc80 Remove dib-nodepool-functional-openstack-ubuntu-bionic
This test hasn't been running.  However, it doesn't quite do what we
want -- it installs nodepool from the Zuul checkout; this means we are
not testing DIB against the latest released nodepool with this job.
We'll have to rethink it; let's remove this for the moment.

Change-Id: I71814afcc405318f07292bd1cb450369cf9c7b28
2021-04-28 14:57:17 +10:00
Zuul
280d9232c6 Merge "dib-run-parts: stop leaving PROFILE_DIR behind" 2021-04-26 07:00:04 +00:00
Zuul
d03e7f1327 Merge "Ensure redhat efi packages are reinstalled during finalise" 2021-04-26 01:08:04 +00:00
Clark Boylan
da674c4e5b Install pbr before glean to address SNI issues
Some older distros (like centos8 and xenial) don't support SNI in their
easy_install implementations which are used to install setup_requires
for python packages. PBR is a setup_requires for glean. We work around
this problem when installing glean by preinstalling PBR with pip.

Change-Id: Ie9f5c9ed06954cbe51f23fe8cca0655a931a5201
2021-04-23 15:04:26 -07:00
Ian Wienand
7c8e9f8733 dib-run-parts: stop leaving PROFILE_DIR behind
When a build fails, we can exit and leave ${PROFILE_DIR} behind.  Make
sure this is cleaned up with an exit trap.

While we're adding a function, update the syntax of the others for
consistency.

Change-Id: I14499b5ebaaa30126aaa6b3d1bd86ed64f110fda
2021-04-21 10:42:30 +10:00
Steve Baker
5caeba0c68 Ensure redhat efi packages are reinstalled during finalise
The rhel-8.4 qcow2 base image already has the grub2-efi-x64 package
installed on its single partition which has files installed to
/boot/efi..., however a partitioned image will have an empty /boot/efi
partition when running 50-bootloader. This means dnf will not install
grub2-efi-x64 when requested and /boot/efi will remain empty.

This commit makes the following changes:
- Refactors redhat bootloader pkg-map for the following:
  - Make x86_64/amd64, arm64/aarch64 adjancent so they don't diverge
  - Map grub-efi to packages installed to /usr
  - Map grub-efi-{arch} to packages installed to /boot/efi
- Removes packages grub-efi-{arch} before installing grub-efi and
  grub-efi-{arch}

Change-Id: Ia197feea34f43bd870fed30829b740596e6b2f48
2021-04-21 10:56:37 +12:00
Zuul
d7becbeb2b Merge "Add Debian Bullseye Zuul job" 2021-04-12 13:16:23 +00:00
Dmitriy Rabotyagov
cb4e9fc072 Add Debian Bullseye Zuul job
Change-Id: I7d7994565ab7ed62e49efd80766fe19a906499db
2021-04-09 10:20:10 +03:00
Zuul
833d433e92 Merge "Improved the documentation for DIB_DNF_MODULE_STREAMS" 2021-04-08 10:34:30 +00:00
Zuul
cb96117146 Merge "Fix: IPA image buidling with OpenSuse." 2021-04-08 07:05:22 +00:00
Zuul
ffe3aa1610 Merge "Properly set grub2 root device when using efi" 2021-04-08 07:05:17 +00:00
Chandan Kumar (raukadah)
21a752ee4d Improved the documentation for DIB_DNF_MODULE_STREAMS
https://review.opendev.org/c/openstack/diskimage-builder/+/785138
adds the support for DIB_DNF_MODULE_STREAMS which is now available
for all Yum based distros.

This patch enhances the docs for using it for all Yum
based distributions.

Signed-off-by: Chandan Kumar (raukadah) <chkumar@redhat.com>
Change-Id: I29e726679c2b675b3c0cd95a3ff48fdad7cd5431
2021-04-08 11:22:43 +05:30
Clark Boylan
3294aecca2 Properly set grub2 root device when using efi
We've noticed that centos8 arm64 images have a root devices of
/dev/mapper/loop7p3 which make sense within a dib image build context
but not at boot time. Dib intends to use labels to set the root device
but when efi is used we end up running grub2-mkconfig against the efi
grub config path before we configure grub to use labels.

Fix this by running grub2-mkconfig after its configuration is set.
This should avoid confusion and complicated paths through the scripts
that configure this for us. We then copy the resulting config to the efi
specific grub.cfg location for platforms that have it.

There is also a small refactoring that is done to try and make the ~3
boot variants more clear:

 1) Booting with legacy bios
 2) Booting with uefi without a signed shim that directly calls grub
 3) Booting with uefi and a signed shim that calls grub

Options 1 and 2 share the /boot/grub*/grub.cfg file. Option 3 needs its
grub.cfg to live alongside distro specific efi target.

Change-Id: Ie9790da9d1bbea58197b37b15a48e77f8a93c1ac
2021-04-07 15:46:10 -07:00
Chandan Kumar (raukadah)
ced54fea75 Make DIB_DNF_MODULE_STREAMS part of yum element
While building cloud images, it is common to set modules
for CentOS and RHEL images. Earlier it was part of rhel-common
which was specific to RHEL OS not for CentOS. Moving it
under yum element as module/stream can be enabled or disabled
via dnf itself.

Signed-off-by: Chandan Kumar (raukadah) <chkumar@redhat.com>
Change-Id: Idc0f277f97e92e4d003f059f01b59f1b5513da34
2021-04-07 16:06:09 +05:30
Xinliang Liu
8c86d876e3 Fix centos stream set mirror
This intents to fix job dib-functests-bionic-python3-image.
And no CentOS-Stream-centosplus.repo file[1].

[1]: http://rpm.pbone.net/info_idpl_72967298_distro_centosother_com_centos-stream-repos-8-2.el8.noarch.rpm.html

Change-Id: I9d69413f31d0a9d83e992d05d177f683b7361337
2021-04-02 06:56:45 +00:00
smoshiur1237
93d11a7cf8
Fix: IPA image buidling with OpenSuse.
At this moment the IPA image building with OpenSuse is broken and here, it was failing during the release check for Opensue because etc/SuSE-release is not valid anymore and deprecated for openSuse. Its renamed to /etc/os-release for openSuse rlease 15. This PR will solve the issue to build IPA image with OpenSuse base image. There is another PR opened in ironic-python-agent-builder, which adds all the missing packages, setuptools upgrade and svc mapping to do the build successful.
https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/778726

Bug-Report: https://bugs.launchpad.net/diskimage-builder/+bug/1921510

Change-Id: Id2759be29bfcbf2ecf1ce67e171686924b506b1a
2021-03-29 13:46:04 +03:00
Matthew Thode
b4f768117f
update gentoo keywords to support gcc-10
open-iscsi and open-isns need keywording to support gcc-10, move it out
of being keyworded only for musl profiles.

remove unneeded keywords for python-exec and python-exec-conf (marked
stable)

use the full package name for the dev-lang/python-exec-conf package

Change-Id: I44eaf8c2230e9e2089a72fce46954f4336626843
Signed-off-by: Matthew Thode <mthode@mthode.org>
2021-03-18 23:24:31 -05:00
Zuul
4aae99e64e Merge "Change paths for bootloader files in iso element" 2021-03-16 09:56:07 +00:00
Zuul
0e5922c6b8 Merge "replace the link which is in the 06-hpdsa file" 2021-03-16 06:34:40 +00:00
RotanChen
a9386ba147 replace the link which is in the 06-hpdsa file
The old link does't work,this one does.

Change-Id: I128b5841b2bd4897e2b2c0e82ad31049ce7b0c29
2021-03-12 19:16:50 +08:00
Steve Baker
522113bc6c Only add rhel base repos when REG_REPOS is not set
For offline (satellite based) installs the base repos won't be
available and the base packages will come from a different named repo
in satellite which will be specified by REG_REPOS.

This change will ensure no base repos are added when REG_REPOS are
specified so offline image builds are possible. All required base
repos need to be added to REG_REPOS when it is used. Documentation[1]
already includes base repos, so this should not be disruptive.

[1] https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/16.1/html/director_installation_and_usage/creating-whole-disk-images#disk-image-environment-variables

Change-Id: Iafb81d50dffdac40d3b011670200b8da4c3a58f0
2021-03-11 11:01:29 +13:00
Steve Baker
27a326dafb Support secure-boot bootloader where possible
As of grub2 >= 2.02-95 on redhat family distros, calling grub2-install
on an EFI partition will fail with: "this utility cannot be used for
EFI platforms because it does not support UEFI Secure Boot."

This version of grub is now in centos8-stream and non-eus repos of
RHEL-8. It is not currently possible to build whole-disk UEFI images
on these distros, and when this package is promoted this will also
affect centos8 and RHEL-8 eus. The grub maintainers made this change
because the grub2-install generated /boot/efi/EFI/BOOT/BOOTX64.EFI
will never be capable of booting with Secure Boot.

This change defines a $EFI_BOOT_DIR for every distro element. When
directory /boot/efi/$EFI_BOOT_DIR exists a grub.cfg file in will be
generated there. This change also installs the shim package on redhat
family distros, which installs a copy of the shim bootloader to
/boot/efi/EFI/BOOT/BOOTX64.EFI. Using centos as an example, this
allows UEFI to boot the shim /boot/efi/EFI/BOOT/BOOTX64.EFI which
then chains to /boot/efi/EFI/centos/grubx64.efi.

If /boot/efi/$EFI_BOOT_DIR doesn't exist (such as for Ubuntu,
/boot/efi/EFI/ubuntu) the current behaviour of running grub-install to
generate /boot/efi/EFI/BOOT/BOOTX64.EFI will continue. For distros
such as Ubutnu where packaging does not populate /boot/efi/EFI/ubuntu
with .efi files, secure boot can be added in the future by copying
.efi files to /boot/efi/EFI/ubuntu and copying the shim file to
/boot/efi/EFI/BOOT/BOOTX64.EFI.

Change-Id: I90925218ff2aa4c4daffcf86e686b6d98d6b0f21
2021-03-11 10:27:59 +13:00
Zuul
a0fd571a3c Merge "Add efibootmgr utility for UEFI boot menu management" 2021-03-09 02:11:10 +00:00
Zuul
6a99e314b4 Merge "Use the same bootloader pkg-map for all redhat family" 2021-03-09 02:00:11 +00:00
Zuul
739b529a33 Merge "Add aarch64 support for rhel" 2021-03-09 01:52:31 +00:00
Zuul
b91aae61f7 Merge "Don't use hardcode while override base image file" 2021-03-09 00:03:09 +00:00
Zuul
853e9d7235 Merge "Remove fedora-31 testing" 2021-03-08 23:59:15 +00:00
Zuul
3b5f5b55b1 Merge "Fix hooks order for CentOS/Fedora when mirror used" 2021-03-08 23:51:12 +00:00
Zuul
4c5b1179f3 Merge "Fix installation of proliant tools" 2021-03-08 23:47:32 +00:00
Daniel Pawlik
8b0e2417a6 Change get-pip url
The path for get-pip.py script in version 3.5 has been changed
with this commit [1].

[1] 2360f025eb

Change-Id: Ifde16e40b4e241c6c4c93df44330c403ee903e6f
2021-03-08 09:19:28 +01:00
Xinliang Liu
0c51b49414 Add aarch64 support for rhel
Change-Id: I86ccc56e37b214a45ba620b731b51f58d73471f8
2021-03-08 07:00:15 +00:00
Steve Baker
69bf654eba Add efibootmgr utility for UEFI boot menu management
As of grub2 >= 2.02-95 calling grub2-install on an EFI partition will
fail with: "this utility cannot be used for EFI platforms because it
does not support UEFI Secure Boot." In the case of ironic deployments,
ironic-python-agent will call efibootmgr to set the local boot for
subsequent boots.

This change adds efibootmgr to the image as well so any other UEFI
menu changes can be made manually.

Change-Id: I765ded15da07d6227d1e337960e54ad0e0d6ca39
2021-03-05 10:00:49 +13:00
Maksim Malchuk
c4c21967d8 Fix hooks order for CentOS/Fedora when mirror used
The python3/python3-pyyaml packages both are never installed and dnf
itself never updated when $DIB_DISTRIBUTION_MIRROR set and used.

This change fix the order of the operations:
 1. yum/dnf configure.
 2. *.repo patching.
 3. yum/dnf update/install execution.

Change-Id: Ifbbf1f0190fe8c8a77fb3be820e8056447e755f6
Signed-off-by: Maksim Malchuk <maksim.malchuk@gmail.com>
2021-03-04 10:54:52 +00:00
Maksim Malchuk
ca83a4c3cb Don't use hardcode while override base image file
The trvial fix allow override to work in air-gapped envirments where
the command 'curl -s https://cloud.centos.org/...' would fail.

Change-Id: I84296d8816042e4cd4cb02f15746b86d600d13d6
Signed-off-by: Maksim Malchuk <maksim.malchuk@gmail.com>
2021-03-04 10:54:32 +00:00
Francois RIGAULT
5efb11de0d Fix installation of proliant tools
Use the actual virtualenv to install proliant tools, and make sure the
dependencies needed for sum firmware upgrade are included.

Change-Id: Ie1dbb868450918567b2903cdeccda35af1904417
Closes-Bug: #1916346
2021-03-03 19:04:50 +01:00
Mateusz Kowalski
3b22ee8784
Change paths for bootloader files in iso element
This PR updates locations for files used by the bootloader depending on
the target operating system built. The current logic does not take into
account latest versions of operating systems and makes it impossible to
build ISOs against those.

With this change it is possible to correctly build CentOS 8, Ubuntu
18.04 and Ubuntu 20.04 images.

Closes-Bug: #1916913
Change-Id: I3ed0041640f539e82805d03ba26fe46217f3ac3c
2021-03-03 11:01:14 +01:00
Steve Baker
f7ba7ba1de Use the same bootloader pkg-map for all redhat family
The only difference between the rhel and redhat entries is rhel has
the extra grub-efi-x86_64 mapping. All redhat family releases would
benefit from having this too, so this change removes the whole rhel
entry and adds grub-efi-x86_64 to the redhat family.

The assumption is that anything which applies to rhel also applies to
centos-stream, and in this case doesn't harm centos or fedora either.

Change-Id: I0dc44c1f2b57516742f4c3e43cfc8874d6b90fa2
2021-03-03 13:24:15 +13:00