Commit Graph

1283 Commits

Author SHA1 Message Date
Zuul
3c3ef6e32b Merge "Check and remove existing image interface configurations" 2021-09-10 15:01:28 +00:00
Zuul
48ef1c4d96 Merge "simple-init: allow disabling DHCP fallback" 2021-09-10 14:54:03 +00:00
Zuul
3eae88795d Merge "simple-init: support installing Glean from packages" 2021-09-10 14:54:00 +00:00
Riccardo Pittau
6ed7f3b139 Fix debian-minimal security repos
Debian stable security repos is now stable-security, as well as other
versions.

Move the Debian bullseye job from experimental to non-voting check.

Change-Id: I451cacda6573727de9448b5857bed5181850b4ad
2021-09-10 11:40:48 +02:00
Zuul
f03c50b4a1 Merge "Fix doc typo" 2021-09-09 00:24:00 +00:00
Zuul
8250b33d68 Merge "yum-minimal: use DNF tools on host" 2021-09-09 00:09:56 +00:00
Michal Arbet
db1409f550 Fix cron not installed in debian
This patch adds cron to debian package-installs
as cron should be included in OS.

Change-Id: I50140dc98c240911e769acf2b3c11958ffad5494
2021-09-01 23:49:27 +02:00
Piotr Parczewski
169f755165 Fix doc typo
Change-Id: I5b2729fa874e8e19a9f0dc7b1aed2ce7f5090c97
2021-09-01 19:58:52 +02:00
Ian Wienand
5f47584196 yum-minimal: use DNF tools on host
The latest Debian bullseye release doesn't provide yum any more, only
DNF.  This breaks the minimal builds that are using on-host yum tools
to start the chroot.  Probe for yumdownloader, and if it's not there,
use DNF.

Note this requires "dnf download" which may not be packaged.  See
I21cfbd3935e48be4b92591ea36c7eed301230753 for a sample work-around
that installs this plugin in the nodepool-builder container.

Change-Id: Ia7f1e4d115cc67c378d865d91af94a07b8cdc6cc
2021-08-30 14:14:32 +10:00
Eduardo Santos
442d11b236 Bump Ubuntu release to focal
Change-Id: I01689cfb01b095ef69573a48be55353ea7aa2931
2021-08-26 23:28:33 -03:00
Steve Baker
59420be677 Use non-greedy modifier for SUBRELEASE grep
Some fedora mirrors are serving image icon html, and this grep is
over-matching on those mirrors.

Change-Id: Ibd737f44a8d2eee1902b6c5363c61d591feb75c9
2021-08-23 14:12:17 +12:00
Xinliang Liu
a6ee4d0c21 Introduce openEuler distro
Add openeuler-minimal element and add CI functional tests for both
x86_64 and arm64.

OpenEuler is an open source community driven YUM/DNF distro like
Fedora. It references Fedora and CentOS a lot for the rpm packages
building. So somewhat it can be treated as a redhat family distro
and reuse the YUM/DNF related elements to help build openEuler images.

For more info about openEuler, see: https://openeuler.org/en

Depends-On: https://review.opendev.org/c/zuul/zuul-jobs/+/803413
Change-Id: I3e06e49b524364c3a4edeba8bce7a8c06b9c7b76
2021-08-04 03:06:55 +00:00
Zuul
4f5689a409 Merge "Permit specification of extra bootstrap packages" 2021-08-03 23:13:04 +00:00
Gonéri Le Bouder
168bb25e69 Fedora: bump DIB_RELEASE to 34
Fedora 34 is the new stable release.

Change-Id: I2fba6935064823ace418b00f9735e52640c933f3
2021-08-03 16:37:16 +00:00
Jay Faulkner
91da6ab885 Permit specification of extra bootstrap packages
This change permits the yum-minimal element to be used in downstream
custom distributions, which may have additional packages containing repo
config or GPG keys needed.

This could also be utilized at a later time to move the
distribution-specific logic in this method to each distribution element
separately.

Change-Id: Ic1434bb2fe7301086cf11ba6bd7f2ee187c5e6c8
2021-08-02 11:57:11 -07:00
Zuul
0b14750e3f Merge "Update IRC networks" 2021-07-23 02:06:47 +00:00
Zuul
03275c2dd4 Merge "Add a keylime-agent element and a tpm-emulator element" 2021-07-22 10:34:31 +00:00
Takashi Kajinami
4107dc3bc4 Update IRC networks
The following two channels were migrated to OFTC.
 #tripleo
 #openstack-dib

Also, the following channel was migrated to Libera Chat[1].
 #opensuse-cloud

[1] https://en.opensuse.org/openSUSE:IRC_list

Change-Id: Ia4c729a8d284bbfcbdb3b8621ae29d9be57886f5
2021-07-22 19:17:51 +09:00
Zuul
02bda20dce Merge "Auto find greatest Fedora cloud image sub-release" 2021-07-22 07:53:44 +00:00
Zuul
585f34e068 Merge "Replace deprecated import of ABCs from collections" 2021-07-22 07:38:16 +00:00
Danni Shi
05d8f3ae38 Add a keylime-agent element and a tpm-emulator element
Story: #2002713

Task: #41304
Change-Id: Ia5226faabae8accb03f401aa4de3c8311b583455
2021-07-20 10:05:41 -04:00
Zuul
ddd70501d4 Merge "Convert multi line if statement to case" 2021-07-19 23:31:06 +00:00
Takashi Kajinami
b6254398e7 Replace deprecated import of ABCs from collections
ABCs in collections should be imported from collections.abc and direct
import from collections is deprecated since Python 3.3.

Change-Id: Idacff95cbb276eda0bc55de771ce6c701363c2e1
2021-07-17 01:02:19 +09:00
Zuul
556f4f6aa6 Merge "Add a growvols utility for growing LVM volumes" 2021-07-12 10:12:13 +00:00
Zuul
3f2feb6e3b Merge "cache-url : turn down verbose curl" 2021-07-12 09:55:28 +00:00
Zuul
d286f64a76 Merge "Add element block-device-efi-lvm" 2021-07-08 01:08:26 +00:00
Zuul
7831d71066 Merge "fedora-container: install dnf-plugins-core" 2021-07-07 06:51:05 +00:00
Zuul
d4f2d79f89 Merge "Do not uninstall non-installed packages" 2021-07-07 04:55:10 +00:00
Ian Wienand
bc39a5afe7 fedora-container: install dnf-plugins-core
Add dnf-plugins-core to the package-installs; this lets things like
"dnf copr" work automatically and is in-line with fedora-minimal base
packages.  While we're here, clean up some unneeded packages, and
remove the pkg-map that isn't relevant for Fedora builds.

Change-Id: Iad5a4717bcb55928377cc159b3360b0a70c5c5ac
2021-07-07 11:02:10 +10:00
Ian Wienand
12b60c4088 Mount /sys RO
As noted inline, this works around potential issues by being a strong
indication you are in a container (e.g. [1]).  Since nothing should be
changing anything on the host/build system, this is a generically
safer way to operate.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1975588

Change-Id: Ic6802c4ffc2e825f129af10717860a2d1770fe80
2021-07-05 11:45:02 +10:00
Steve Baker
a6e0bf83db Add a growvols utility for growing LVM volumes
There is currently no automated way of growing LVM volumes on boot
like single partition images do with their growroot mechanism. This
lack likely contributes to LVM not being widely used on VM and
baremetal workloads, since growing to the full disk requires workload
knowledge to determine which volumes to grow and by what amount.

The growvols element contributes a growvols python script which can be
run on firstboot (via systemd or cloud-init) or manually via
automation such as ansible. It is also an interactive script which
displays the full list of modifying commands before prompting for
confirmation to run them all.

By default the script will grow the root volume, but arguments allow
any volume to grow by a specified amount, or a percentage of the
available disk space.

Blueprint: whole-disk-default
Change-Id: Idcf774384e56cce03e56c0e19c7d08a768606399
2021-07-01 11:16:31 +12:00
Ian Wienand
ba5bcce036 cache-url : turn down verbose curl
curl's "-v" is a bit too verbose for "-x", especially when what you're
downloading bounces through a few redirects as is common.  Turn this
down and put it behind "-xx" or greater.

Change-Id: I6d91166bb237f2a1818cae7532e794ef0f01288b
2021-06-24 10:09:00 +10:00
Steve Baker
ab8d2910c4 Add element block-device-efi-lvm
Element block-device-efi-lvm has been added which is like
block-device-efi but defines an LVM logical group in the root
partition. Three logical volumes are defined in that group, mounted to
/, /var, and /home.

This volume layout will not meet all requirements, but this is more of
an example demonstrating the capability to encourage more usage of
this existing feature.

This is based on the overcloud-partition-uefi element in
tripleo-image-elements, and I believe this capability is too useful to
have the only working example buried in a related project repo.

This change also fixes the element string matching in
_arg_defaults_hack, the 'vm' test was also matching against 'lvm' and
'block-device-efi-lvm' elements. Also the 'block-device-' test now
properly tests for this being the prefix of the block-device element.

This change also makes block-device-efi fsck-passno compliant with the
documentation[1] so that / has value 1 and all other mounts are set to
2.

[1] https://www.man7.org/linux/man-pages/man5/fstab.5.html

Change-Id: If86a0e49186ce5a65cc0084101d31ce59a97b854
Blueprint: whole-disk-default
2021-06-01 17:27:28 +12:00
Zuul
75ee18b01b Merge "bootloader: remove extlinux/syslinux path" 2021-05-28 04:51:23 +00:00
Matthew Thode
2ff65d512e
Do not uninstall non-installed packages
The bootloader element uses the grub-efi-$arch package to remove already
installed packages (for redhat).  The uninstall of a non-installed
package fails with a non-zero exit code on gentoo.  The gentoo base
tarball does not include a bootloader and the grub-efi-$arch package is
only used for uninstalls, so zero out the variable to allow bootable
images to be generated.

Change-Id: If8572abd6e19a02f2f63b33d4f83a7054774d7e6
Signed-off-by: Matthew Thode <mthode@mthode.org>
2021-05-25 23:32:16 -05:00
Ian Wienand
3ccca50c14 Fix DISTRO_NAME in Fedora elements
Fix typo in documentation around name variables.

Change-Id: I88fd1dd828ce1b606398198dd0317096363480f1
2021-05-17 10:02:09 +10:00
Ian Wienand
80ff74ca4d Add fedora-containerfile element
This builds Fedora from the containerfile element.

Depends-On: https://review.opendev.org/c/zuul/nodepool/+/790531
Depends-On: https://review.opendev.org/c/opendev/glean/+/790368

Change-Id: I5cf9b0505f2d542b3611ebbf9494f9be11e61f34
2021-05-13 10:43:45 +10:00
Ian Wienand
f6748a4cd4 bootloader: remove extlinux/syslinux path
This is a first pass through the bootloader, that removes the extlinux
and syslinux install/cleanup path.

Change-Id: Ifb107796cdb6748430a124bf13ced93db9689bff
2021-05-13 10:33:06 +10:00
Ian Wienand
4d16ccdbef bootloader: disable BLS for Fedora
As noted inline, the switch to "boot loader spec" grub entries breaks
our setting of the root device.  This happened some time ago, and it's
not 100% clear to me why our existing Fedora builds haven't broken on
this.  However, the new containerfile based builds do seem to be
hitting this.

Disable it for now.

Change-Id: Ia3472947799bb35ffccfa92937cdd0d68b12a25c
2021-05-11 16:48:58 +10:00
Carlos Goncalves
72442ba656 Auto find greatest Fedora cloud image sub-release
Fedora cloud images have sub-releases in their filename. It is not
exacly clear how this is generated but we do know how we can determine
the greatest programatically.

Change-Id: I7fc56897c681fe037db211c290edcdd23cdd5d5b
2021-05-10 09:14:17 +00:00
Ian Wienand
2a14088eac containerfile: automatically search for distro docker files
This makes the container file element search the active element list
for `containerfiles/${DIB_RELEASE}` for building.  This makes it easy
to write wrappers for ubuntu/fedora/etc. containerfile elements.

Change-Id: I68f1d928e54a70bad76985ddd3e156bb5f978b0d
2021-05-10 15:36:28 +10:00
Zuul
b536dbba8e Merge "Add containerfile element" 2021-05-10 01:50:40 +00:00
Zuul
c3243be696 Merge "Install epel-release from URL" 2021-05-10 01:28:32 +00:00
James E. Blair
edff632186 Add containerfile element
This is a base element which uses a containerfile (Dockerfile) to
build a container image, then the filesystem is extracted from that
image and forms the root of the dib image.

You can add as little or as much to the dockerfile as desired.

Change-Id: I4e821aa2ce7feb8841ef31da56de1a31aa9218b5
2021-05-07 13:54:59 +10:00
Zuul
cb0c117659 Merge "debian-minimal: bullseye: /updates -> -security" 2021-04-30 09:25:35 +00:00
Zuul
231770434b Merge "debian-minimal: Set bullseye version" 2021-04-30 05:59:26 +00:00
Zuul
46fb885ed8 Merge "Fix centos stream set mirror" 2021-04-30 05:21:23 +00:00
Jeremy Stanley
50b1566fa4 debian-minimal: bullseye: /updates -> -security
With the release of Debian bullseye and later, security updates are
provided in the bullseye-security suite instead of bullseye/updates.

Change-Id: I63580ec96a53e5e8ef8d105e766d838029727917
2021-04-28 17:07:22 +00:00
Ian Wienand
3071457355 debian-minimal: Set bullseye version
Currently Debian sets /etc/debian_version to "bullseye/sid" and, due
to a series of issues explained in [1] more fully "lsb_release -c" in
the OpenDev environment doesn't return the distribution code name.
Overriding this to the final release version fixes this.

[1] http://lists.opendev.org/pipermail/service-discuss/2021-April/000222.html

Change-Id: I00c1741dac6ad5f2c4bf855a207f17d8985bc763
2021-04-28 09:48:22 -07:00
Zuul
280d9232c6 Merge "dib-run-parts: stop leaving PROFILE_DIR behind" 2021-04-26 07:00:04 +00:00
Zuul
d03e7f1327 Merge "Ensure redhat efi packages are reinstalled during finalise" 2021-04-26 01:08:04 +00:00
Clark Boylan
da674c4e5b Install pbr before glean to address SNI issues
Some older distros (like centos8 and xenial) don't support SNI in their
easy_install implementations which are used to install setup_requires
for python packages. PBR is a setup_requires for glean. We work around
this problem when installing glean by preinstalling PBR with pip.

Change-Id: Ie9f5c9ed06954cbe51f23fe8cca0655a931a5201
2021-04-23 15:04:26 -07:00
Ian Wienand
7c8e9f8733 dib-run-parts: stop leaving PROFILE_DIR behind
When a build fails, we can exit and leave ${PROFILE_DIR} behind.  Make
sure this is cleaned up with an exit trap.

While we're adding a function, update the syntax of the others for
consistency.

Change-Id: I14499b5ebaaa30126aaa6b3d1bd86ed64f110fda
2021-04-21 10:42:30 +10:00
Steve Baker
5caeba0c68 Ensure redhat efi packages are reinstalled during finalise
The rhel-8.4 qcow2 base image already has the grub2-efi-x64 package
installed on its single partition which has files installed to
/boot/efi..., however a partitioned image will have an empty /boot/efi
partition when running 50-bootloader. This means dnf will not install
grub2-efi-x64 when requested and /boot/efi will remain empty.

This commit makes the following changes:
- Refactors redhat bootloader pkg-map for the following:
  - Make x86_64/amd64, arm64/aarch64 adjancent so they don't diverge
  - Map grub-efi to packages installed to /usr
  - Map grub-efi-{arch} to packages installed to /boot/efi
- Removes packages grub-efi-{arch} before installing grub-efi and
  grub-efi-{arch}

Change-Id: Ia197feea34f43bd870fed30829b740596e6b2f48
2021-04-21 10:56:37 +12:00
Zuul
d7becbeb2b Merge "Add Debian Bullseye Zuul job" 2021-04-12 13:16:23 +00:00
Dmitriy Rabotyagov
cb4e9fc072 Add Debian Bullseye Zuul job
Change-Id: I7d7994565ab7ed62e49efd80766fe19a906499db
2021-04-09 10:20:10 +03:00
Zuul
833d433e92 Merge "Improved the documentation for DIB_DNF_MODULE_STREAMS" 2021-04-08 10:34:30 +00:00
Zuul
cb96117146 Merge "Fix: IPA image buidling with OpenSuse." 2021-04-08 07:05:22 +00:00
Zuul
ffe3aa1610 Merge "Properly set grub2 root device when using efi" 2021-04-08 07:05:17 +00:00
Chandan Kumar (raukadah)
21a752ee4d Improved the documentation for DIB_DNF_MODULE_STREAMS
https://review.opendev.org/c/openstack/diskimage-builder/+/785138
adds the support for DIB_DNF_MODULE_STREAMS which is now available
for all Yum based distros.

This patch enhances the docs for using it for all Yum
based distributions.

Signed-off-by: Chandan Kumar (raukadah) <chkumar@redhat.com>
Change-Id: I29e726679c2b675b3c0cd95a3ff48fdad7cd5431
2021-04-08 11:22:43 +05:30
Clark Boylan
3294aecca2 Properly set grub2 root device when using efi
We've noticed that centos8 arm64 images have a root devices of
/dev/mapper/loop7p3 which make sense within a dib image build context
but not at boot time. Dib intends to use labels to set the root device
but when efi is used we end up running grub2-mkconfig against the efi
grub config path before we configure grub to use labels.

Fix this by running grub2-mkconfig after its configuration is set.
This should avoid confusion and complicated paths through the scripts
that configure this for us. We then copy the resulting config to the efi
specific grub.cfg location for platforms that have it.

There is also a small refactoring that is done to try and make the ~3
boot variants more clear:

 1) Booting with legacy bios
 2) Booting with uefi without a signed shim that directly calls grub
 3) Booting with uefi and a signed shim that calls grub

Options 1 and 2 share the /boot/grub*/grub.cfg file. Option 3 needs its
grub.cfg to live alongside distro specific efi target.

Change-Id: Ie9790da9d1bbea58197b37b15a48e77f8a93c1ac
2021-04-07 15:46:10 -07:00
Chandan Kumar (raukadah)
ced54fea75 Make DIB_DNF_MODULE_STREAMS part of yum element
While building cloud images, it is common to set modules
for CentOS and RHEL images. Earlier it was part of rhel-common
which was specific to RHEL OS not for CentOS. Moving it
under yum element as module/stream can be enabled or disabled
via dnf itself.

Signed-off-by: Chandan Kumar (raukadah) <chkumar@redhat.com>
Change-Id: Idc0f277f97e92e4d003f059f01b59f1b5513da34
2021-04-07 16:06:09 +05:30
Riccardo Pittau
256c798bc4 Convert multi line if statement to case
Having multi-line if-elif statements is not ideal, case improves
readability clarifying the code.

Change-Id: I3383584e09763d4ae8eab2f36a93ee399dae8382
2021-04-07 07:17:19 +00:00
Xinliang Liu
8c86d876e3 Fix centos stream set mirror
This intents to fix job dib-functests-bionic-python3-image.
And no CentOS-Stream-centosplus.repo file[1].

[1]: http://rpm.pbone.net/info_idpl_72967298_distro_centosother_com_centos-stream-repos-8-2.el8.noarch.rpm.html

Change-Id: I9d69413f31d0a9d83e992d05d177f683b7361337
2021-04-02 06:56:45 +00:00
smoshiur1237
93d11a7cf8
Fix: IPA image buidling with OpenSuse.
At this moment the IPA image building with OpenSuse is broken and here, it was failing during the release check for Opensue because etc/SuSE-release is not valid anymore and deprecated for openSuse. Its renamed to /etc/os-release for openSuse rlease 15. This PR will solve the issue to build IPA image with OpenSuse base image. There is another PR opened in ironic-python-agent-builder, which adds all the missing packages, setuptools upgrade and svc mapping to do the build successful.
https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/778726

Bug-Report: https://bugs.launchpad.net/diskimage-builder/+bug/1921510

Change-Id: Id2759be29bfcbf2ecf1ce67e171686924b506b1a
2021-03-29 13:46:04 +03:00
Dmitry Tantsur
4dd8982c53 simple-init: support installing Glean from packages
Such package is in progress for RDO:

Depends-On: https://review.rdoproject.org/r/c/openstack/glean-distgit/+/32656
Change-Id: I2655b6b9037477c26743d51b0c9a9e31a23707fb
2021-03-25 12:49:03 +01:00
Dmitry Tantsur
fe89856d7d simple-init: allow disabling DHCP fallback
In some cases this is not desired (see the dependecy for details).

Change-Id: Ie549d0c9769a2b6c58de9e61cb621ca96c1cd886
Depends-On: https://review.opendev.org/c/opendev/glean/+/781500
2021-03-19 12:57:50 +01:00
Matthew Thode
b4f768117f
update gentoo keywords to support gcc-10
open-iscsi and open-isns need keywording to support gcc-10, move it out
of being keyworded only for musl profiles.

remove unneeded keywords for python-exec and python-exec-conf (marked
stable)

use the full package name for the dev-lang/python-exec-conf package

Change-Id: I44eaf8c2230e9e2089a72fce46954f4336626843
Signed-off-by: Matthew Thode <mthode@mthode.org>
2021-03-18 23:24:31 -05:00
Zuul
4aae99e64e Merge "Change paths for bootloader files in iso element" 2021-03-16 09:56:07 +00:00
Zuul
0e5922c6b8 Merge "replace the link which is in the 06-hpdsa file" 2021-03-16 06:34:40 +00:00
RotanChen
a9386ba147 replace the link which is in the 06-hpdsa file
The old link does't work,this one does.

Change-Id: I128b5841b2bd4897e2b2c0e82ad31049ce7b0c29
2021-03-12 19:16:50 +08:00
Steve Baker
522113bc6c Only add rhel base repos when REG_REPOS is not set
For offline (satellite based) installs the base repos won't be
available and the base packages will come from a different named repo
in satellite which will be specified by REG_REPOS.

This change will ensure no base repos are added when REG_REPOS are
specified so offline image builds are possible. All required base
repos need to be added to REG_REPOS when it is used. Documentation[1]
already includes base repos, so this should not be disruptive.

[1] https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/16.1/html/director_installation_and_usage/creating-whole-disk-images#disk-image-environment-variables

Change-Id: Iafb81d50dffdac40d3b011670200b8da4c3a58f0
2021-03-11 11:01:29 +13:00
Steve Baker
27a326dafb Support secure-boot bootloader where possible
As of grub2 >= 2.02-95 on redhat family distros, calling grub2-install
on an EFI partition will fail with: "this utility cannot be used for
EFI platforms because it does not support UEFI Secure Boot."

This version of grub is now in centos8-stream and non-eus repos of
RHEL-8. It is not currently possible to build whole-disk UEFI images
on these distros, and when this package is promoted this will also
affect centos8 and RHEL-8 eus. The grub maintainers made this change
because the grub2-install generated /boot/efi/EFI/BOOT/BOOTX64.EFI
will never be capable of booting with Secure Boot.

This change defines a $EFI_BOOT_DIR for every distro element. When
directory /boot/efi/$EFI_BOOT_DIR exists a grub.cfg file in will be
generated there. This change also installs the shim package on redhat
family distros, which installs a copy of the shim bootloader to
/boot/efi/EFI/BOOT/BOOTX64.EFI. Using centos as an example, this
allows UEFI to boot the shim /boot/efi/EFI/BOOT/BOOTX64.EFI which
then chains to /boot/efi/EFI/centos/grubx64.efi.

If /boot/efi/$EFI_BOOT_DIR doesn't exist (such as for Ubuntu,
/boot/efi/EFI/ubuntu) the current behaviour of running grub-install to
generate /boot/efi/EFI/BOOT/BOOTX64.EFI will continue. For distros
such as Ubutnu where packaging does not populate /boot/efi/EFI/ubuntu
with .efi files, secure boot can be added in the future by copying
.efi files to /boot/efi/EFI/ubuntu and copying the shim file to
/boot/efi/EFI/BOOT/BOOTX64.EFI.

Change-Id: I90925218ff2aa4c4daffcf86e686b6d98d6b0f21
2021-03-11 10:27:59 +13:00
Zuul
a0fd571a3c Merge "Add efibootmgr utility for UEFI boot menu management" 2021-03-09 02:11:10 +00:00
Zuul
6a99e314b4 Merge "Use the same bootloader pkg-map for all redhat family" 2021-03-09 02:00:11 +00:00
Zuul
739b529a33 Merge "Add aarch64 support for rhel" 2021-03-09 01:52:31 +00:00
Zuul
b91aae61f7 Merge "Don't use hardcode while override base image file" 2021-03-09 00:03:09 +00:00
Zuul
3b5f5b55b1 Merge "Fix hooks order for CentOS/Fedora when mirror used" 2021-03-08 23:51:12 +00:00
Zuul
4c5b1179f3 Merge "Fix installation of proliant tools" 2021-03-08 23:47:32 +00:00
Daniel Pawlik
8b0e2417a6 Change get-pip url
The path for get-pip.py script in version 3.5 has been changed
with this commit [1].

[1] 2360f025eb

Change-Id: Ifde16e40b4e241c6c4c93df44330c403ee903e6f
2021-03-08 09:19:28 +01:00
Xinliang Liu
0c51b49414 Add aarch64 support for rhel
Change-Id: I86ccc56e37b214a45ba620b731b51f58d73471f8
2021-03-08 07:00:15 +00:00
Steve Baker
69bf654eba Add efibootmgr utility for UEFI boot menu management
As of grub2 >= 2.02-95 calling grub2-install on an EFI partition will
fail with: "this utility cannot be used for EFI platforms because it
does not support UEFI Secure Boot." In the case of ironic deployments,
ironic-python-agent will call efibootmgr to set the local boot for
subsequent boots.

This change adds efibootmgr to the image as well so any other UEFI
menu changes can be made manually.

Change-Id: I765ded15da07d6227d1e337960e54ad0e0d6ca39
2021-03-05 10:00:49 +13:00
Maksim Malchuk
c4c21967d8 Fix hooks order for CentOS/Fedora when mirror used
The python3/python3-pyyaml packages both are never installed and dnf
itself never updated when $DIB_DISTRIBUTION_MIRROR set and used.

This change fix the order of the operations:
 1. yum/dnf configure.
 2. *.repo patching.
 3. yum/dnf update/install execution.

Change-Id: Ifbbf1f0190fe8c8a77fb3be820e8056447e755f6
Signed-off-by: Maksim Malchuk <maksim.malchuk@gmail.com>
2021-03-04 10:54:52 +00:00
Maksim Malchuk
ca83a4c3cb Don't use hardcode while override base image file
The trvial fix allow override to work in air-gapped envirments where
the command 'curl -s https://cloud.centos.org/...' would fail.

Change-Id: I84296d8816042e4cd4cb02f15746b86d600d13d6
Signed-off-by: Maksim Malchuk <maksim.malchuk@gmail.com>
2021-03-04 10:54:32 +00:00
Francois RIGAULT
5efb11de0d Fix installation of proliant tools
Use the actual virtualenv to install proliant tools, and make sure the
dependencies needed for sum firmware upgrade are included.

Change-Id: Ie1dbb868450918567b2903cdeccda35af1904417
Closes-Bug: #1916346
2021-03-03 19:04:50 +01:00
Mateusz Kowalski
3b22ee8784
Change paths for bootloader files in iso element
This PR updates locations for files used by the bootloader depending on
the target operating system built. The current logic does not take into
account latest versions of operating systems and makes it impossible to
build ISOs against those.

With this change it is possible to correctly build CentOS 8, Ubuntu
18.04 and Ubuntu 20.04 images.

Closes-Bug: #1916913
Change-Id: I3ed0041640f539e82805d03ba26fe46217f3ac3c
2021-03-03 11:01:14 +01:00
Steve Baker
f7ba7ba1de Use the same bootloader pkg-map for all redhat family
The only difference between the rhel and redhat entries is rhel has
the extra grub-efi-x86_64 mapping. All redhat family releases would
benefit from having this too, so this change removes the whole rhel
entry and adds grub-efi-x86_64 to the redhat family.

The assumption is that anything which applies to rhel also applies to
centos-stream, and in this case doesn't harm centos or fedora either.

Change-Id: I0dc44c1f2b57516742f4c3e43cfc8874d6b90fa2
2021-03-03 13:24:15 +13:00
Steve Baker
5c1f9a3238 Don't install centos-linux-release on 8-stream
This package doesn't exist in the stream base repo, and neither does
centos-linux-repos.

These are presumably replaced by centos-stream-release and
centos-stream-repos. This change adds an else block to handle the
non-stream base packages.

Change-Id: I32249199c3dfa44fc24fba28d24f314112c2e200
2021-02-23 12:57:05 +13:00
Noam Angel
0e700b25dc Check and remove existing image interface configurations
This change will remove any existing interface configurations in the image. They are not necessary
and could interrupt with deployments. In any case they should not exist if we use 
dhcp-all-interfaces element.

Change-Id: I35a4b5ea6e2315de3b0d9f8353ac2b6f4b995697
2021-02-16 08:46:32 +00:00
Zuul
cbbcf377d8 Merge "Remove the deprecated ironic-agent element" 2021-02-03 08:45:43 +00:00
Zuul
6cfc94c8bf Merge "Install last stable version of get-pip.py script" 2021-02-03 01:16:13 +00:00
Zuul
d4cbaf12f0 Merge "Fix CentOS Stream 8 base repo in centos element" 2021-02-02 14:10:23 +00:00
Zuul
6ebd8cb974 Merge "simplify updating python versions in gentoo" 2021-02-02 07:28:12 +00:00
Zuul
68c47f91fb Merge "Set eus repositories if REG_RELEASE is set" 2021-02-02 07:24:59 +00:00
David Hill
71449a60ff Set eus repositories if REG_RELEASE is set
Set eus repositories if REG_RELEASE is set instead of the base repos
as the current behavior is to use the non-EUS repositories for RHEL
8.2 deployment which breaks image building for customers.

Change-Id: I8e687b27922c3f6fc3d69794866795ab89ecc346
2021-02-01 09:44:16 -05:00
Daniel Pawlik
383fab59e9 Install last stable version of get-pip.py script
The get-pip script does not work well with Python 2 and
it raise error during the installation [1].

[1] https://github.com/pypa/get-pip/issues/83

Change-Id: I3755c34da313ef647547c6ae18b59cc04c2cdd60
2021-02-01 08:29:04 +01:00
Matthew Thode
4d5ba1497c
simplify updating python versions in gentoo
portage now generates /etc/python-exec/python-exec.conf based on the
order of PYTHON_TARGETS in /etc/portage/make.conf

fixes an issue where ARCH was being detected as amd64 not x86_64
fixes kernel installs (virtual/dist-kernel)

standardizes simple if statements (note, the 'shorthand' method will
pass the exit code back to shell but the 'longhand' does not).

Change-Id: I74041c232bc6ab4d6e67a4ecfaa759aa4a5feb6c
Signed-off-by: Matthew Thode <mthode@mthode.org>
2021-01-27 11:34:09 -06:00
Daniel Pawlik
5da3ec0f06 Install epel-release from URL
The epel-release is not available for RHEL 7, so the
package python3-PyYAML was not installed.

Change-Id: Id81ab9d668fce46349d1abd4fb19cf51a9c1ee17
2021-01-27 12:33:30 +01:00
azvyagintsev
3f8a85f932 Add 'DIB_UBUNTU_MIRROR_DISTS' to ubuntu-minimal
* Add "DIB_UBUNTU_MIRROR_DISTS":
    Default: ``updates,security,backports``
    Notes: For some deployment, is may be required
           to disable backport|update|etc packages
           integration.

Change-Id: Ic7dcd29ea658a66763b4422915149e4d3fe663cc
2021-01-27 10:32:49 +02:00
Javier Pena
59f6922e22 Fix CentOS Stream 8 base repo in centos element
The file name for a CentOS 8 repo file is BaseOS.repo, not Base.repo.

Change-Id: I97f5cbd8e5bc17b453db4fcee6e8492652da73aa
2021-01-22 14:54:01 +01:00
Dmitry Tantsur
fd1848282f Remove the deprecated ironic-agent element
Despite having several issues (like missing firmware), it is still
used by people. It seems that the only way to stop that is to remove it.

Change-Id: I4baed8e8ab663c624dcc8d06ff0293d57b082abb
2021-01-21 14:06:08 +01:00
Zuul
2737d909bf Merge "Fix building error with element dracut-regenerate" 2021-01-18 06:45:18 +00:00
Zuul
3c2d18d33e Merge "Fix centos 8.3 partition image building error with element iscsi-boot" 2021-01-18 06:43:25 +00:00
Matthew Thode
45796c65c5
add missing packages for python reinstall on gentoo
Change-Id: I1022c2c6db843701e9e6ece87b0ac87071723b99
2021-01-17 02:34:50 -06:00
Xinliang Liu
447354f933 Fix centos 8.3 partition image building error with element iscsi-boot
This use the same workaround as element bootloader's[1] to fix CentOS
8.3 partition image building error with iscsi-boot:
...
2021-01-12 07:11:02.439 | + grub2-mkconfig -o /boot/grub2/grub.cfg
2021-01-12 07:11:02.654 | /usr/bin/grub2-editenv: error: cannot rename the file /boot/grub2/grubenv.new to /boot/grub2/grubenv: No such file or directory.
2021-01-12 07:11:02.665 | Generating grub configuration file ...
2021-01-12 07:11:03.112 | /usr/bin/grub2-editenv: error: cannot rename the file /boot/grub2/grubenv.new to /boot/grub2/grubenv: No such file or directory.
...

[1]: https://review.opendev.org/c/openstack/diskimage-builder/+/750279
Closes-Bug: 1911120

Change-Id: I2de5444f7e1a145df9abb03fa4c367e8bb914e03
2021-01-14 02:05:20 +00:00
Xinliang Liu
4c4f49abcc Fix building error with element dracut-regenerate
This fixes below building error When build centos with element
dracut-regenerate on ubuntu by specifying the initrd and kernel version.
...
2021-01-12 03:44:15.758 | dracut: Cannot find module directory /lib/modules/5.4.0-58-generic/
2021-01-12 03:44:15.759 | dracut: and --no-kernel was not specified
2021-01-12 03:44:15.765 | Traceback (most recent call last):
2021-01-12 03:44:15.766 |   File "/tmp/in_target.d/finalise.d/50-dracut-regenerate", line 102, in <module>
2021-01-12 03:44:15.766 |     main()
2021-01-12 03:44:15.766 |   File "/tmp/in_target.d/finalise.d/50-dracut-regenerate", line 82, in main
2021-01-12 03:44:15.766 |     raise e
2021-01-12 03:44:15.766 | subprocess.CalledProcessError: Command '['dracut', '--list-modules']' returned non-zero exit status 1.
...

Change-Id: I87ae20b3fa2f291bb107e607137fcd7b1c0a4996
2021-01-14 01:30:52 +00:00
Chandan Kumar (raukadah)
eeb4d901a7 Enable dracut list installed modules
While looking at DIB logs, it is very hard to debug dracut
issues due to missing modules, listing dracut modules will
give an insight.

Related-Bug: #1907457

Signed-off-by: Chandan Kumar (raukadah) <chkumar@redhat.com>
Change-Id: I4d277bdfd648adba5a749d22d905c66b807e249a
2020-12-28 14:24:20 +00:00
Zuul
82a21632e8 Merge "Remove dib-block-device console entrypoint" 2020-12-15 00:01:09 +00:00
Zuul
6ed5621c5f Merge "Remove entry-point for element-info" 2020-12-15 00:01:04 +00:00
Daniel Pawlik
8c714716ed Remove centos-repos package for Centos 8.3
With Centos 8.3, centos-repos package has been replaced by
other packages [1].

[1] https://lists.centos.org/pipermail/centos-devel/2020-September/056069.html

Also Increase flake8 and pyflakes version in lower-constraints.txt as
this was already broken.

Change-Id: Ife139fcaff0c2d944098ea353259971d2d3f18b8
2020-12-14 16:55:21 +00:00
Dmitry Tantsur
9be4221b4a Install git-core instead of git on RH systems
The git package pulls in heavy dependencies for just a few rarely
used commands. git-core should be enough (and already uses for Suse).

Change-Id: I96b71072c22c26b3b651466053b9e9561527cbe5
2020-11-26 17:16:31 +01:00
Zuul
71e3b81788 Merge "Run autoremove on post-install step" 2020-11-23 23:29:50 +00:00
Zuul
2eb1c9161f Merge "simple-init: also remove en* interfaces from the images" 2020-11-23 15:42:30 +00:00
Zuul
39807fc2f5 Merge "Add support for vlan interfaces in dhcp-all-interfaces.sh" 2020-11-23 15:27:12 +00:00
Zuul
f52ad7c839 Merge "Fix python-stow-versions" 2020-11-23 05:15:54 +00:00
Dmitry Tantsur
1462b80781 simple-init: also remove en* interfaces from the images
Modern distros use more creative interface naming, e.g. CentOS 8
adds both eth0 and ens3 (!). Remove everything.

Change-Id: Ibdebdb09ea790787840cf9b817d4eb549ef18249
2020-11-22 20:17:42 +01:00
Dmitry Tantsur
19378bbd9d Fix dynamic-login with grub2
Grub2 messes up arguments with double quotes in them, changing from
key="value" to "key=value" [1]. Support this format as well.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=950760

Change-Id: Icf4c9f86009d29a342d6e0e21644af680066b0b2
2020-11-20 15:00:54 +01:00
Dmitriy Rabotyagov
d0ce0d34e5 Run autoremove on post-install step
Autoremove can touch packages, that are assumed to be present
for post-install step.

A good example is dkms, which is dropped in
install.d, but dkms element try to use it in post-install step.
This results in the following log [1]

[1] http://paste.openstack.org/show/797809/

Change-Id: I635af230c6b250fee273039935cf19506e83b3d1
2020-11-17 09:23:35 +00:00
Dmitriy Rabotyagov
75a5415dbc Fix python-stow-versions
DIB_PYTHON_PREFIX variable was neither defined nor needed.

Change-Id: I36ebe5a6dee511c0c89cc251e9ec340ec7ed915d
2020-11-17 09:23:32 +00:00
Ian Wienand
8a12243517 Remove dib-block-device console entrypoint
This is never called externally to dib, so doesn't need to be an
entrypoint.  Call it from within dib using the running python
executable and from the lib/ directory; this means we do not need to
have the virtualenv activated to run disk-image-create.

Change-Id: Ie9b551824792864402b0c63ccc350dc5c92dcc3f
2020-11-09 16:43:19 +11:00
Ian Wienand
6f73e123d0 Remove entry-point for element-info
This is really an internal dib tool.  Move it to the lib directory,
and call it with the python we are running under.

This is one less reason to require the virtualenv to be activated when
you run 'disk-image-create'.

Change-Id: Id689683a0b1fdcb446b04ba967284a216133d743
2020-11-09 16:39:35 +11:00
Bob Fournier
5d23d8e6b0 Add support for vlan interfaces in dhcp-all-interfaces.sh
Provide ability to run dhcp client on VLAN interfaces created on
top of an Ethernet interface.  See also
<https://storyboard.openstack.org/#!/story/2008298>`__ for further details.

Change-Id: Ic3ffd7b8e23b1e996cfe6c79ce0ff47e521f30be
2020-11-03 10:04:17 -05:00
Nicolas Hicher
c160538465 yum-minimal: Add centos-stream-repos package for centos-8-stream
centos-repos package is not available on centos-8-stream:

2020-11-02 15:35:20.962 | No Match for argument centos-repos
2020-11-02 15:35:20.992 | No Match for argument centos-release-stream

The build fails later on install_pkg_manager sed command:

2020-11-02 15:35:22.903 | sed: can't read
/var/cache/nodepool/dib_tmp/dib_build.iPSHptNW/mnt/etc/yum.repos.d/*repo: No
such file or directory

This change ensure centos-stream-repos package is installed to provide
/etc/yum.repos.d/*repo files

This change replace 'centos-release-stream' package with the new
'centos-stream-release' package [1]

[1] https://lists.centos.org/pipermail/centos-devel/2020-September/056069.html

Change-Id: I6c397bf7b5797a02e5f006c18ee63c9cdf66b38c
2020-11-03 09:09:03 -05:00
azvyagintsev
f6315c9757 Allow processing 'focal' ubuntu release in lvm
* Those would unlock ubuntu-focal builds

Change-Id: I628d17bfcf07fdff7d2290be7db0701095486bbf
2020-10-28 16:55:49 +02:00
Zuul
061cc48583 Merge "Ensure yum-utils is installed in epel element" 2020-10-13 09:56:57 +00:00
Javier Pena
f13eb1bcea Ensure yum-utils is installed in epel element
We are running yum-config-manager/dnf config-manager in the epel
element. Even though the yum-utils package is declared in the
package-installs.yaml file, the package-installs pre-install.d
script is executed after the one in the epel element, so image
build fails.

This commit ensures yum-utils or dnf-plugins-core are installed
before running the command.

Change-Id: Ib292b0b2b31bd966e0c5e8f2b2ce560bba89c45c
2020-10-08 09:49:57 +02:00
Lon Hohberger
91ce15afd1 Fix runtime error if DIB_DNF_MODULE_STREAMS is not set
Initial patch for I78d7bcf214a45245e2073428120fcbdd968e1acd
works without the envvar set, however, 'set -eu' causes it to break
if unset.

This makes the module configuration variable not required to be
set, consistent with other DIB scripts.

Change-Id: I5ca80f518d0371a18c107c061dc923876463af57
Signed-off-by: Lon Hohberger <lhh@redhat.com>
2020-10-07 07:28:46 -04:00
Zuul
580256f374 Merge "Disable growpart in cloud-init-disable-resizefs" 2020-10-07 08:41:03 +00:00
Michal Nasiadka
520e12ec29 Disable growpart in cloud-init-disable-resizefs
This disables growpart module in cloud-init, not resizing / partition
to maximum disk free size by default.

Change-Id: I69984a9141fa8abb12dc5d51bd334f9280deca67
2020-10-06 14:03:57 +02:00
Zuul
6c788f5a09 Merge "Revert "Avoid disabling rhel-7-server-rh-common-rpms"" 2020-09-24 16:57:09 +00:00
Zuul
cbeca0babd Merge "Add support for CentOS 8 Stream cloud image" 2020-09-24 14:55:23 +00:00
Zuul
7ef82de0db Merge "rhel-common: Provide method to select module streams" 2020-09-24 14:32:19 +00:00
yatinkarel
0c341f5820 Revert "Avoid disabling rhel-7-server-rh-common-rpms"
Originally it was added for missing python-cheetah dep
for openstack-nova. Nova has removed usage of it long
ago with [1]. rhel-7-server-rh-common-rpms should be
disabled once it's usage is over as it packages from
it can conflict with other openstack repos.

If some package which is needed by OpenStack Packages
is missing then instead of adding rhel-7-server-rh-common-rpms
repo consideration should be to add it in RDO.

[1] https://review.opendev.org/#/c/40205

This reverts commit c7219a5a60.

Change-Id: Iad3a1c353c10bb35f9c9ef4076b65f5c84b803b2
2020-09-21 10:35:22 +05:30
Dmitry Tantsur
77fac28078 Rename duplicating 10-debian-minimal.bash
This file is present in both dpkg and debian-minimal element,
causing a failure to build anything with debian.

Change-Id: I8213d581a79bb432281f31955a44418e4047d9e1
2020-09-18 14:52:56 +10:00
Zuul
1355421ce8 Merge "bootloader: remove dangling grubenv links" 2020-09-17 01:01:42 +00:00
Zuul
58034ef253 Merge "Install gzip instead of busybox-gzip on suse" 2020-09-17 00:36:55 +00:00
Zuul
8eb20d2faf Merge "Install grep before busybox on suse distros" 2020-09-16 19:22:59 +00:00
Lon Hohberger
5299371957 rhel-common: Provide method to select module streams
Some OpenStack releases on RHEL require specific modules
in order to function correctly.  This adds the ability
to set DIB_DNF_MODULE_STREAMS which then are selected
prior to package installation.

Change-Id: I78d7bcf214a45245e2073428120fcbdd968e1acd
Signed-off-by: Lon Hohberger <lhh@redhat.com>
2020-09-16 08:43:30 -04:00
Zuul
3d4e8d749d Merge "Move centos python3 installation after RHEL subscription" 2020-09-16 08:50:23 +00:00
Zuul
d8108885df Merge "Copy apt gpg keys directly into trusted.gpg.d" 2020-09-16 08:34:32 +00:00
Zuul
32441d052d Merge "update various gentoo bits" 2020-09-16 08:34:31 +00:00
Clark Boylan
c091227f4e Install gzip instead of busybox-gzip on suse
e2fsprogs pulls in info which needs gzip. But that conflicts with the
preexisting busybox-gzip. Install normal gzip to ensure that things can
proceed without conflict.

Change-Id: I3bed5bc141eaa0f3a15bfbf9da3e2d6e7c964a76
2020-09-14 16:00:47 -07:00
Clark Boylan
45688b20a2 Install grep before busybox on suse distros
busybox-grep is pulled in by the base image and it conflicts with rsync
package installs because rsync needs grep proper. Install grep proper
prior to installing the base distro to avoid this issue later when
trying to install rsync.

Change-Id: I2ec2bf39a2af214443a93776231657b25035e54f
2020-09-14 11:57:40 -07:00
Ian Wienand
8f12d9530e bootloader: remove dangling grubenv links
The rhel8.2 .qcow2 images have moved from a single xfs partition into
an EFI style layout with separate /boot/efi partition.

This means the root partition has /boot/grub2/grubenv as a symlink to
../efi/EFI/redhat/grubenv, which is dangling when we are running the
bootloader element as /boot/efi is blank.  grub-install tries to call
rename() in the process of re-writing this file, which fails and bails
out dib.

Remove this symlink if it exists

Change-Id: I5591144a3617dbae148b0c5d2a6a404942ffcd4e
Parital-Bug: #1893029
Redhat-Bug: https://bugzilla.redhat.com/show_bug.cgi?id=1871669
2020-09-08 18:04:42 +10:00
Noam Angel
e9b1997267 Move centos python3 installation after RHEL subscription
I088fc4284e889147ca9a375d4a159264cff53484 tried to slot the python3
install between the 00-dnf-update and before 01-package-installs;
however it also needs to run *after* the RHEL subscription
00-rhel-registration.

Thus a better place for it is 01-00-centos-python3, which will order
it after subscription and package updates, but before any use of
package-installs.

To avoid confusion over naming, move 00-0-dnf-update back to just
00-dnf-update.

Change-Id: Ib7c82895769e4889d47e10c4b37e06a42c053903
2020-09-07 11:22:55 +10:00
Matthew Thode
8cc08418d7 Copy apt gpg keys directly into trusted.gpg.d
This avoids having to have gnupg2/apt-key dependencies in the base,
and is now well supported by modern Debuntu.

Signed-off-by: Matthew Thode <mthode@mthode.org>
Change-Id: I7065b2fab6125d9635ef99ff65d374b8b6b4c3a2
2020-08-28 15:58:07 +10:00
Xinliang Liu
fd850475ea Make iscsi-boot element support centos 8
This patch makes iscsi-boot element support not only just
DISTRO_NAME centos7 but also centos and centos-minimal.

Change-Id: I8db8b01f35b2e572666badd8d2316d24a5e4287a
2020-08-25 09:28:27 +00:00
Matthew Thode
97e4627d87
update various gentoo bits
Update an rc-update call to only be made if running openrc instead of
all gentoo profiles (systemd does not have rc-update).

Add python3-pyyaml package mappings.

Update serial console to support multiple arches.

Update open-iscsi and open-isns keywords (looks like upstream merged
some musl fixes).

Update the kernel and initramfs file name globs for the
gentoo-kernel-bin usage.

Change-Id: I259bffed3a3e3f92be2210ead6bdfa383917d457
Signed-off-by: Matthew Thode <mthode@mthode.org>
2020-08-24 10:21:35 -05:00
Zuul
d140cc239a Merge "Revert "source-repositories: git is a build-only dependency"" 2020-08-20 20:37:48 +00:00
Paul Belanger
f093244611 Revert "source-repositories: git is a build-only dependency"
This breaks having git as a package-install in other elements. And results in diskimage-builder always erasing git on the image.

This reverts commit 3c65025559.

Change-Id: Ifb18c4243aa08055d98602aa4fdb30b16dae061d
2020-08-19 19:22:05 +00:00
Zuul
c2d719528e Merge "update gentoo to allow building arm64 images" 2020-08-19 04:35:28 +00:00
Zuul
e1995088af Merge "Remove glance-registry" 2020-08-18 08:12:17 +00:00
Zuul
5df71c0082 Merge "Makes EFI images bootable by bios" 2020-08-18 07:59:01 +00:00
Zuul
f49d6c996b Merge "Fedora 32 support" 2020-08-18 07:43:50 +00:00
Zuul
5c79f1cbd9 Merge "source-repositories: git is a build-only dependency" 2020-08-18 06:07:44 +00:00
Matthew Thode
bea81bd234
update gentoo to allow building arm64 images
Adds:

1. grub-efi package mappings
2. efi-64 support
3. default (openrc) arm64 profile
4. systemd arm64 profile

Cleans up the keywords and use flags in 02-gentoo-02-flags.  Most stuff
was stablized.  Also cleaned up some formatting for the if statements.

Enables less trusted overlays (up to the end user to verify).

in 10-gentoo-image I cleaned up some bash lint things as well.
using && instead of -a and avoiding $?

Change-Id: I3dffe1aab4bbdc4946a9bf2269bf0cde49529a4e
2020-08-17 23:50:39 -05:00
Pierre Crégut
6e71bd4239 Makes EFI images bootable by bios
For Bios and EFI compatibility, grub must be installed twice.
This patch adds the bios version when EFI is selected. The GPT EFI  block partitioning
already adds the bios partition, but the bootloader only called grub once.

Change-Id: Iee6c8b3b97b3cfff4562bcb30a50800f5ade894a
Closes-Bug: #1889089
2020-08-18 14:41:21 +10:00
Ian Wienand
7b6819213e Fedora 32 support
Update for Fedora 32 support.

Change-Id: I51c5645856a76e2877c013d72e9849a758ba12ff
2020-08-17 19:40:02 +10:00
Julia Kreger
46d12ae7d3 Handle NetworkManager for dhcp-all-interfaces
NetworkManager takes a distinctly differnet network management
approach and the bulk of the dhcp-all-interfaces code is largely
targetted at distribution specific configuration. Some which may
or may not override settings, or only partially assert desirable
settings.

As such, we need to set appropriate configuration, such as the
correct client to be used, and timeouts based upon user supplied
settings.

By default this change sets the client to be dhclient on redhat
styled machines, as the packaging default, while it works for
ramdisk usage, it does not reset the interface between retries,
which can be critical if the infrastucture operator is attempting
to configure LACP trunks to the end node.

Change-Id: I0e0cfbdbf7ef2b2861b934ccd7dab9d83a35c8f0
Story: 2008001
Task: 40648
2020-08-11 08:12:31 -07:00
Carlos Goncalves
3c65025559 source-repositories: git is a build-only dependency
Git is a build-time dependency that should not end up in the final
image should source-repositories be the only element requiring it at
image build time.

Change-Id: Id0798c0a753f893d02c2d13fddea0c04b28cb7ca
2020-08-11 15:53:15 +02:00
Carlos Goncalves
e4b6a2faef Add support for CentOS 8 Stream cloud image
This patch adds support for CentOS 8 Stream [1] to the centos element
(cloud image). Users should set DIB_RELEASE=8-stream.

[1] https://www.centos.org/stream/

Change-Id: Ib8f542031c46326ffed812fa60cbc9e56db9d6fd
2020-08-10 11:33:38 +02:00
Zuul
1291dcba51 Merge "Deprecate dib-python; remove from in-tree elements" 2020-08-07 09:57:12 +00:00
Zuul
ce3b6afd4d Merge "Pre-install python3 for CentOS" 2020-08-07 09:53:26 +00:00
Zuul
ab65277ca4 Merge "Fixes DIB_IPA_CERT certificate copy issue" 2020-08-07 03:03:35 +00:00
gugug
ae08dc8902 Remove glance-registry
glance-registry service was deprecated in Queens release and has been
removed in Victoria [1].

[1] https://review.opendev.org/#/c/738671/

Change-Id: Id2dfc3455b377b885521556f37f4b0f2da197655
2020-08-07 11:43:30 +10:00
Ian Wienand
8662297517 Deprecate dib-python; remove from in-tree elements
We are at the point that all distributions we are building have Python
3, so any tools running in the chroot can assume Python 3 exists.
This makes dib-python redundant; mark it as deprecated and start to
remove it from elements where it is no longer required.

Change-Id: I5d852843ec65d3b04444b77c54c5b82424455cd8
2020-08-07 10:38:16 +10:00
Ian Wienand
4dbfab66a1 Pre-install python3 for CentOS
CentOS 7 is the only distro we support currently that doesn't have
Python 3 installed in some form in the base images.  For centos 7 add
an early install of it in the yum element so we can have all the
in-chroot scripts assume Python 3.  There is only one package that
causes issues; yaml which comes from EPEL.  Everywhere else it is a
base package, but we don't have a way to say "enable epel to install
this".  Just hack it in, we don't want to go reworking the world for
CentOS 7 at this point.

Also add python3 and it's yaml library to the centos 8 path.  This
brings in the "user" python3 in /urs/bin/python3 (the "system" python3
is already installed).  Again, this just lets us assume
/usr/bin/python3 in scripts for all platforms.

package-installs is one of these things running python in the chroot,
and unfortunately we have elements that use it at 01- level in
pre-installd.  Thus to make sure python3 is there nice and early, run
it at 0 level, but make sure it comes after yum/dnf update.

Change-Id: I088fc4284e889147ca9a375d4a159264cff53484
2020-08-07 10:34:03 +10:00
Pierre Crégut
840f8ed2b2 Adds gnupg2 for apt-keys in ubuntu-minimal
Change present in debian-minimal pushed to ubuntu-minimal.

Change-Id: I736bffe6dd9fb57e450505d62c6ca000f678ded2
Closes-Bug: #1889076
2020-07-27 16:45:53 +02:00
Zuul
a7fba927ca Merge "Support non-x86_64 DIB_DISTRIBUTION_MIRROR variable for CentOS 7" 2020-07-24 09:38:36 +00:00
Jeffrey Zhang
581ffa023b Support non-x86_64 DIB_DISTRIBUTION_MIRROR variable for CentOS 7
Other architectures are stored under "altarch" for CentOS 7, update
the match.

Convert the delimiters to "," to avoid a subtle problem with "|" --
POSIX states

  Within the BRE and the replacement, the BRE delimiter itself can be
  used as a literal character if it is preceded by a backslash.

So "s|\(foo\|bar\)|moo|" doesn't do what you might think; the inner
pipe becomes a literal | and this will *not* match "foo" or "bar".

Change-Id: Ic1642325e3a59a10453c356d8d839ce649812af8
2020-07-20 10:33:01 +10:00
vmud213
90eafe078c Fixes DIB_IPA_CERT certificate copy issue
Change-Id: I0b4c5d8fa5a45aeb4d5d999ca5b7e0798162b92b
2020-07-17 09:48:30 +00:00
Matthew Thode
2559933f8f
add openrc init system support to serial console element
Change-Id: Id58e53a592b5f4601b893ba6c828bad07942b07b
Signed-off-by: Matthew Thode <mthode@mthode.org>
2020-07-14 13:59:09 -05:00
Zuul
70892a7a16 Merge "update gentoo-releng gpg key" 2020-07-14 00:39:15 +00:00
Zuul
ea6403388e Merge "Switch from unittest2 compat methods to Python 3.x methods" 2020-07-13 02:21:12 +00:00
Your Name
e384da1a98
update gentoo-releng gpg key
Simplify gpg checking by caching a keyring instead of keys to import.

Change-Id: I5ed74ec0e12732aec40ef31377e72d7ddc347f95
Signed-off-by: Matthew Thode <mthode@mthode.org>
2020-07-12 17:50:00 -05:00
Michael Johnson
8b08d212c3 Fix DIB scripts python version
Now that DIB is python3 only we can remove a hack that made sure
scripts outside the chroot ran with the correct version of python.
This is necessary as python3 does not resolve symbolic links to the
binary like python2.x did, which causes element scripts to fail finding
modules when DIB was run from inside a venv.

This patch does the following:
1. Reverts 9c7b8d1714 which was the
   workaround for mixed python2/3 environments.
2. Updates the scripts to use "python3" instead of "python".

Change-Id: If2402bb02fc8a4778fa9434fa167ea1fafd87c28
2020-07-07 12:53:51 -07:00
Zuul
b6d4bef9eb Merge "Use kpartx option to update partition mappings" 2020-07-07 08:27:29 +00:00
melissaml
bf0dc265ae Switch from unittest2 compat methods to Python 3.x methods
With the removal of Python 2.x we can remove the unittest2 compat
wrappers and switch to assertCountEqual instead of assertItemsEqual

We have been able to use them since then, because
testtools required unittest2, which still included it. With testtools
removing Python 2.7 support [3][4], we will lose support for
assertItemsEqual, so we should switch to use assertCountEqual.

[1] - https://bugs.python.org/issue17866
[2] - https://hg.python.org/cpython/rev/d9921cb6e3cd
[3] - testing-cabal/testtools#286
[4] - testing-cabal/testtools#277

Change-Id: I870286a2557e41099597c22dc9747743e1077615
2020-07-07 11:11:28 +08:00
Zuul
9f7c35b238 Merge "add musl profile to gentoo" 2020-07-06 18:52:39 +00:00
Zuul
131ddac3cb Merge "Stop bringing the test environment into the mocks" 2020-07-06 13:28:27 +00:00
Zuul
7e00908a28 Merge "add more python selections to gentoo" 2020-07-06 02:59:59 +00:00
Zuul
5d16300f00 Merge "Download latest CentOS cloud image" 2020-07-05 21:06:43 +00:00
Zuul
9ab6811e84 Merge "Add support for CentOS 8 Stream" 2020-07-05 21:06:41 +00:00
yatinkarel
e999368d2c Disable all enabled epel repos in CentOS8
EPEL centos8 have different epel repos like
epel, epel-modular, epel-playground etc, so
we need to disable all not just epel.

Also ensure other epel repositories in CentOS7
like epel-testing are also disabled.

Related-Bug: #1885315
Change-Id: I02b3b83fa2047702d5f069d3ca1c9c0bcc1dab52
2020-06-30 10:15:30 +05:30
Matthew Thode
64787d8ec2
add musl profile to gentoo
Change-Id: Id954ffe4d9b019c7e7bb648725ff7f976c929b4d
2020-06-22 18:37:30 -05:00
Carlos Goncalves
1f9619cdb5 Download latest CentOS cloud image
DIB was retrieving the oldest cloud image file which, presently, means
retrieving CentOS 8.1 instead of CentOS 8.2. Even though DIB runs a
system update and so catches up to latest, this takes bandwidth, time
and final image space (8.1 + system update = 765M qcow2, vs 8.2 + system
update = 518M qcow2).

This patch fixes that by taking the first image name in a descending
order list.

Change-Id: I648fe19f1f76c03c97492b6ac7be6381f6f9261b
2020-06-22 11:03:36 +02:00
Carlos Goncalves
367dfc9294 Add support for CentOS 8 Stream
This patch adds support for CentOS 8 Stream [1] to the centos-minimal
element. Users should set DIB_RELEASE=8-stream.

[1] https://www.centos.org/stream/

Change-Id: Id0825de735ab957c10daf35fb3c641f850cc6847
2020-06-22 10:36:30 +02:00
Matthew Thode
c7b6f0d6a1
add more python selections to gentoo
Change-Id: If4420e9ae7870739aac8a673ff290f9ed1391acf
2020-06-14 01:16:18 -05:00
Matthew Thode
52a8b2fa01
update grub cmdline to current kernel parameters
vga=normal is depreciated, use gfxpayload=text instead

Change-Id: I2e5be3b07acce14a9f3d47e24938d9da3c5b6c48
2020-06-14 00:39:42 -05:00
Chris MacNaughton
8f57ed9b9e
Stop bringing the test environment into the mocks
Change-Id: I1780f186cf107242cacd6d3da7a5bc81a330b536
Closes-Bug: #1883224
2020-06-12 10:37:36 +02:00
Zuul
2e6a7f949c Merge "Remove virtualenv activation" 2020-06-12 01:17:17 +00:00
Dmitry Tantsur
e793cc4038 Remove virtualenv activation
Since the original merge of this code
(04208e7c79) several things have
changed; particularly now we ship dib-run-parts as part of dib, not as
a separate package.

We setup $_LIB to point to the shipped library diretory via
pkg_resources lookups.  We now call dib-run-parts (as mentioned,
shipped as a dib library now), source scripts, etc. via $_LIB and thus
do not rely on $PATH.  Consequently we don't need this activation
part.

Which is helpful, because "venv" (as opposed to virtualenv) doesn't
have activate_this.py.  So this fixes installation under that for
Python 3.

We update the functional tests to use the virtualenv_command exported
by the ensure-pip role, which will test the venv path.  There is no
need for dib_python as we are Python 3 only now.

Change-Id: Iede929ea2d278008220aac8b1d678ba41eba0d8a
2020-06-11 16:49:15 +10:00
Zuul
3614900572 Merge "Fix yumdownloader cache dir" 2020-06-10 16:24:53 +00:00
Zuul
a0714c2300 Merge "Debuntu: add apt-transport-https" 2020-06-09 10:33:56 +00:00
Zuul
caf72a4c56 Merge "Drop six usage" 2020-06-09 10:28:14 +00:00
Simon Westphahl
4a424ecabb Use kpartx option to update partition mappings
Fix cases of 'mkfs' failing because the partitions never showed up. Partition
mappings will now be updated instead of just adding them with 'kpartx'. That
means that 'kpartx' will also remove devmappings for deleted partitions.

Traceback of failing mkfs call:

2020-05-11 22:03:25.523 | INFO diskimage_builder.block_device.utils [-] Calling [sudo sync]
2020-05-11 22:03:25.539 | INFO diskimage_builder.block_device.utils [-] Calling [sudo kpartx -avs /dev/loop0]
2020-05-11 22:03:25.581 | INFO diskimage_builder.block_device.utils [-] Calling [sudo mkfs -t ext4 -i 4096 -J size=64 -L cloudimg-rootfs -U 21c6f9eb-4d52-4e5c-b9b7-796735de8909 -q /dev/mapper/loop0p1]
2020-05-11 22:03:25.700 | ERROR diskimage_builder.block_device.blockdevice [-] Create failed; rollback initiated
2020-05-11 22:03:25.700 | Traceback (most recent call last):
2020-05-11 22:03:25.700 |   File "/home/zuul/dib/lib/python3.6/site-packages/diskimage_builder/block_device/blockdevice.py", line 406, in cmd_create
2020-05-11 22:03:25.700 |     node.create()
2020-05-11 22:03:25.700 |   File "/home/zuul/dib/lib/python3.6/site-packages/diskimage_builder/block_device/level2/mkfs.py", line 133, in create
2020-05-11 22:03:25.700 |     exec_sudo(cmd)
2020-05-11 22:03:25.700 |   File "/home/zuul/dib/lib/python3.6/site-packages/diskimage_builder/block_device/utils.py", line 143, in exec_sudo
2020-05-11 22:03:25.700 |     raise e
2020-05-11 22:03:25.700 | diskimage_builder.block_device.exception.BlockDeviceSetupException: exec_sudo failed
2020-05-11 22:03:25.700 | INFO diskimage_builder.block_device.level0.localloop [-] loopdev detach
2020-05-11 22:03:25.701 | INFO diskimage_builder.block_device.utils [-] Calling [sudo losetup -d /dev/loop0]
2020-05-11 22:03:25.732 | INFO diskimage_builder.block_device.level0.localloop [-] Remove image file [/tmp/dib_image.muyw7t1h/image0.raw]
2020-05-11 22:03:25.734 | ERROR diskimage_builder.block_device.blockdevice [-] Rollback complete, exiting
2020-05-11 22:03:25.740 | Traceback (most recent call last):
2020-05-11 22:03:25.740 |   File "/home/zuul/dib/bin/dib-block-device", line 8, in <module>
2020-05-11 22:03:25.740 |     sys.exit(main())
2020-05-11 22:03:25.740 |   File "/home/zuul/dib/lib/python3.6/site-packages/diskimage_builder/block_device/cmd.py", line 120, in main
2020-05-11 22:03:25.740 |     return bdc.main()
2020-05-11 22:03:25.740 |   File "/home/zuul/dib/lib/python3.6/site-packages/diskimage_builder/block_device/cmd.py", line 115, in main
2020-05-11 22:03:25.740 |     self.args.func()
2020-05-11 22:03:25.740 |   File "/home/zuul/dib/lib/python3.6/site-packages/diskimage_builder/block_device/cmd.py", line 36, in cmd_create
2020-05-11 22:03:25.740 |     self.bd.cmd_create()
2020-05-11 22:03:25.740 |   File "/home/zuul/dib/lib/python3.6/site-packages/diskimage_builder/block_device/blockdevice.py", line 406, in cmd_create
2020-05-11 22:03:25.740 |     node.create()
2020-05-11 22:03:25.740 |   File "/home/zuul/dib/lib/python3.6/site-packages/diskimage_builder/block_device/level2/mkfs.py", line 133, in create
2020-05-11 22:03:25.740 |     exec_sudo(cmd)
2020-05-11 22:03:25.740 |   File "/home/zuul/dib/lib/python3.6/site-packages/diskimage_builder/block_device/utils.py", line 143, in exec_sudo
2020-05-11 22:03:25.740 |     raise e
2020-05-11 22:03:25.740 | diskimage_builder.block_device.exception.BlockDeviceSetupException: exec_sudo failed

Change-Id: I374f7f22f9e93ef35eb5813712ca59e75f0733e8
Related-Bug: #1698337
2020-06-09 09:07:55 +02:00
Dmitry Tantsur
1e1e36df64 Do not fail in a venv when activate_this.py is not found
The standard Python venv module does not have this script, so currently
DIB unconditionally fails. While a real fix will be provided in
https://review.opendev.org/#/c/704478/ this change at least allows
users to try work around the problem.

Change-Id: I45b79d4d283f2b3ea909612e652672dcb6092488
2020-06-09 12:53:21 +10:00
Carlos Goncalves
9d9dd9249c Fix yumdownloader cache dir
TMPDIR env is not being honored as cache dir in EL 8.

Change-Id: I8281675ec5f0951b3e190a8d6727744a1a5cd8d7
2020-06-07 21:58:49 +02:00
Andreas Jaeger
4493208048 Drop six usage
With python3, six is not needed anymore, drop it.

Change-Id: I70bb679270605ac32ca0cceb9414ea3a210e5842
2020-06-05 12:04:37 +02:00
Ian Wienand
0c94eef7be Revert "dib-lint: use yamllint to parse YAML files"
This reverts commit 6ee2995214 and
e85c2a6f03.

I missed that if you pip install and then run dib-lint, it's not going
to pick up the .yamllint file shipped here.  Thus it gives spurious
errors.

The reason for this was simply better duplicate key detection in yaml
files, which caused us problems with the kernel installs.  However, at
this point it seems just the old "does it load" test from pyyaml will
be enough.

Change-Id: I87a9fc9bb119cfeffad48fc0fa0df31f0181825d
2020-05-28 16:44:49 +10:00
Ian Wienand
d2eaaaf622 Merge "Pre-install xz package in opensuse chroot" 2020-05-28 02:06:30 +00:00
Matthew Thode
041bdd331a
use stage3 instead of stage4 for gentoo builds
The main reason for using the stage4 is now gone (kernel compile).
Install and use the distro provided binary kernel package.

In addition to this, set the locale and timezone, beyond that very
little was done in the gentoo stage4.

Change-Id: I541b7d9b807e2357398ae1c249b1978958dd1137
Signed-off-by: Matthew Thode <mthode@mthode.org>
2020-05-27 18:50:16 -05:00
Colleen Murphy
19b3586d08 Pre-install xz package in opensuse chroot
As of recently, opensuse-minimal images fail to build because of an
error installing the kernel-default package:

> Problem: kernel-default-5.6.12-1.3.x86_64 requires mkinitrd >= 2.7.1, but this requirement cannot be provided
>   not installable providers: dracut-050+suse.61.g0fe0e854-1.1.i586[repo-oss]
>                    dracut-050+suse.61.g0fe0e854-1.1.x86_64[repo-oss]

The problem is there is a recently added package `busybox-links` which
provides a subpackage `busybox-xz` which provides the /usr/bin/xz
utility. Since this is available, the `aaa_base` package installs it
during the root.d base installation phase to fulfill it's dependency on
/usr/bin/xz. On the other hand, the dracut package explicitly requires
the `xz` package, and this is not co-installable with the `busybox-xz`
package, so the dracut package is not installable during the install.d
phase. This change explicitly adds the `xz` package to the initial
chroot provisioning phase so that the /usr/bin/xz requirement is already
fulfilled and `busybox-xz` does not get installed.

Change-Id: Iba8c301eb496657873963e1aa99736aacf87cb00
2020-05-27 10:45:03 -07:00
Ian Wienand
339d713527 Revert "Revert "ubuntu-minimal : only install 16.04 HWE kernel on xenial""
This reverts commit 6e549c33ac.

It uses the new multiple-parameter matching format from
Idff7b067ad4255e6fc4138f7eff313a81b75c8ba to actually do what it says.

Change-Id: I4656ff1a5c46bcfbd8587f2f541825f4ad08820f
2020-05-27 06:18:02 +10:00
Ian Wienand
b71d1c60d2 package-installs : allow a list of parameters
The change Ia6f10741fa6be24b11d6991c8a6b6e07951ff68d introduced having
"when:" as a list of values.  However, this was actually not
sufficient to express the logic required for arm64/x86_64/xenial
kernel matching we wanted.

Because the package name is a key, we can't have multiple entires in
the package-map YAML files.  This means we can't do more advanced
matching and thus we need to be able to match through multiple
parameters.  Similar to Ia6f10741fa6be24b11d6991c8a6b6e07951ff68d we
modify the matching rules to allow a list.

A an example of using this is provided in the README.rst, and this
same example worked through by the unit tests.

This also slightly updates the matching logic to be more sequential.
After each check we either continue on or log the failure and continue
to the next check (rather than set a list of flags then check that at
the end).  This makes it much easier to understand what is being
matched in the logging output from the tool.

Change-Id: Idff7b067ad4255e6fc4138f7eff313a81b75c8ba
2020-05-27 06:17:57 +10:00
Ian Wienand
6ee2995214 dib-lint: use yamllint to parse YAML files
This gives us better linting of YAML files that just opening them.
This would have detected the duplicate keys in
I34e27d821fbefe274e7b007f37b0bd34db2e1d26.

The .yamllint is taken from zuul-jobs where it is also used as a
fairly sane set of default rules.

A few minor newline fixes are added.

Change-Id: I96d6644ae24f7deb84fa50fefbda0f0d33e0e009
2020-05-26 12:04:09 +10:00
Ian Wienand
6e549c33ac Revert "ubuntu-minimal : only install 16.04 HWE kernel on xenial"
This reverts commit 14ff8f942c.

This seems to not be installing the kernel at all, and needs further
investigation.

Change-Id: Ifd809d4b67aff5d80f979235db246a16af0375b3
2020-05-25 18:48:51 +10:00
Ian Wienand
7539e241da ubuntu-minimal: Add Ubuntu Focal test build
Add test builds for Focal on x86 and ARM64

Change-Id: Idb23f0e00d37c7447441ea002aad078e8c61f969
2020-05-21 14:03:54 +10:00
Ian Wienand
14ff8f942c ubuntu-minimal : only install 16.04 HWE kernel on xenial
Only install the HWE kernel by default for Xenial.  This was actually
installing the 16.04 HWE kernel on Bionic by accident, since it seems
to have that package; however it was breaking Focal.

On the other distros, just install the default generic kernel.  Let's
KISS for now if we can ...

Change-Id: I34e27d821fbefe274e7b007f37b0bd34db2e1d26
2020-05-21 14:03:54 +10:00
Ian Wienand
c5acc91574 ubuntu-minimal: fix HWE install for focal
On Focal, install the 20.04 HWE kernel if linux-image-hwe is
specified.

Change-Id: I5d536e8f64e7b987415849d8cd4da7959bba7af7
2020-05-21 14:03:54 +10:00
Ian Wienand
138d3f9b81 package-installs: allow when filter to be a list
Allow the "when:" statements to be a list of values, which are
effectively anded together to filter the package install.

Change-Id: Ia6f10741fa6be24b11d6991c8a6b6e07951ff68d
2020-05-21 14:03:49 +10:00
David Hill
68912bb76f Disable all repositories after attaching a pool
Disable all repositories after attaching a pool as in some cases that
I still can't explain, the "-htb-" repositories are enabled by default
which causes problems later on when trying to install the packages.

Change-Id: I86b3baccd4b0932eb3686a17485f64ee09994dad
2020-05-13 17:18:19 -04:00
Ian Wienand
9080d04923 block device: update variable name
New versions of flake8 fail as "l" can be confused for "1" apparently
(E741) ... not sure I totally agree but since it's only one instance,
update it.

Change-Id: Ic5c47867facd56b53cc6534da4ae3a345c516202
2020-05-13 06:22:02 +10:00
Zuul
534b799de8 Merge "pip-and-virtualenv : fix fedora 30 install" 2020-04-24 05:34:35 +00:00
Zuul
0d0a5909e7 Merge "yum-minimal: strip env vars in chroot calls" 2020-04-24 05:34:33 +00:00
Ian Wienand
a8a5b1ac77 pip-and-virtualenv : fix fedora 30 install
This should be installing the python2 and python3 packages (that's
what pip-and-virtualenv is designed to do), but we dropped the +=
accidentally in ee9ad32b6f.

However, we've moved on anyway and after
I7a6a342461d6001c25e55638ba9b7438c28f2519 F31 doesn't support this
element.  fedora-latest is already updated to f31 in the opendev gate.
Remove the testing as it is no longer relevant.

Change-Id: Id696a90baa1eb05cb4c08501f8dac3665d395682
2020-04-24 12:42:35 +10:00
Ian Wienand
df3ad26f58 yum-minimal: strip env vars in chroot calls
This showed up with dnf in containers when TMPDIR was set; dnf started
trying to write to this directory while in the chroot.

We already do stripping like this in run_in_target -- but this is a
bit of a unique place because it's actually setting up the initial
chroot so the target doesn't actually exist yet; so we just hard-code
it in place here.

Change-Id: If7310cb820846da903bf60daa4486c8bf7cb0136
2020-04-24 12:38:42 +10:00
Ian Wienand
ac7fba7040 pip-and-virtualenv: drop f31 & tumbleweed, rework suse 15 install
This is an alternative approach to commit
68bb43535e.  I think this proposes a
better overall solution that the prior change which had the Python 3
packages being installed, but did not specify the _do_py3 flag to do
the installation steps that redirect the various tool installations.

Fedora 31+ doesn't have python2, and Tumbleweed does have some Python
2 support but there seems to be no reason to bother updating this
element for either with infra very close to removing this completely
[1].  Error out on these platforms, and add a release note.

The 15 path should include the python2 and python3 packages, along
with the flags to do the "cleanup"; i.e. forced removal of distutils
packages that pip 10+ won't touch.  As mentioned in the original
change, the six package causes problems here, but we can clear that
too by explicitly listing it instead of letting it come in via
dependencies.  Again, this element will be removed from the infra 15
builds ASAP; but we can release with this to provide a roll-back point
if we need to revert the removal to fix things temporarily.

Add it to the testing path as well.

[1] https://docs.opendev.org/opendev/infra-specs/latest/specs/cleanup-test-node-python.html

Change-Id: I7a6a342461d6001c25e55638ba9b7438c28f2519
2020-04-23 08:10:26 +10:00
Zuul
0445a5ce2b Merge "Add centos aarch64 tests" 2020-04-17 01:18:54 +00:00
Ian Wienand
3f823488c5 Add centos aarch64 tests
Change-Id: I2fffeeffa82c505a34c5c9972724b557f3b58975
2020-04-16 14:15:50 +10:00
Marcin Juszkiewicz
18e35893f7 Do not try to use MBR on AArch64
When I tried to build CentOS8 image for AArch64 I got error saying that
MBR is not supported. So make sure that it will not be used by default.

Change-Id: Ib67ab7f808d727c3c61932c540d398dbe723972f
2020-04-14 10:12:57 +00:00
Dirk Mueller
68bb43535e opensuse: fix python 2.x install
openSUSE Tumbleweed is dropping python2-* packages so we need
to stop intalling them. We can also stop installing those
for Leap 15. which avoids a pip uninstall issue (as python2-six
was still built with distribute).

Change-Id: Ie93c8addb26aab3d0154c4b5b52423799abede91
2020-04-03 18:22:33 +02:00
Ian Wienand
434d2db5d4 Debuntu: add apt-transport-https
I don't see anywhere we bring this in, especially on a minimal build.
In 2020 it seems like a base dependency, put it alongside
software-properties-common that installs the other apt helper bits.

Change-Id: I5b079eac4912cb4a164e9aa6158ed106a28f576c
2020-04-02 10:11:35 +11:00
Zuul
ac250e5ab0 Merge "centos 8 image build: fix mirror" 2020-03-31 05:03:06 +00:00
Zuul
dc67ccfe12 Merge "Mellanox element: removed ibutils,libibcm,libmlx4-dev" 2020-03-30 22:10:13 +00:00
Ian Wienand
56dc0a2c31 centos 8 image build: fix mirror
We're ending up with "centoscentos" in the mirror location and the
build fails; strip out the $contentdir from the original too.

Change-Id: If09dbbd8028ea510d2ab0d3d8afe484cea611df5
2020-03-31 08:57:15 +11:00
Zuul
21f9ea4f78 Merge "Add Fedora 31 support and test jobs" 2020-03-30 01:26:32 +00:00
Lee Yarwood
ee9ad32b6f Add Fedora 31 support and test jobs
Change-Id: Iad0261ac8db001ffa4d3a4dd6df05fe923402a69
2020-03-30 08:47:00 +11:00
Zuul
56d9956cbd Merge "Add python-stow-versions element" 2020-03-27 13:49:21 +00:00
Lenny Verkhovsky
2f4e1886e5 Mellanox element: removed ibutils,libibcm,libmlx4-dev
Those packages will be installed as part of dependencies

Change-Id: I640dc55c818dd8538bfcd199c33f9af93aae7608
2020-03-23 00:06:53 +02:00
Harald Jensås
1ac31afd62 Use rpm -e instead of dnf for cleaning old kernels
If the running kernel of the system building the image
matches the kernel that is to be removed dnf will fail.

Repalce use use of dnf with rpm -e.

Closes-Bug: #1623409
Change-Id: Ie2481ea8a02b7b0720e46fa179f24badf4aa25c5
2020-03-19 22:35:23 +01:00
Dmitriy Rabotyagov
1de6fe4ba7 Add python-stow-versions element
This element is designed to install latest minor versions of different
python releases, like py27, py35, py36, py37, py38
into stow directory, and later easily enable them with stow.

Change-Id: Iab6d20e7643e549b53c629fb430e58b1c5e72991
2020-03-19 21:21:01 +02:00
Monty Taylor
c113703050 Add support for build-only packages
Sometimes an element needs packages installed so that it can
perform tasks but those package are not appropriate for the
final image content. Add a "build-only" flag to package-install-squash
which will cause package to be installed at the beginning of the
phase and then uninstalled at the end of the phase.

Change-Id: Ie01b795991710c93f6b669c8f14b57eb4412c1d5
2020-03-18 14:40:44 -05:00
Zuul
61b72ca2c2 Merge "Add ensure-venv element, install glean with it" 2020-03-10 05:08:54 +00:00
Zuul
991586c20a Merge "Uncap hacking" 2020-03-10 01:16:02 +00:00
Ian Wienand
82cfcfe551 Add ensure-venv element, install glean with it
All the platforms we care about now have python3 with venv (even
centos7 now) packaged somehow.  Add an ensure-venv element to make
sure that "python3 -m venv" works.  Any other elements that wish to
install non-distribution-packaged Python utilities can use this to
keep them separate from the main system installs.

Port glean to use this, and drop its dependency on pip-and-virtualenv.

Change-Id: Ic16f134fe34293bb68e7c632dd320f523366320d
2020-03-10 11:57:43 +11:00
Zuul
18f46bde30 Merge "fix iscsi-boot element exiting build even if dracut-regenerate used" 2020-02-26 04:05:57 +00:00
Zuul
8deca122f9 Merge "Change tgt pkg-map to target-restore CentOS/RHEL-8" 2020-02-26 04:05:56 +00:00
Zuul
8ad2956911 Merge "Do not include efibootmgr and efivars for ppc architectures" 2020-02-26 04:02:42 +00:00
Bob Fournier
14a3776885 Do not include efibootmgr and efivars for ppc architectures
These don't exist and are not needed for ppc.

Change-Id: I9ba53e22583b148b43f74cd9b8ec79402796d09b
2020-02-25 13:27:13 -05:00
Chandan Kumar (raukadah)
31a32ff6ee Change tgt pkg-map to target-restore CentOS/RHEL-8
In ironic-agent tgt pkg-map, scsi-target-utils was used for
redhat family which is not used in CentOS/RHEL-8.

As per https://opendev.org/openstack/ironic-python-agent-builder/src/branch/master/dib/ironic-python-agent-ramdisk/pkg-map#L5
targetcli got removed and switch to target-restore. It syncs
the same element for RHEL-8 and adds the same for CentOS-8.

Closes-Bug: #1864427

Change-Id: I95a0c09a6739af23cd1e8c88dded198bd69cc53e
Signed-off-by: Chandan Kumar (raukadah) <chkumar@redhat.com>
2020-02-24 14:17:24 +05:30
Ian Wienand
28ebd24844 Uncap hacking
This causes problems for other projects incorporating dib; we don't
have a specific need for a cap.

Fix a few issues, mostly spacing or regex matches.  No functional
changes.

W503 and W504 relate to leaving artithmetic operators at the start or
end of lines, and are mutually exclusive and, due to "ignore"
overriding the defaults both get enabled.  It seems everyone gets this
wrong (https://gitlab.com/pycqa/flake8/issues/466).  Don't take a
position on this and ignore both.

Use double # around comments including YAML snippets using "# type: "
which now gets detected as PEP484/mypy type hints.

Change-Id: I8b7ce6dee02dcce31c82427a2441c931d136ef57
2020-02-24 10:34:46 +11:00
Carlos Goncalves
8226384cf0 Add CentOS 8 support
* Add "centos" element, a CentOS version-independent element. This is in
  line with the same work done for RHEL in Stein cycle.
* Deprecate the centos7 element. CentOS 7 support itself it not
  deprecated though. The new "centos" element provides the same support
  level as the "centos7" element.
* Add functional testing

The default CentOS version is 8. You can adjust it using the DIB_RELEASE
environment variable.

Change-Id: I373ba2296c4613765676e59aabd9c651345298d1
2020-02-19 10:44:56 +01:00
Zuul
9d08848f25 Merge "Fix cache-url -f" 2020-02-18 22:55:23 +00:00
Zuul
0ec74b948c Merge "Allow python3 to be used in Debian" 2020-02-17 18:18:07 +00:00
Noam Angel
48eac8b899 fix iscsi-boot element exiting build even if dracut-regenerate used
in CentOS build case building an image with "iscsi-boot" and "dracut-regenerate" will exit building because of statement "[ "$found" = 0 ]"

Change-Id: I1a6d60e9ec5f5cb508866c8376465c3e73551a30
2020-02-12 09:09:20 +00:00
Iury Gregory Melo Ferreira
6986441e2c Add efi packages for ironic-agent
On IPA we are using efivar and efibootmgr, we already added the
packages on ipa-builder.
Adding the pacakges on diskimage-builder, so that people who use
it to build the images won't get into trouble.

Change-Id: I9ab6588f20302b4808b09dc060aced5fd267a3d2
2020-02-11 17:26:59 +01:00
Riccardo Pittau
cf7d39e4cd Allow python3 to be used in Debian
Debian default Python interpreter version is 2.7, but it's
possible to install a Python 3 interpreter from the base
repository.
With this change, if we set DIB_PYTHON_VERSION to 3, we install
the python3 package from base, with python3-libs, python3-pip and
python3-setuptools, and redefine python_path, effectively allowing
Python 3 interpreter to be used in Debian.
See a result of the job for building the ipa image here:
https://review.opendev.org/705773

Change-Id: Idabfa94c2bff6e0de6daa0866084d5db14d7dcb0
2020-02-07 18:10:19 +01:00
Tobias Henkel
bad433fa92
Fix cache-url -f
When there is a hashsum mismatch diskimage-builder forces downloads
with the -f switch of cache-url. This is currently broken because bash
escapes the quotes in curl_opts. This tricks curl trying to download
'no-cache' instead of the url. This can be fixed by using an array for
curl_opts which does the correct thing here.

Change-Id: Id9f1579dda9a3e0a2b08dd5faaeef0e2e580d419
2020-02-05 10:19:12 +01:00
Zuul
500e60dbf4 Merge "Enable possibility to select HWE kernel for Ubuntu minimal" 2020-02-04 09:00:47 +00:00
Antoine Musso
168127b60a dib-lint: test elements have README.rst file
Add a basic test to ensure that all elements have a README.rst file.
This way they will be exhaustively listed in the Sphinx documentation.

Add dummy README.rst for 'disable-selinux' and 'rpm-distro' elements.

Change-Id: Ia5252ddd89b5ae5c6e9a12a66ef10f912fd54da5
2020-01-20 11:43:43 +01:00
Fatih Degirmenci
e2af38e3a5 Enable possibility to select HWE kernel for Ubuntu minimal
Change-Id: I206a77590f575e472e31b4f867fd5fd35475542d
2020-01-17 10:47:37 +00:00
Carlos Goncalves
ae2be0b464 Fix Yum repositories and GPG keys for CentOS 8.1
CentOS 8.1 split repositories and GPG keys out into subpackages. This
broke DIB support for CentOS 8.

7e41cef418
26a0d73ced

Change-Id: If3de6efa6074e059dc9fdd47c7bdc19d26d4d7f2
2020-01-15 19:39:00 +01:00
Matthew Thode
87e7f7b869
allow building of Gentoo images for non-systemd profiles
Change-Id: I61e3ba391eaaf9b300f88b082c03116388401ba3
2019-12-20 17:51:55 -06:00
Mohammed Naser
6f1b51627f modprobe.d: use $TMP_MOUNT_PATH
The hook inside extra-data.d runs outside the chroot when
building the image which means that we need to prefix paths
inside the hook to avoid running things on the host.

We also run it with sudo because if we're running DIB not
as root, /etc is uid 0 and we'll get a permission denied.

Change-Id: I1838890fe124c84c879285a471bcc78fe47d6c23
2019-12-18 11:42:02 -05:00
Zuul
71e1bcbdb8 Merge "Install rng-tools in Red Hat family distro images" 2019-12-17 06:25:32 +00:00
Zuul
f6d32a684f Merge "Add arm64 based functional test" 2019-12-16 21:31:51 +00:00
Ian Wienand
082397a86a Add arm64 based functional test
An initial functional test for bionic/arm64 builds, put it in the new
arm64 check queue.

Change-Id: I5f8a047f41c6555da7211b758c55f7a87b3aa5d1
2019-12-16 10:51:01 +11:00
Zuul
db9501cab2 Merge "Fix wrong URL in ironic-agent element" 2019-12-15 23:43:33 +00:00
Carlos Goncalves
3ff7365ee8 Install rng-tools in Red Hat family distro images
Make sure rngd, a hardware RNG entropy gatherer daemon, is installed on
all DIB-built Red Hat family distro images. rngd comes installed by
default in a typical base installation as it's proven to help speed
things up.

Nova attaches the virtio-rng-pci device to VMs. virtio-rng-pci is a
device that provides feed random data. However, it is of little to no
use if the virtual machine is not configured to make use of given
device. That is where rngd can help by facilitating entropy to the pool
from virtio-rng-pci.

$ openstack image set --property hw_rng_model=virtio [...]
$ openstack flavor set --property hw_rng:allowed=True [...]

DIB-built minimal images do not come with rngd installed. This patch
makes sure the daemon is installed. Its systemd service comes already
enabled.

Change-Id: I34a989dbfc57d4c98113ac25c81dfb500945ff0a
2019-12-12 20:26:06 +00:00
Zuul
74dff74907 Merge "Set correct python version for non-chroot scripts" 2019-12-12 16:48:17 +00:00
Madhuri Kumari
46491eb1b5 Fix wrong URL in ironic-agent element
Change-Id: Iaa107094b1409f93b101b23cea2e874c60ea26f5
2019-12-12 16:39:48 +05:30
Michael Johnson
9c7b8d1714 Set correct python version for non-chroot scripts
Some phases of diskimage-builder run outside the chroot environment,
such as the extra-data.d scripts, and don't have access to dib-python.
This means these scripts may choose the wrong python version by using
"#!/usr/bin/env python" to execute. The svc-map element is an example.

This patch creates a temporary directory and symbolic link for the
correct version of python, then manipulates the environment PATH
to preference the symbolic link "python" command.
This will allow elements with these scripts to work correctly with
the version of python diskimage-builder is running under.

Change-Id: I289d621e1bfbba0eb174dff977d1a5c92c04e4fa
Co-Authored-By: Ian Wienand <iwienand@redhat.com>
2019-12-11 22:52:28 +00:00
Ian Wienand
801fa77a16 Work around Trusty ext4 metadata_csum errors on Bionic
As described inline, Bionic hosts will build invalid Trusty images.
Hack around this by disabling metadata_csum in the ext4 mkfs.

Change-Id: Ibd67d58ca830a9e60605d0700ee2b17906c804e6
2019-12-11 16:02:13 +11:00
Zuul
361a751d36 Merge "Allow zypper repos to be overrideable" 2019-12-04 06:14:55 +00:00
Zuul
68b41854b2 Merge "Install ndisc6 package in element script" 2019-12-03 22:41:09 +00:00
Zuul
9b00377f31 Merge "Fix regex for mirror URL substitution" 2019-12-03 21:56:19 +00:00
Zuul
0b1eba25d5 Merge "Break retry loop on success in dhcp-all-interfaces" 2019-12-03 21:56:18 +00:00
Zuul
e3a904d4a6 Merge "Fix login.defs config for tumbleweed" 2019-12-03 21:56:16 +00:00
Carlos Goncalves
670df3326d Fix regex for mirror URL substitution
The base URL of EPEL repository installed by the epel-release package in
CentOS 8 at least now defaults to https.

The error seen when building an CentOS 8 image was:
"Error: Cannot find a valid baseurl for repo: epel"

This patch fixes it so that it will always match regardless of being
http or https.

Change-Id: I9ec5536ee72047c929a1ef6950ff4e9092842a4c
2019-12-03 19:13:18 +01:00
Harald Jensås
7948fee7e2 Install ndisc6 package in element script
The ndisc6 package is not yet available in EPEL 8.
See: https://bugzilla.redhat.com/show_bug.cgi?id=1779134

Until the package is available set the pkg-map to "" for
the ndisc6 package when distro is redhat and install the
package using || true in the element script instead so
that CentOS 8 build's do not fail because of the missing
package.

Once the package is in EPEL 8 this change can be reverted.

Related-Bug: #1754219
Change-Id: Icd4bad8852ce5ba40fb0e7b0d335191efbe88c67
2019-12-03 14:57:33 +01:00
Michele Baldessari
f9dcbd30cc Make sure DIB_DEBUG_TRACE has a default value
After the introduction of 'Add output for mis-configured element
scripts' we started seeing CI failures in tripleo where
instack-undercloud is being used (rocky/queens):

  /usr/lib/python2.7/site-packages/diskimage_builder/lib/dib-run-parts: line 108: DIB_DEBUG_TRACE: unbound variable
  INFO: 2019-12-02 16:24:33,423 -- ############### End stdout/stderr logging ###############
  ERROR: 2019-12-02 16:24:33,423 -- Hook FAILED.

Let's make sure that by default the env variable is set
to 0.

Change-Id: I38c76c0edee436f1e7dd0c9a868cea1e6ee3271d
Closes-Bug: #1854904
2019-12-03 08:26:12 +01:00
Colleen Murphy
ad67aa213b Allow zypper repos to be overrideable
Without this change, operating system elements that use the
zypper-minimal element always must use download.opensuse.org as their
repository source. This change makes ZYPPER_REPOS overrideable, which
allows the user to create custom operating system elements that can use
private repositories as their source for base packages. For example,
with only this change, it is possible to create a sles-minimal element
that generates a SLE 15 SP1 image just by overriding DIB_ZYPPER_REPOS
and DIB_OPENSUSE_PATTERNS.

Change-Id: I46e40fbe4408d4204056a27b182b21213f1176ff
2019-12-02 16:25:53 -08:00
Colleen Murphy
4fb5a57b8a Fix login.defs config for tumbleweed
On openSUSE Tumbleweed, the login.defs config file was moved under
/usr[1]. This change allows the login.defs config change to work for
both old and new locations.

[1] https://build.opensuse.org/request/show/736424

Change-Id: Ia5eff5e7b0709836278361b1b8daa788619eff75
2019-12-02 16:21:45 -08:00
Hervé Rousseau
b91e212434 Break retry loop on success in dhcp-all-interfaces
If rdisc6 is available, a node using this element will loops until
DIB_DHCP_TIMEOUT is reached because of a missing 'break' when rdisc6
return code is 0.
This will mark the dhcp-interface@.service unit as failed (because it
has the same timeout) and not bring any network interface online.

Change-Id: I034dcda94d765f236950ebcbee36789f5bdc515f
Closes-Bug: #1854717
Signed-off-by: Hervé Rousseau <hroussea@cern.ch>
2019-12-02 15:50:20 +01:00
Zuul
a231bc6b9f Merge "Add IPv6 support in dhcp-all-interfaces" 2019-11-26 23:54:54 +00:00
Zuul
20e468c604 Merge "Stop installing pydistutils.cfg" 2019-11-26 02:12:41 +00:00
Monty Taylor
75341292f9 Stop installing pydistutils.cfg
Support for easy_install codepaths is increasingly broken, and now
putting allow-hosts in this file breaks most recent pip. Just stop
installing the file - people should be using pip anyway.

Change-Id: I0a6b2432f81d80fbcbb336403fe555003880fa9f
2019-11-25 11:46:42 +11:00
Zuul
fdb6225acc Merge "Add output for mis-configured element scripts" 2019-11-22 06:57:40 +00:00
Zuul
9dd161471a Merge "Ensure nouveau is blacklisted in initramfs too" 2019-11-22 06:57:39 +00:00
Zuul
f268c72ed9 Merge "Adds support for GPG keyring" 2019-11-22 06:57:38 +00:00
Zuul
05a6f898fd Merge "Only wait for checksum processes" 2019-11-22 06:57:37 +00:00
Zuul
6143d40def Merge "Introduce manual setting of DIB_INIT_SYSTEM" 2019-11-22 06:43:25 +00:00
Ian Wienand
bee30b43d2 Only wait for checksum processes
When running under nodepool in a foreground, non-daemonized situation
without a tty (i.e. within a container) we're seeing this "wait" hang
indefinitely.

It is probably related to "outfilter.py" and output file descriptors,
although TBH we haven't completely root-caused it.  I won't claim this
is a great solution, but it should hopefully let the dib process
finish and just die, where outfilter will disappear.

Change-Id: If78da54df3d4c240fee16aee4413ec554b37c1d6
2019-11-21 14:04:09 +11:00
Andreas Florath
3636b40f74 Introduce manual setting of DIB_INIT_SYSTEM
The current implementation evauates the dib-init-system
script too early.  Also it looks that there is no simple
way of getting the info about the init system automatically:
another element can install (later on) a different
init system.  Therefore the only reliable way of setting
this is manual.

Change-Id: I6e9ffa1bdb3154f488f4fd335b197699b86aacd4
Signed-off-by: Andreas Florath <andreas@florath.net>
2019-11-21 12:38:15 +11:00
Michael Johnson
824042d608 Add output for mis-configured element scripts
I commonly get asked for help when people are attempting to create
local image elements and they cannot get them to work.
diskimage-builder silently ignores element scripts that it doesn't
find to it's liking, such as non-executable or files with extensions
(.sh is a common mistake).
This patch extends the '-x' tracing flag down to dib-run-parts and
will cause it to print out helpful messages when these files would
otherwise be silently ignored.

Examples:
Ignoring non-executable files: 10-do-not-run-me
Ignoring non-conforming filenames: 10-I-can-run.sh

I am not enabling these by default as they can create extra noise
and require additional filesystem IO to produce.

Change-Id: Ic804efca3015c199440b4b10da951d71a815c64f
2019-11-20 15:05:30 -08:00
David Hill
f7ee1cd733 Add grub-efi-x86_64 pkg-map
Add grub-efi-x86_64 pkg-map

Change-Id: I64fab237c3fcd5e4fe1e74eb17ddd2a9aa8110f5
Closes-bug: #1852762
2019-11-16 21:15:28 -05:00
Harald Jensås
f94508d537 Add IPv6 support in dhcp-all-interfaces
When the rdisc6 utility is available probe for router
advertisement. configure eni and rhel-netscripts interfaces
to do IPv6 address configuration according to the flags
in the RA recived from the router.

The systemd service file timeout is DIB_DHCP_TIMEOUT * 2,
so that DHCPv4 can timout, and dhcpv6 run before the service
times out.

Retries are commented in dhclient.conf, without it we end up
trying DIB_DHCP_TIMEOUT * 60 before the client move on to
IPv6.

WHEN:
  Stateful address conf.    :          No
  Stateful other conf.      :          No
THEN:
  Do not run dhclient at all, autoconfiguration via
  SLAAC only.

WHEN:
  Stateful address conf.    :          No
  Stateful other conf.      :          Yes
THEN:
  Run "dhclient -6 -S", The ``-S`` option makes the
  dhcp client not request an address, only other
  options such as DNS servers and NTP servers from
  DHCPv6 server.

WHEN:
  Stateful address conf.    :          Yes
  Stateful other conf.      :          Yes
THEN:
  The dhcp client should request an address _and_ other
  options such as DNS servers and NTP servers from
  DHCPv6 server.

NOTE: No IPv6 support added for suse-netscripts

Closes-Bug: 1754219
Change-Id: Icdc79875c33f894ab7eaec8afdfb33a731efff99
2019-11-13 09:31:01 +01:00
Felipe Alencastro
48ff601098 Adds support for GPG keyring
Currently DIB_ADD_APT_KEYS only supports GPG armor keys, while
default Debuntu apt gpg keys are in keyring format.

Change-Id: I361c375e25b03a08b19052b10c6733939c8df921
2019-11-07 17:32:05 -03:00
Ian Wienand
b52b560fb0 Revert "Drop vhdutil dependency, use qemu-img"
This reverts commit a3e9e7f89e.

We still have some issues with vhd creation on RAX

In short, it appears that images fail to resize unless they have a
specific "creator" field.  Revert this while we consider the options.

[1] https://bugs.launchpad.net/nova/+bug/862653

Change-Id: I2b6a3bfbfe28432fbb6a2ce4a0211939d224b8d5
2019-10-30 09:28:58 +11:00
nishagbkar
9e149ce8bb Deprecates the existing "ironic-agent" element in DIB
The "ironic-agent" is copied to ironic-python-agent-builder and
hence it is deprecated from DIB.

Remove from functional testing

Change-Id: Ibc4f75b9d7e2a31994fc86d05bd57975f00fb74f
Task: 36198
Story: 2005114
2019-10-29 10:00:47 +11:00
Zuul
392ebeec68 Merge "pip-and-virtualenv: include python3-venv for Debuntu" 2019-10-28 00:01:51 +00:00
Zuul
24e204eb5c Merge "Drop vhdutil dependency, use qemu-img" 2019-10-25 06:51:59 +00:00
Ian Wienand
f2e0b01336 pip-and-virtualenv: include python3-venv for Debuntu
This package is not installed by default on Debuntu, but is on RH
platforms.  This is causing a build breakage as DIB_PYTHON_VIRTUALENV
tries to use this (I3414fb9e503f94ff744b560eff9ec0f4afdbb50e).

Add the package.

Change-Id: I9a551c57dd128bbb4b095c847f634c777b2cb553
2019-10-25 16:26:33 +11:00
Zuul
220c342e76 Merge "Add security suite name override in debian-minimal" 2019-10-25 05:16:38 +00:00
Zuul
c97cb559d3 Merge "Ensure machine-id is not included in images" 2019-10-25 04:28:28 +00:00
Zuul
fccb6ce32b Merge "Fix syntax error in selinux-fixfiles-restore" 2019-10-24 22:30:27 +00:00
Oliver Walsh
ceaf79d191 Ensure nouveau is blacklisted in initramfs too
To ensure dracut does not load nouveau we need to explicitly disable it via
omit_drivers.
This change adds a method to drop in arbitary dracut conf files to an element
which are picked up by dracut-regenerate and included in the chroot where we
run dracut.

The disable-nouveau element just adds a conf file with
`omit_drivers += " nouveau"`
The default dracut conf files in /usr/lib include a similar file to omit the
nvidia kernel modules.

Change-Id: I6375e4843fd08d1410141fbbd8658042dcd5ad05
Closes-bug: 1842664
2019-10-23 10:16:00 +11:00
Zuul
5bc5f8aff3 Merge "bootloader: make serial console configurable" 2019-10-22 03:19:18 +00:00
Oliver Walsh
ef794db4d1 Fix syntax error in selinux-fixfiles-restore
Seeing this at the end of the tripleo overcloud full build:
99-selinux-fixfiles-restore: line 69: [: too many arguments

Change-Id: I8fb10f3d3d38723b41190ae1898757e6df073945
2019-10-18 12:30:21 +01:00