Commit Graph

335 Commits

Author SHA1 Message Date
Ian Wienand
36d642a6a0 Native zuulv3 tests
This finalises the ports of the legacy jobs to zuul native jobs.

The dib-setup-gate-mirrors role preconfigures the repo templates,
etc. for the openstack-ci-mirrors element.

The dib-functests role runs the tests as specified by dib_functests,
and can run under python2 or 3.

Change-Id: Ied67a31f0d31503d13eccad8662c29740c93f33e
2018-10-29 12:46:15 +11:00
Zuul
668978b0c2 Merge "Remove redundant sources change/update" 2018-10-23 09:16:35 +00:00
Zuul
ec025ff03d Merge "Fix DIB_DISTRIBUTION_MIRROR_UBUNTU_IGNORE regex typo" 2018-10-19 04:01:21 +00:00
Zuul
07d263de0b Merge "Allow debootstrap to cleanup without a kernel" 2018-10-19 03:23:09 +00:00
Zuul
22cd19fea7 Merge "ubuntu-common: Update default DIB_RELEASE to bionic" 2018-10-19 03:23:08 +00:00
Zuul
39c6d7c747 Merge "Move common ubuntu environment setting to ubuntu-common element" 2018-10-19 02:50:36 +00:00
Zuul
46ec4992ff Merge "Add support for Fedora 28, remove EOL Fedora 26" 2018-10-18 23:44:51 +00:00
Ian Wienand
86080a7db0 Fix DIB_DISTRIBUTION_MIRROR_UBUNTU_IGNORE regex typo
This should not be quoted.  Introduced in
I6ffbde07fa0e103641ee5c5f9d9e854e5b2168dc

Change-Id: Iac54c86ebdb90351a2dd663b876ab4675ee0435a
2018-10-18 21:49:44 +11:00
Paul Belanger
7173658df2 Add support for Fedora 28, remove EOL Fedora 26
This updates diskimage-builder to support current Fedora releases (27
and 28) and removes support for Fedora 26 which is EOL as of June
2018.

Change-Id: I602b22ed4d5397b39dc1eef67964f6fbdcd93060
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-10-18 19:43:12 +11:00
Zuul
e09faf9a8d Merge "simplify overlay logic for Gentoo" 2018-10-18 05:51:05 +00:00
Zuul
b3d405f624 Merge "simplify python3.6 selection on gentoo" 2018-10-18 05:51:05 +00:00
Zuul
763493e1f1 Merge "enable caching for gentoo builds" 2018-10-18 05:51:03 +00:00
Zuul
2d327c2506 Merge "Set EPEL mirror during openstack-ci-mirrors" 2018-10-18 04:40:33 +00:00
Zuul
fe15b197fb Merge "ubuntu: Add options to ignore mirror components and use insecure repos" 2018-10-18 04:34:00 +00:00
Zuul
56634e0938 Merge "Turn down pkg-map and hook copy tracing output" 2018-10-18 03:56:12 +00:00
Zuul
d3d17b2e43 Merge "Fix DIB ubuntu-minimal running on bionic (18.04)" 2018-10-18 03:08:37 +00:00
Ian Wienand
8b0111477e Set EPEL mirror during openstack-ci-mirrors
We want to set DIB_EPEL_MIRROR for the epel element, which means we
need access to the CI mirror info script in the chroot.  Copy it into
the temp directory with extra-data.d and update the environment file
to find it.

Change-Id: Ia12f0cbdeb194eef3155497ceb5ffc4a452aad76
2018-10-18 13:53:53 +11:00
Ian Wienand
c47ee6e121 ubuntu: Add options to ignore mirror components and use insecure repos
When using the upstream cloud images with the "ubuntu" element, they
have universe and multiverse enabled which we don't mirror.

To use the infra mirrors as a DIB_DISTRIBUTION_MIRROR with this
element, we need to be able to skip redirecting to universe and
multiverse, and additionally enable insecure repos (as we don't gpg
sign our mirrors).

Add and document two new variables with the ubuntu element to do this.
This is then setup by the openstack-ci-mirrors element so that we use
local mirrors duing dib functional testing for the "ubuntu" element.

Change-Id: I6ffbde07fa0e103641ee5c5f9d9e854e5b2168dc
2018-10-18 13:53:48 +11:00
Matthew Thode
3ad8235180 simplify overlay logic for Gentoo
openssl/cryptography versions are updated/stable
musl profiles need newer versions of open-iscsi masked as upstream
doesn't want to work with multiple libcs

Change-Id: If5baf339516390ae332015928557c6bb734486c2
2018-10-18 00:21:31 +00:00
Matthew Thode
1e257f59cc simplify python3.6 selection on gentoo
python 3.6 is now stable in gentoo

Change-Id: Ide96c039b03817c216ab58930ff5c897556e6c8a
2018-10-18 00:21:16 +00:00
Ian Wienand
6c077d7c2a Turn down pkg-map and hook copy tracing output
This is a lot of very low value noise in the logs as these iterate
through all the elements (often doing nothing).  Turn it down and add
an echo so we just see what elements it is working on.

Change-Id: I0687de4722766189db9d4a7bd7d3cfb45d387b62
2018-10-18 11:03:17 +11:00
Matthew Thode
89f95a5351
enable caching for gentoo builds
To facilitate this I've created two new environment variables to set
the environment and default options for package actions.

eval is needed for the export as it preserves quotes.

Change-Id: Ib03651ee8dacd48cd1c135afd57cd31101356056
Signed-off-by: Matthew Thode <mthode@mthode.org>
2018-10-15 23:17:37 -05:00
Ian Wienand
39fb794ebc Allow debootstrap to cleanup without a kernel
Currently there's more-or-less an assumption that a kernel is
installed, so module blacklists are simply echoed into the modprobe
blacklist.  This may not be the case with some ongoing container work.

Although we don't need to blacklist modules for containers, it also
doesn't hurt.  Move the debootstrap element to the new modprobe
element, and allow it to create the blacklist directory.

Change-Id: I0f057caf473951df56a2af9633e3a5b53e0809b1
2018-10-08 16:02:25 +11:00
Pierre Riteau
bacceba41d Fail build due to missing kauditd only when SELinux is enabled
With the check added in commit 7566819139,
diskimage-builder fails to build RPM-based images if kauditd is not
running. However, this is only valid for environments where SELinux is
enabled. If SELinux is disabled (which is identified by an empty _runcon
variable), proceed with running setfiles.

Change-Id: I1b056f20a3a55f7333391207d9e1049d25ece041
Closes-Bug: #1779273
2018-10-05 10:02:14 +01:00
Zuul
3e6d658687 Merge "allow building non-gentoo images on gentoo hosts" 2018-10-03 11:08:11 +00:00
Michael Johnson
330626482a Fix DIB ubuntu-minimal running on bionic (18.04)
Diskimage-builder fails to build ubuntu-minimal images when run on
a Ubuntu bionic-beaver (18.04) instance.
The user gets "Couldn't create tempfiles for splitting up" when
apt-get update is run in the ubuntu-minimal element root.d.

The issue is that the /tmp inside the chroot is not getting the
proper permissions applied from the base-files package. This is likely
because the pip-cache element has already created the directory before
the base-files package is installed.

This patch changes the order of pip-cache to root.d/11-pip-cache so that
it runs after teh base OS root.d elements run.

Change-Id: I6fd1cb2a23422206884165eb502b260f0c1e52f7
2018-10-01 19:59:30 -07:00
Jesse Pretorius
f4c5326b8e ubuntu-common: Update default DIB_RELEASE to bionic
The ubuntu-minimal README states that the latest Ubuntu LTS
is the default, but currently that is not true. This patch
changes the default to the current LTS.

Change-Id: I10f28314d1a5969c20094194637cfe31219d228c
2018-09-20 02:06:04 +01:00
Jesse Pretorius
8c69da65ac Remove redundant sources change/update
The apt sources are set out in root.d/75-ubuntu-minimal-baseinstall
and the cache is updated, cleaned and a dist-upgrade is done there.

As such, this file is unnecessary.

Change-Id: Idab5ede3f235bc204c4bdebf40fbcf4a12e5bc2f
2018-09-20 00:05:07 +00:00
Jesse Pretorius
30493f7655 Move common ubuntu environment setting to ubuntu-common element
The ubuntu, and ubuntu-minimal elements both make use of a common set
of environment settings to determine the distribution name.

The ubuntu-minimal element also does a few extra things which would
appear to apply to both sets and bring in extra architecture support.
As such, these are included in the common element.

This intends to be part of a series of patches which will eventually
create a new element to build a minimal ubuntu-systemd-container
element which can be used for lxc/nspawn containers.

Change-Id: Ia4e620f7d3fa6215484a8d218cea2f28bd1ffaee
2018-09-20 00:04:55 +00:00
Zuul
b86af3dc6a Merge " IPA requires iptables" 2018-09-13 02:05:42 +00:00
Ian Wienand
f6a2452d4c Only append DIB_BOOTLOADER_DEFAULT_CMDLINE to default grub entry
The grub.cfg has two variables [1]

 GRUB_CMDLINE_LINUX : used on all boots
 GRUB_CMDLINE_LINUX_DEFAULT : additionally used on all "normal" boots

The problem with I2298675dda1f699c572b3423e7274bc8bd7c1c9d is that it
appened the values in DIB_BOOTLOADER_DEFAULT_CMDLINE to both of these,
resulting in duplicated arguments.  I don't think we considered that
GRUB_CMDLINE_LINUX_DEFAULT actually already appends to the
GRUB_CMDLINE_LINUX values.

Make DIB_BOOTLOADER_DEFAULT_CMDLINE only append itself to
GRUB_CMDLINE_LINUX_DEFAULT.  That seems to line up sensibly with the
name of the variable.

Documentation is enhanced around this, and a releasenote added.

[1] https://help.ubuntu.com/community/Grub2/Setup

Change-Id: I76b5442a9090c19a6540ed2d4ab324546f241ebf
Closes: #1791736
2018-09-13 09:51:50 +10:00
Zuul
b29aee3383 Merge "Install sudo on Gentoo images by deault" 2018-09-11 00:09:33 +00:00
Zuul
9f93b2ce3d Merge "Fix CentOS image build failure when dib runs on system where audit disabled" 2018-09-06 11:24:59 +00:00
Olivier Bourdon
7566819139 Fix CentOS image build failure when dib runs on system where audit disabled
Without this fix, building a CentOS image on Ubuntu where audit=0 is passed
as a kernel boot parameter will lead to the following error:
disk-image-create centos7 dhcp-all-interfaces cloud-init-nocloud \
    devuser yum epel baremetal
... dib-run-parts Running tmpdir/hooks/cleanup.d/99-selinux-fixfiles-restore
... Error connecting to audit system.

Change-Id: I229d9b72f88bffddca42da57f01c27e902427071
2018-09-04 08:23:02 +02:00
Charalampos Kominos
c85141291e Fix bootloader packages for aarch64
Due to the arm naming convention, building centos images for arm64 and
aarch64 does not yield the same result. In order to locate grub2 on
aarch64 the correct mapping is added.

Change-Id: I1bb227b2523e420e394fec8c52c6c79fcdd31c53
Closes-Bug:#1789414
Signed-off-by: Charalampos Kominos <Charalampos.Kominos@enea.com>
2018-08-31 17:01:47 +02:00
Zuul
b602c05551 Merge "Add netcat to redhat-common map-packages" 2018-08-23 11:26:12 +00:00
Zuul
f33526d20c Merge "better handle existing keywords files/directories" 2018-08-10 06:07:06 +00:00
Carlos Goncalves
138b14b010 Install ca-certificate with redhat-common
Change-Id: I36d44ca8b4f966476657ec2bb1ecc1458bb524fd
2018-08-04 00:30:49 +02:00
Carlos Goncalves
cf553ce0ac Add netcat to redhat-common map-packages
Change-Id: I273038611febe5e30f30dd8d3ba8990dbdc94647
2018-08-03 17:20:19 +02:00
Zuul
53186f23a0 Merge "cache-url requires curl" 2018-07-31 00:20:56 +00:00
Oliver Walsh
a514feba99 Add DIB element to blacklist nouveau
This ensures nouveau is not loaded at boot, which is required when installing
NVIDIA GPU drivers and to avoid issues with PCI passthrough of NVIDIA GPUs.

The option to disable kernel modesets ensures that it can be unloaded again if
it happens to be loaded after boot (e.g manually or implicitly by X).

bp tripleo-vgpu

Change-Id: I60815de86e7b22dfb39555af9d2d53564841e2ab
Related-bug: 1774674
2018-07-25 16:35:58 +01:00
Oliver Walsh
73e27a8504 modprobe DIB_MODPROBE_BLACKLIST should be optional
modprobe element currently fails when DIB_MODPROBE_BLACKLIST is not set.
As there are now two methods to control blacklisting this should be optional.

Change-Id: Ibf3c31a95177ba88c1b93228490c7f36f5b70b57
2018-07-25 16:33:47 +01:00
Zuul
9adf12fe4a Merge "Fix for proper LVM support" 2018-07-24 07:32:11 +00:00
Clint Byrum
d8907e78b5 cache-url requires curl
In some cases cache-url can get pulled in without curl, causing it to
fail.

Co-Authored-By: Adam Harwell <flux.adam@gmail.com>

Change-Id: Ibd66c2ca4f8cc423783555d8a99b1184f43adff2
2018-07-23 09:56:58 +00:00
Zuul
b79952af2a Merge "Add new modprobe element" 2018-07-11 07:21:40 +00:00
Zuul
bbf69a90f3 Merge "Fix /etc/network/interfaces file contents" 2018-07-10 07:46:56 +00:00
Zuul
2343d4b577 Merge "Add keyring if supplied" 2018-07-10 07:46:55 +00:00
Olivier Bourdon
caf565673b Fix for proper LVM support
Without this fix, a LVM based ubuntu-minimal image will fail
booting due to the fact that the boot process will not be able
to retrieve the root filesystem using LABEL=(cloud)img-rootfs

Change-Id: If4ecf65868563f7b799160a58af6312bedf956bf
2018-07-09 14:15:57 +00:00
Hironori Shiina
7e4e6cfff4 Add expected semicolons for dhclient.conf
This patch adds an expected semicolon to an end of statement in
dhclient.conf for dhcp-all-interfaces element. Without this fix, an
error occurs when an image is booted with a message,
'semicolon expected.'.

Change-Id: I8311dbc67cc2815223111da01e7a7517c7d6f059
2018-07-06 13:42:25 +09:00
Sam Yaple
c144246cc9
Add keyring if supplied
When building with debootstrap, debootstrap will use the key to check
that everything is properly signed. It will not `apt-key add` the key
into the final environment, however.

Early adding the key after debootstrap before we need to read from the
private repo again prevents unsigned issues. This also maintains the
integrity of the packages in the environment throughout the build.

Change-Id: I5ca75ae4620c9fb26b512cb30f8cd79fa7a0373a
2018-07-02 14:33:35 -04:00