Commit Graph

4353 Commits

Author SHA1 Message Date
Ian Wienand
3bc89edd32 yum-minimal : update mirrors for Centos 8
The repo format has slightly changed for CentOS 8 (s/os/baseos/).

Make the chroot builder look for a more specific repos.d directory
first named for the distro variable, then fall back to to top-level
dir (this avoids having to constantly change fedora).

Update the gate mirror setup and roles for new Centos 8 paths too.

Change-Id: I5b7f0c3624cac1d7aa7ed8bf6286b85d808b9c9a
2019-10-03 00:22:05 +00:00
Ian Wienand
456b6cf394 Remove "failovermethod=priority" for Fedora (dnf)
This is no longer a valid option for dnf, and it puts out a lot of
warnings constantly about the invalid entry [1].  Remove it.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1653831

Change-Id: Iba0585cab3e5e78e9324196f276b2341e7bb9e3c
2019-10-01 06:01:15 +00:00
Ian Wienand
a77a214339 Install Python 3 libselinux packages for Fedora
Install the Python 3 libselinux packages for Fedora platforms.  I
think this is the right choice; Fedora is a Python-3 only distro so we
shouldn't default to installing the python2 libraries.

This has a practical effect if you're using Ansible with
ansible_python_interpreter=/usr/bin/python3 as it needs these
packages.

There is some small chance of breakage if you're using Ansible still
with Python 2, I guess.  In infra I notice we bring this in with
"zuul-worker" project-config element.  On balance, I think that if you
need the Python 2 packages for some reason, it should be a special
install and not part of redhat-common.

Change-Id: Ibcec0b3660d01b861838c2ae87ca43d98953ce32
2019-09-20 17:33:38 +10:00
Logan V
c7e907794c Ensure machine-id is not included in images
Two bugs are addressed.

1) The sysprep element was broken in that it only truncates
   /etc/machine-id, but not /var/lib/dbus/machine-id. systemd will
   not generate a new machine-id if /var/lib/dbus/machine-id is
   present[1], it will simply copy it to /etc/machine-id.

   We observed machine-ids being packaged in /var/lib/dbus/machine-id
   on several distros: Ubuntu Bionic, Fedora 29, Debian Stretch.

   CentOS 7 and Ubuntu Xenial do not contain packaged machine-id as
   far as I can tell.

   All test builds were performed using -minimal elements.

2) A second bug existed where debian-minimal did not run the sysprep
   element at all, so a stretch image I tested contained a populated
   /etc/machine-id AND a populated /var/lib/dbus/machine-id.

[1] https://www.freedesktop.org/software/systemd/man/machine-id.html#Initialization

Change-Id: Ibb28b6e90d966a845de38a2cd5a1e8babd2604bc
2019-09-20 03:17:50 +00:00
Jens Harbott
fb8258912b Update gentoo test to current system profile
Gentoo is at 17.1 now, most code was updated in [0], update the test job
accordingly.

[0] I0f2e7976845b1d3c55ffe8869eec0bc04a191252

Change-Id: I30e018e67f3637ce36b3ef6e20380453fe6b2bb9
2019-09-17 12:43:28 +00:00
Zuul
b907ecf15c Merge "Revert "Fixed use of flake8"" 2019-09-16 05:37:28 +00:00
Ian Wienand
8c85a7ed5d Revert "Fixed use of flake8"
This reverts commit 74317a3445.

What I missed was that "dib-lint" is distributed and people use this
to lint their out-of-tree elements (e.g. project-config).  Because
dib-lint runs flake8 that really makes it a valid runtime dependency.

However, I don't think we need to pin to these particular versions of
flake8 in requirements (which I think was causing the original issues,
as it's fairly specific and easily conflicts).  It seems to make more
sense to use the current uncapped "hacking" versions; so move them
from test-requirements.txt.  Add a note so we don't forget again.

Change-Id: Ic2eee8f5b64c7020e98e0b1ef43a40f24411891a
2019-09-16 10:17:49 +10:00
Andreas Jaeger
f0dd0bd63a Only install doc requirements if needed
We only need doc requirements for the doc jobs, so only install them
there.

Also, update docs job to not publish dirtrees.

Change-Id: I928f4e24970a680bf340ce15a08a4e72432fb0e4
2019-09-13 10:09:59 +02:00
Zuul
b29f5db9c7 Merge "Move doc related modules to doc/requirements.txt" 2019-09-13 07:52:04 +00:00
Bob Fournier
8cab82bf9d Use x86 architeture specific grub2 packages for RHEL
Similar to https://review.opendev.org/#/c/663693/, the x64 packages
should be used for x86 architectures.

Change-Id: I5e8a4d58e96d65eb60fc539b8a1d56853b12faac
Closes-Bug: 1843820
2019-09-12 15:06:17 -04:00
jacky06
9e3400fa9c Move doc related modules to doc/requirements.txt
Change-Id: I5106dc365856932f00a2fbdf4e7a4d58e48cca8a
2019-09-09 14:13:36 +08:00
Zuul
33a089b099 Merge "Add fedora-30 testing to gate" 2019-09-09 00:32:40 +00:00
Zuul
b94588c862 Merge "Do not delete cracklib from /usr/share" 2019-09-06 10:25:33 +00:00
Zuul
5148c18928 Merge "Fixed use of flake8" 2019-09-06 10:01:12 +00:00
Zuul
48edd472e7 Merge "Allow configurable gzip binary name" 2019-09-06 09:47:06 +00:00
Zuul
11a5a86758 Merge "Uninstall linux-firmware and linux-firmware-whence" 2019-09-06 08:43:47 +00:00
Ian Wienand
68f0ace08a Add fedora-30 testing to gate
Forgotten in If60eb0b87e45efc0e71db2ddcd814223539f07b7

Change-Id: Ia2887c84510f09e37d0cbc27d13f770655bc95fc
2019-09-06 17:31:46 +10:00
Carlos Goncalves
f909000e5a Uninstall linux-firmware and linux-firmware-whence
linux-firmware and linux-firmware-whence (meta package for mostly iwl
firmwares) packages account for approx. 289 M install size on a F30
system, and linux-firmware for approx. 176 M on CentOS 7. Users needing
these firmwares are eventually baremetal users and are not looking for a
very minimal operating system base install like virtual image users are.
Thus, a non-minimal OS element is better suited for them. Alternatively,
it could be later considered a dedicated firmware element.

This is inline with I8ce65e1d357d15e8ed8995ad1dcaea02bbd1986f.

Change-Id: If104fc3c1e9349b8d501a2351fff1ab4c0dbc6a4
2019-09-06 15:32:51 +10:00
Sorin Sbarnea
74317a3445 Fixed use of flake8
Flake8 was wrongly listed as a runtime dependency. That mistake messed
system flake8 version. Flake8 is a test requirement not a runtime one.

Also flake8 version is already determined by hacking version, so we
update it to recent version.

Change-Id: Iccd9dc6f2508127f17712d16dfce6538ff83807e
2019-09-04 13:40:35 +00:00
Dirk Mueller
d40c87876a Rename openSUSE 15.1 testing to 15
This is consistent with the previous simplication of
build targets in the opendev environment to refer to
"opensuse15" being the alias of "latest stable openSUSE Leap 15.x".

Change-Id: I904a3ca0d6dbddd2bb1a673836ab6a0ad249526d
2019-08-30 22:44:40 +02:00
Logan V
d9e85efd7c Allow configurable gzip binary name
Add a new environment variable $DIB_GZIP_BIN allowing builders to
specify a different gzip (such as pigz) to be used when compressing
tgz images.

Change-Id: Ifb617568140a149e2fda241e07ff8a59429e6697
2019-08-30 17:46:20 +02:00
Logan V
b98d482d5f Do not delete cracklib from /usr/share
We have an application breaking because /usr/share/cracklib is being
deleted from the image. The application installs its dependencies,
including cracklib, but since yum shows that cracklib is already
installed, it does not reinstall it.

Change-Id: Id6fccf76c706dbc6c2124abcfd12c1f10cef5e09
2019-08-30 15:11:26 +02:00
Zuul
9ef7f73b6a Merge "Allow extra repositories to be added to images" 2019-08-30 07:02:46 +00:00
Zuul
3e0d61ac9a Merge "Fedora 30 functional and boot tests" 2019-08-30 06:27:36 +00:00
Zuul
f4698b5864 Merge "rpm-distro: ensure we selinux relabel underlying directories" 2019-08-30 04:31:00 +00:00
Zuul
064f93acd8 Merge "yum-minimal: install fedora-release-cloud" 2019-08-30 04:24:00 +00:00
Ian Wienand
a5a6482ac1 Fedora 30 functional and boot tests
Update testing for Fedora 30

Change-Id: If60eb0b87e45efc0e71db2ddcd814223539f07b7
2019-08-28 11:21:46 +10:00
Zuul
76b09c845c Merge "zypper-minimal: Don't get confused by etc/resolv.conf symlink" 2019-08-21 20:43:08 +00:00
Dirk Mueller
6c23b97f4a zypper-minimal: Don't get confused by etc/resolv.conf symlink
Newer openSUSE distributions install an absolute link to /run/netconfig
as /etc/resolv.conf in $TARGET_ROOT. as that points outside
TARGET_ROOT, we unintentionally wipe the system resolv.conf here
and break our ability to finish building the image.

Change-Id: I9d5aaa9fad2f81dcabfe19e2f1e6b6e50af597d7
2019-08-21 19:57:25 +02:00
Zuul
3b2b87dde1 Merge "block-device-efi : expand disk size calculation" 2019-08-21 02:30:32 +00:00
Zuul
8690579efc Merge "dracut-regenerate: catch failures and exit code" 2019-08-21 01:48:10 +00:00
Zuul
3de4b71b9e Merge "update gentoo systemd profile to 17.1 from 17.0" 2019-08-20 21:21:24 +00:00
Ian Wienand
aee4fc0d35 simple-init: add configurable RA timeout with network-manager
This is a follow-on to I475a253091cbaf63687b91c748c31a6753bb0f57 as we
are still seeing issues on some clouds with unconfigured networking.

We increase the timeout, but also make it configurable so we can
fiddle it without a dib release in the gate.

To follow-on from the experimentation done by clarkb, I can confirm by
emperical testing on a Centos 7 image (from today, today being this
change's date) that setting

 net.ipv6.conf.all.autoconf=0

by itself is "fatal" and the interfaces do not come up; i.e. nm does
not by default seem to re-enable ipv6 for the interface.  However,
explicitly adding:

 IPV6INIT=yes
 IPV6_AUTOCONF=yes

to the interface file *does* seem to make it work, even if
"all.autoconf=0" is set (then again, there's also bugs about the
effect of this [1]).  However, no extant distribution (I can currently
find) does anything like this by default.

If this continues, this may be an option.  Another might be to avoid
the use of the nm-settings-ifcfg-rh profiles and move directly to nm
ini files with glean.

[1] https://bugzilla.kernel.org/show_bug.cgi?id=11655

Change-Id: I869ebffc8cde3bbff573f6583fd9dd02a5598590
2019-08-20 17:07:17 +10:00
Matthew Thode
9755c4f9a2
update gentoo systemd profile to 17.1 from 17.0
Upstream is now publishing 17.1 profile systemd stages

Also updates the docs that were forgotten in the last patch

Change-Id: I0f2e7976845b1d3c55ffe8869eec0bc04a191252
2019-08-19 15:13:09 -05:00
Ian Wienand
f23318d579 rpm-distro: ensure we selinux relabel underlying directories
As described inline, we need to ensure the underlying directories in
the image are correctly labeled, or we get all manner of services
failing during boot with selinux in enforcing mode.  Although the
problem is generic, this first shows up in Fedora 30 as systemd has
become more strict about namespace failures (I think) [1].

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1663040#c22

Change-Id: I52c1cc719884879169b606b00651aa26f5b783f1
2019-08-16 15:44:55 +10:00
Carlos Goncalves
9eb175e440 Allow extra repositories to be added to images
This patch adds option DIB_YUM_MINIMAL_EXTRA_REPOS to yum-minimal to
allow DIB users to include extra repositories to their final image.

Change-Id: I89549f4b0f4c9470143b5064817acab5043e31c5
2019-08-15 16:20:21 +02:00
Ian Wienand
efa3f3675a yum-minimal: install fedora-release-cloud
Something (possibly [1], but that change is at best cryptic) has
changed such that we don't get correct /etc/os-release files
installed.  This flows on to grub half-installing itself, enough to
not fail the build but not enough to make something bootable.

Installing the -cloud release package gets it back, and seems like a
sane choice for dib.

[1] 617b1bed34

Change-Id: Iff0413887fad798273b2bfcb140cc07f36d54a04
2019-08-15 15:56:13 +10:00
Ian Wienand
5492843aa8 block-device-efi : expand disk size calculation
As noted in the change, 7fd52ba841
increased the size of the EFI partition considerably.  This has meant
that our padding upwards of the disk size is insufficient and EFI
builds (arm64 in particular) is failing due to out-of-disk errors
during final image operations like installing kernels.

Similar to the discussion we had in
I65fa13a088eecdfe61636678578577ea2cfb3c0c, this feels a bit ugly
because we're mixing logic here with sizes specified in block-device
config files.  But it boils down to the same problem; we are
calculating the disk size here and passing it to the block-layer, so
unless we want to make large changes to the status quo about where
these sizes are calculated, small adjustments here are the most KISS
solution.

Thus we check if we have selected the EFI bootloader element, and thus
assume there will be a large system EFI partition and expand the disk
size accordingly.

Change-Id: Ifa05366c2f2b95259f3312e4dde8c85347075ba1
2019-08-14 15:49:38 +10:00
Ian Wienand
c596fb7dbd Don't show all elements found
This debug statement lists every element found and its dependencies on
every build; it's just noise unless you're debugging the element
dependency solver itself.  Remove from output.

Change-Id: I9281b953d958a3fd5e20edbc560a341a2fcc3deb
2019-08-14 13:07:16 +10:00
Zuul
0c6a4dbd92 Merge "Fixes packages for arm64 bootloader" 2019-08-13 09:21:37 +00:00
Zuul
24e620ae6d Merge "Fix the pypi element for multiple mirror URLs" 2019-08-13 07:04:20 +00:00
Zuul
1af5e6010d Merge "Create /etc/machine-id for RHEL images" 2019-08-13 07:00:31 +00:00
Ian Wienand
bac8b4246e dracut-regenerate: catch failures and exit code
This seems to miss the exit code of the dracut process, which actualy
caused some issues in I8511669e188717494daf2bc1384a6dd346f942a4 where
it would have been much clearer to stop after the initramfs generation
failed.

Add some debug messages, and catch any errors from the final call.

Change-Id: I6f89441ec4709f5199535e15a7cc53a3a8af273d
2019-08-13 15:51:05 +10:00
Zuul
4131942356 Merge "Fixes DIB_DISTRIBUTION_MIRROR_UBUNTU_IGNORE matching when empty" 2019-08-12 13:22:44 +00:00
liyingjun
bf0d7ede55 Fixes packages for arm64 bootloader
Should install "grub2-efi-aa64 grub2-efi-aa64-modules" instead of
"grub2-efi grub2-efi-modules" for arm64

Change-Id: Iee3191b0944b3b862890d166a9d36bd592fe8f7e
Closes-bug: #1839816
2019-08-12 17:18:36 +08:00
Manuel Torrinha
a38ac762f1 Fixes DIB_DISTRIBUTION_MIRROR_UBUNTU_IGNORE matching when empty
- DIB_DISTRIBUTION_MIRROR_UBUNTU_IGNORE matches when it is empty or not
 set and DIB_DISTRIBUTION_MIRROR is being used. Checking for it being 
 set and not empty solves this.
 - Normalizing bash conditionals for readability

Closes-Bug: #1808359
Change-Id: I87853fcda4c8b29a3f1720a2778debeb3acc3a53
Signed-off-by: Manuel Torrinha <manuel.torrinha@tecnico.ulisboa.pt>
2019-08-09 10:26:48 +00:00
Michael Johnson
bac0fa3eb2 Fix the pypi element for multiple mirror URLs
The 'pypi' mirror element is generating invalid pip.conf files
when more than one "DIB_PYPI_MIRROR_URL" is specified.
This patch fixes the pip.conf file rendering to use a proper form
when there are multiple "extra_index_url" provided.

Closes-Bug: #1839558
Change-Id: Ibda0e7390955683560e09b486f636775643ff57c
2019-08-08 22:52:43 +00:00
Andreas Jaeger
9a145cf0d0 Stop regex warning
python 3.6 warns about regexes like:
DeprecationWarning: invalid escape sequence \+

I noticed that debugging a trove job and it really led me in the wrong
way. Fix this with making it a raw string.

Change-Id: I58ee1a49d62316c6c3f0588832c97f659f7e460b
2019-08-08 15:31:52 +02:00
Ian Wienand
1f2a874e8e Create /etc/machine-id for RHEL images
Per the inline comment, a machine-id is required for kernels to
install correctly (this may well be a bug, but the linked issue
remained inconclusive).

Add a call to make the machine-id before install packages.

Change-Id: If75d04376e62bfdfe14ee3ca4d0bd5c8b383c1b0
Redhat-Bug: https://bugzilla.redhat.com/show_bug.cgi?id=1737355
2019-08-07 18:18:34 +10:00
Zuul
37909a0e81 Merge "Cleanup: remove useless statement" 2019-08-05 02:46:12 +00:00