Commit Graph

510 Commits

Author SHA1 Message Date
Zuul
c97cb559d3 Merge "Ensure machine-id is not included in images" 2019-10-25 04:28:28 +00:00
Zuul
fccb6ce32b Merge "Fix syntax error in selinux-fixfiles-restore" 2019-10-24 22:30:27 +00:00
Zuul
5bc5f8aff3 Merge "bootloader: make serial console configurable" 2019-10-22 03:19:18 +00:00
Oliver Walsh
ef794db4d1 Fix syntax error in selinux-fixfiles-restore
Seeing this at the end of the tripleo overcloud full build:
99-selinux-fixfiles-restore: line 69: [: too many arguments

Change-Id: I8fb10f3d3d38723b41190ae1898757e6df073945
2019-10-18 12:30:21 +01:00
Zuul
1e08be004b Merge "centos7; use numeric DIB_RELEASE" 2019-10-14 21:31:49 +00:00
Ian Wienand
9f688f53da centos7; use numeric DIB_RELEASE
With the introduction of centos 8 we have constructs like

 if [[ $DISTRO =~ (centos|fedora) && $DIB_RELEASE -ge 8 ]]

This is intended to match the "centos7" element (from the =~) but it
was missed that this is setting the DIB_RELEASE to "GenericCloud".

I think it makes more sense for this to be a numeric release, and
makes constructs like above work.  There really isn't any other type
of image to choose here; thus we move it into a new, centos7
specific variable.

Note that when the centos 8 images are available, we want to move to a
generic "centos" element that will handle both 7 and 8 together (same
as rhel) based on DIB_RELEASE and deprecate centos7; this works with
that environment too.

Change-Id: I2e6b7848070d6452c0563e2a122447627c6e6bf7
2019-10-14 14:34:36 +11:00
Clark Boylan
d1e49214ce Remove RA solicit delay
It turns out that this breaks ipv6 config with NM. Instead what we want
is for glean to not up interfaces on boot (see the depends-on).

Change-Id: I6c5bc76c433e29f02d3266ab8f669015125ec954
Depends-On: https://review.opendev.org/#/c/688031
2019-10-11 15:29:32 -07:00
Ian Wienand
5b5385cf84 CentOS 8 minimal testing and support
This adds CentOS 8 into functional and boot tests.

This completes centos-minimal support, documentation is updated and a
release note is added.

Change-Id: I435c2967b4f49faeb6d6edf189907b9f96e80357
2019-10-08 00:17:14 +02:00
Ian Wienand
85a4ec2b2d Add NetworkManager and dhcp-client for CentOS 8
As described inline, NetworkManager and dhcp-client make up the basic
networking for centos 8 installs; bring them into the base image.

Although in infra we then use simple-init, some other users find this
helpful.

Change-Id: Ib9f32e73bf9109cc1b659fe1deceb1a15301ffeb
2019-10-07 10:47:09 +00:00
Ian Wienand
314b11b6dd Fix networking for CentOS 8
By default network-scripts package isn't installed, so the directories
for these files don't exist either.  Skip by default for Centos 8.

Change-Id: I194ec3735e17f27e586386541dc51f775b01e510
2019-10-07 10:47:09 +00:00
Ian Wienand
643415f366 simple-init: Use wrappers to call pip for glean install
Use the wrapper calls from Ia267a60eecfa8f4071dd477d86daebe07e9a7e38
to install glean.

Using this wrapper means we cover all cases without more and more
branches; it should work for python2, python3 and also the special
case of RHEL/CentOS where dib-python points to the special
/usr/libexec/platform-python (which is python3.6 with inbuilt pip)

Change-Id: If624e8bb66ce0761fc0d5f34c2bed8b93a7daeee
2019-10-07 10:47:09 +00:00
Ian Wienand
cbe1a0fc6b simple-init: default to NetworkManager for CentOS and Fedora
NetworkManager with simple-init has proven to be stable in OpenStack
infra, switch to it by default for CentOS and Fedora.  For CentOS 8
and Fedora, add a check to make it the only option.  Thus only CenOS 7
remains optionally using the legacy scripts; this is likely not used
anywhere (infra is really the primary user, where NetworkManager is
already used); we can likely remove this variable (and hence path) in
a future cleanup.

In the setup, remove rhel7 element which was never really tested.
Reorganise the fallthrough to call out the default paths as doing
nothing.

Change-Id: Ic996956da4b85f7d95179b8df9881d5f52c091af
2019-10-07 10:46:57 +00:00
Zuul
027092d407 Merge "pip-and-virtualenv : deprecate source for CentOS 8, new variables" 2019-10-05 05:53:36 +00:00
Zuul
dd1fa4f8c2 Merge "yum-minimal: Don't install yum, install libcurl" 2019-10-05 02:59:20 +00:00
Zuul
132bfb086e Merge "Use $YUM instead of direct calls in more places" 2019-10-05 02:59:19 +00:00
Zuul
a33c643e43 Merge "Add environment switch for centos8 to use dnf" 2019-10-05 00:18:16 +00:00
Zuul
cec04b2d55 Merge "Update redhat-common pkg-map for centos 8" 2019-10-04 10:16:39 +00:00
Zuul
71f4e370f4 Merge "Add security mirror override for debian-minimal" 2019-10-04 08:46:22 +00:00
Andrei Nistor
0e5e358063 bootloader: make serial console configurable
Currently, the serial console is hardcoded to ttyS0 in the bootloader
element.  This is a challenge for users that want to build images for
some baremetal servers. Supermicro servers, for example, use ttyS1 for
the serial over lan interface.

This patch adds a new environment variable DIB_BOOTLOADER_SERIAL_CONSOLE
that can be set to override the default.

Change-Id: Ie8173be8690ac0b7164ce9e5b66d3c1c18f844d6
2019-10-03 21:49:53 +00:00
Jeremy Stanley
9b201b58b9 Add security mirror override for debian-minimal
Add option to set the security mirror URL independently in the
debian-minimal element, since this can not be overriden by the
standard DIB_DISTRIBUTION_MIRROR variable.

Change-Id: I145844a410d06a479e68db1bf6d5d0159389305c
2019-10-03 13:49:47 +10:00
Ian Wienand
18215274d8 pip-and-virtualenv : deprecate source for CentOS 8, new variables
As described inline, deprecate the "source" install for CentOS 8.
Overwriting the packaged tools has long been a pain-point in our
images, and the best outcome is just not to play the game [1].

However, the landscape remains complicated.  For example, RHEL/CentOS
8 introduces the separate "platform-python" binary, which seems like
the right tool to install platform tools like "glean" (simple-init)
with.  However, platform-python doesn't have virtualenv (only the
inbuilt venv).

So that every element doesn't have to hard-code in workarounds for
these various layouts, create two new variables DIB_PYTHON_PIP and
DIB_PYTHON_VIRTUALENV to just "do the right thing".  If you need is
"install a pip package" or "create a virtualenv" this should work on
all the platforms we support.  If you know more specifically what you
want (e.g. must be a python3 virtualenv) then nothing stops elements
calling that directly (e.g. python3 -m virtualenv create); these are
just helper wrappers for base elements that need to be broadly
compatible.

[1] http://lists.openstack.org/pipermail/openstack-infra/2019-September/006483.html

Change-Id: Ia267a60eecfa8f4071dd477d86daebe07e9a7e38
2019-10-03 00:22:18 +00:00
Ian Wienand
5f3b7cd7b7 yum-minimal: Don't install yum, install libcurl
Don't install the "yum" package, which is a backwards compat around
dnf.  With 687003f we should not need the backwards compat links any
more.

Add libcurl to avoid conficts with in the curl "-minimal" packages
that happens on CentOS 8.  But skip it on Fedora, because it seems to
create more problems there (not going to pretend it isn't all a
hack ... but it seems to work).

Change-Id: I1de2703eb5075a0a22837b6898bd8eb960d080dd
2019-10-03 00:22:18 +00:00
Ian Wienand
b57714af75 Use $YUM instead of direct calls in more places
A few places we either assume centos uses "yum" directly, or have
switching based on the distro type.

In both cases, we can use ${YUM} directly to avoid ambiguity

Change-Id: I71095a9bd1862f8956b5982fbbb3e1d213926c14
2019-10-03 00:22:18 +00:00
Ian Wienand
ddb2811255 Add environment switch for centos8 to use dnf
Set YUM to dnf for Centos 8; this matches similar done in
fedora-minimal.

Change-Id: I2b2c41a73e468fe9045ee5b7b812da66f20d8584
2019-10-03 00:22:18 +00:00
Ian Wienand
84cf2e1b82 Update redhat-common pkg-map for centos 8
The libselinux packages etc don't exist for Python 2 on Centos 8 [1].
Ensure the package map installs the python3 versions.

We could probably invert the logic now, and make it so Centos 7 is the
"special" version that overrides things to install python2.  Left
alone for now to avoid changing too much at once.

[1] https://bugs.centos.org/view.php?id=16458

Change-Id: I944cf4f2902c28728aa5bb9e2a00b3eef122d52e
2019-10-03 00:22:18 +00:00
Ian Wienand
1176a45525 Update locales for Centos 8
CentOS 8 has the "new" split-up locales packages.  Fedora 24 is now
long gone, so take out the old branch and apply the lang package
install to Centos 8 as well.

The manual locale cleanup is not necessary on Centos 8; skip it.

Change-Id: Ib65fc15fe471348793fd6efb034517f11abd905e
2019-10-03 00:22:18 +00:00
Ian Wienand
0e230642c5 dib-python : handle centos 8
Match on centos 8 and use the inbuilt system python like on RHEL

Change-Id: I81d94481422b4982777160f736dcf463e2fc45d0
2019-10-03 00:22:18 +00:00
Ian Wienand
3bc89edd32 yum-minimal : update mirrors for Centos 8
The repo format has slightly changed for CentOS 8 (s/os/baseos/).

Make the chroot builder look for a more specific repos.d directory
first named for the distro variable, then fall back to to top-level
dir (this avoids having to constantly change fedora).

Update the gate mirror setup and roles for new Centos 8 paths too.

Change-Id: I5b7f0c3624cac1d7aa7ed8bf6286b85d808b9c9a
2019-10-03 00:22:05 +00:00
Ian Wienand
456b6cf394 Remove "failovermethod=priority" for Fedora (dnf)
This is no longer a valid option for dnf, and it puts out a lot of
warnings constantly about the invalid entry [1].  Remove it.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1653831

Change-Id: Iba0585cab3e5e78e9324196f276b2341e7bb9e3c
2019-10-01 06:01:15 +00:00
Ian Wienand
a77a214339 Install Python 3 libselinux packages for Fedora
Install the Python 3 libselinux packages for Fedora platforms.  I
think this is the right choice; Fedora is a Python-3 only distro so we
shouldn't default to installing the python2 libraries.

This has a practical effect if you're using Ansible with
ansible_python_interpreter=/usr/bin/python3 as it needs these
packages.

There is some small chance of breakage if you're using Ansible still
with Python 2, I guess.  In infra I notice we bring this in with
"zuul-worker" project-config element.  On balance, I think that if you
need the Python 2 packages for some reason, it should be a special
install and not part of redhat-common.

Change-Id: Ibcec0b3660d01b861838c2ae87ca43d98953ce32
2019-09-20 17:33:38 +10:00
Logan V
c7e907794c Ensure machine-id is not included in images
Two bugs are addressed.

1) The sysprep element was broken in that it only truncates
   /etc/machine-id, but not /var/lib/dbus/machine-id. systemd will
   not generate a new machine-id if /var/lib/dbus/machine-id is
   present[1], it will simply copy it to /etc/machine-id.

   We observed machine-ids being packaged in /var/lib/dbus/machine-id
   on several distros: Ubuntu Bionic, Fedora 29, Debian Stretch.

   CentOS 7 and Ubuntu Xenial do not contain packaged machine-id as
   far as I can tell.

   All test builds were performed using -minimal elements.

2) A second bug existed where debian-minimal did not run the sysprep
   element at all, so a stretch image I tested contained a populated
   /etc/machine-id AND a populated /var/lib/dbus/machine-id.

[1] https://www.freedesktop.org/software/systemd/man/machine-id.html#Initialization

Change-Id: Ibb28b6e90d966a845de38a2cd5a1e8babd2604bc
2019-09-20 03:17:50 +00:00
Bob Fournier
8cab82bf9d Use x86 architeture specific grub2 packages for RHEL
Similar to https://review.opendev.org/#/c/663693/, the x64 packages
should be used for x86 architectures.

Change-Id: I5e8a4d58e96d65eb60fc539b8a1d56853b12faac
Closes-Bug: 1843820
2019-09-12 15:06:17 -04:00
Zuul
b94588c862 Merge "Do not delete cracklib from /usr/share" 2019-09-06 10:25:33 +00:00
Zuul
11a5a86758 Merge "Uninstall linux-firmware and linux-firmware-whence" 2019-09-06 08:43:47 +00:00
Carlos Goncalves
f909000e5a Uninstall linux-firmware and linux-firmware-whence
linux-firmware and linux-firmware-whence (meta package for mostly iwl
firmwares) packages account for approx. 289 M install size on a F30
system, and linux-firmware for approx. 176 M on CentOS 7. Users needing
these firmwares are eventually baremetal users and are not looking for a
very minimal operating system base install like virtual image users are.
Thus, a non-minimal OS element is better suited for them. Alternatively,
it could be later considered a dedicated firmware element.

This is inline with I8ce65e1d357d15e8ed8995ad1dcaea02bbd1986f.

Change-Id: If104fc3c1e9349b8d501a2351fff1ab4c0dbc6a4
2019-09-06 15:32:51 +10:00
Dirk Mueller
d40c87876a Rename openSUSE 15.1 testing to 15
This is consistent with the previous simplication of
build targets in the opendev environment to refer to
"opensuse15" being the alias of "latest stable openSUSE Leap 15.x".

Change-Id: I904a3ca0d6dbddd2bb1a673836ab6a0ad249526d
2019-08-30 22:44:40 +02:00
Logan V
b98d482d5f Do not delete cracklib from /usr/share
We have an application breaking because /usr/share/cracklib is being
deleted from the image. The application installs its dependencies,
including cracklib, but since yum shows that cracklib is already
installed, it does not reinstall it.

Change-Id: Id6fccf76c706dbc6c2124abcfd12c1f10cef5e09
2019-08-30 15:11:26 +02:00
Zuul
9ef7f73b6a Merge "Allow extra repositories to be added to images" 2019-08-30 07:02:46 +00:00
Zuul
3e0d61ac9a Merge "Fedora 30 functional and boot tests" 2019-08-30 06:27:36 +00:00
Zuul
f4698b5864 Merge "rpm-distro: ensure we selinux relabel underlying directories" 2019-08-30 04:31:00 +00:00
Zuul
064f93acd8 Merge "yum-minimal: install fedora-release-cloud" 2019-08-30 04:24:00 +00:00
Ian Wienand
a5a6482ac1 Fedora 30 functional and boot tests
Update testing for Fedora 30

Change-Id: If60eb0b87e45efc0e71db2ddcd814223539f07b7
2019-08-28 11:21:46 +10:00
Zuul
76b09c845c Merge "zypper-minimal: Don't get confused by etc/resolv.conf symlink" 2019-08-21 20:43:08 +00:00
Dirk Mueller
6c23b97f4a zypper-minimal: Don't get confused by etc/resolv.conf symlink
Newer openSUSE distributions install an absolute link to /run/netconfig
as /etc/resolv.conf in $TARGET_ROOT. as that points outside
TARGET_ROOT, we unintentionally wipe the system resolv.conf here
and break our ability to finish building the image.

Change-Id: I9d5aaa9fad2f81dcabfe19e2f1e6b6e50af597d7
2019-08-21 19:57:25 +02:00
Zuul
8690579efc Merge "dracut-regenerate: catch failures and exit code" 2019-08-21 01:48:10 +00:00
Zuul
3de4b71b9e Merge "update gentoo systemd profile to 17.1 from 17.0" 2019-08-20 21:21:24 +00:00
Ian Wienand
aee4fc0d35 simple-init: add configurable RA timeout with network-manager
This is a follow-on to I475a253091cbaf63687b91c748c31a6753bb0f57 as we
are still seeing issues on some clouds with unconfigured networking.

We increase the timeout, but also make it configurable so we can
fiddle it without a dib release in the gate.

To follow-on from the experimentation done by clarkb, I can confirm by
emperical testing on a Centos 7 image (from today, today being this
change's date) that setting

 net.ipv6.conf.all.autoconf=0

by itself is "fatal" and the interfaces do not come up; i.e. nm does
not by default seem to re-enable ipv6 for the interface.  However,
explicitly adding:

 IPV6INIT=yes
 IPV6_AUTOCONF=yes

to the interface file *does* seem to make it work, even if
"all.autoconf=0" is set (then again, there's also bugs about the
effect of this [1]).  However, no extant distribution (I can currently
find) does anything like this by default.

If this continues, this may be an option.  Another might be to avoid
the use of the nm-settings-ifcfg-rh profiles and move directly to nm
ini files with glean.

[1] https://bugzilla.kernel.org/show_bug.cgi?id=11655

Change-Id: I869ebffc8cde3bbff573f6583fd9dd02a5598590
2019-08-20 17:07:17 +10:00
Matthew Thode
9755c4f9a2
update gentoo systemd profile to 17.1 from 17.0
Upstream is now publishing 17.1 profile systemd stages

Also updates the docs that were forgotten in the last patch

Change-Id: I0f2e7976845b1d3c55ffe8869eec0bc04a191252
2019-08-19 15:13:09 -05:00
Ian Wienand
f23318d579 rpm-distro: ensure we selinux relabel underlying directories
As described inline, we need to ensure the underlying directories in
the image are correctly labeled, or we get all manner of services
failing during boot with selinux in enforcing mode.  Although the
problem is generic, this first shows up in Fedora 30 as systemd has
become more strict about namespace failures (I think) [1].

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1663040#c22

Change-Id: I52c1cc719884879169b606b00651aa26f5b783f1
2019-08-16 15:44:55 +10:00
Carlos Goncalves
9eb175e440 Allow extra repositories to be added to images
This patch adds option DIB_YUM_MINIMAL_EXTRA_REPOS to yum-minimal to
allow DIB users to include extra repositories to their final image.

Change-Id: I89549f4b0f4c9470143b5064817acab5043e31c5
2019-08-15 16:20:21 +02:00