Deploying many nodes with the generated image shouldn't have the same
/etc/machine-id so clearing it and letting systemd generate a new
id upon first boot seems to be the best way to achieve this.
Change-Id: I73d0577d31464521b3989312fd9d982a1312a268
Closes-bug: 1707526
Closes-bug: 1672461
Change I008f8bbc9c8414ce948c601e3907e27764e15a52 has shown that we
build redhat images without the "semange" tool available, which comes
from the policycoreutils-python package (see also
I3f9e2c322d042a5dddba33451c0fc21a4d32a88a).
I403e7806ae10d5dd96d0727832f4da20e34b94c7 added some of the selinux
libraries to yum-minimal for ansible support, but not to others.
Given both these changes, it seems that selinux[-targeted],
libselinux[-python] and policycoreutils[-python] can reasonably
considered part of all base images. Move the selinux related packages
into redhat-common.
This also adds it explicitly to install_test_deps.sh. It was actually
being dragged in by the docker install, but is a required component
for building (should be in bindep, but not there with that yet).
Change-Id: Idd4ae71ee6deee84604823b6b5dc4a845f316e01
Related-Bug: #1707788
The MBR Partition Table Entry (PTE) allows one to specify many
possible partition types and one of the benefits of this is being able
to specify the CHS variant or the LBA variant.
By default, LBA only creates partitions of type 0x83 (of course,
that's only because the documentation doesn't tell you how to make it
do anything else).
I will take up Ian's suggestion in patch set 2 for a more rigorous
test in an independent patch set.
Change-Id: If3068535980eac2e58d4025444c65147a8c7fedc
Closes-Bug:#1703352
Currently, the cleanup script is using existence of
semanage binary to check if selinux is enabled. However
this is misleading and can lead to problems when selinux
is disabled in a system where the binary exist.
This patch changes the detection logic to use /sys/fs/selinux
directory which is a in-memory filesystem created only when
selinux is really enabled.
Change-Id: I008f8bbc9c8414ce948c601e3907e27764e15a52
Related-Bug: 1706386
tar is an essential package but nothing pulls it explicitly. This causes
some issues in the openSUSE CI jobs like the following one
"Failed to execute tar: No such file or directory", "Failed to write
file: Broken pipe", "Failed to retrieve image file. (Wrong URL?)",
"Exiting."], "stdout": "", "stdout_lines": []}
Just like 'sed', add 'tar' to the list of packages for the openSUSE
minimal builds.
Change-Id: Ia36e3d9fd6b78862a6831ba80b43d4614a349ca0
As described in the comments inline, on a selinux enabled kernel (such
as a centos build host) you need to have permissions to change the
contexts to those the kernel doesn't understand -- such as when you're
building a fedora image.
For some reason, setfiles has an arbitrary limit of 10 errors before
it stops. I believe we previously had 9 errors (this mean 9
mis-labeled files, which were just waiting to cause problems).
Something changed with F26 setfiles and it started erroring
immediately, which lead to investigation. Infra builds, on
non-selinux Ubuntu kernel's, would not have hit this issue.
This means we need to move this to run with a manual chroot into the
image under restorecon.
I'm really not sure why ironic-agent removes all the selinux tools
from the image, it seems like an over-optimisation (it's been like
that since Id6333ca5d99716ccad75ea1964896acf371fa72a). Keep them so
we can run the relabel.
Change-Id: I4f5b591817ffcd776cbee0a0f9ca9f48de72aa6b
For builds inside the infra, we don't want to pack the cache
inside the image (as it might be different at the time the image
runs). In an opensuse-minimal image this saves about 10MB of image
size.
Change-Id: I5ecabd46f0a662798bda3e4468395ad8308d0055
As described in the comment and associated bugzilla, the behaviour of
setfiles has changed in Fedora 26 to require "-m" situations where
labeled file-systems are mounted below non-labeled file-systems. Our
loopback/chroot system appears to trigger this nicely, leading to a
setfiles call that does nothing without this.
Change-Id: I276c6f6a4fb44f4bea5004f6b4214f94757728ae
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
As described in the referenced bug, the dependency solver in yum
doesn't handle weak dependencies well and in some cases, such as
Fedora 26, can end up choosing coreutils-single (the busybox-esque
single binary) instead of actual coreutils, which then causes problems
with conflicting packages later.
Change-Id: I2907bf3b74c146986b483d52cc6ac437036330b4
On a system where the packaged pip/virtualenv is up-to-date with
upstream (such as Fedora 26 ... for now), we don't reinstall, which
then violates a bunch of assumptions later on. Force install.
Change-Id: I6ebcda0351997fa7e32f0e6e77a98b2c33764e3f
It turns out dnf argparse can't handle negative numbers without "=".
It's actually documented in the man page
--latest-limit <number> ... If <number> is negative skip <number>
of latest packages. If a negative number is used use syntax
--latest-limit=<number>
But who reads that :) This started failing with Fedora 26
Change-Id: I884af94c07fa11b010f69863047a04711b14f21e
We expect LC_ALL for non-C locales to be working inside
images, so always install glibc-locale for openSUSE.
Change-Id: I8fe92773e377539070d9d9fe2960a6202bb80a18
In preparation for promoting the openSUSE jobs to voting ones we should
use the OpenStack mirrors. As such, the opensuse elements are modified
to make use of the DIB_DISTRIBUTION_MIRROR variable which is normally
exported by the openstack-ci-mirrors element.
Change-Id: Ie588c1c1eec13190cfb2ec718ba51f8c9878283f
We added the DIB_distro_DISTRIBUTION_MIRROR arguments with
I92964b17ec3e47cf97e3a3091f054b2a205ac768 as a way that we could
source a list of mirrors and then have the distro elements choose
which one applied to them.
However, this hasn't worked out to be so useful. The
openstack-ci-mirrors element is working as a mirror setup script -- it
translates the openstack CI mirror list variables into the generic
"DIB_DISTRIBUTION_MIRROR" as appropriate for each distro's build.
Also, it turns out there's other things that need to be done, such as
turning off gpg checking, which mean the idea of "just export
variables" hasn't turned out as valid ... you need actual code
involved to get it right.
AFAICT we never actually documented these, and they do not seem to be
in use. They have caused considerable confusion when dealing with new
platforms as we try to keep consistency. Remove them.
[1] http://codesearch.openstack.org/?q=DIB_.*_DISTRIBUTION_MIRROR&i=nope&files=&repos=
Change-Id: Ifc4ab700631ffdfbe790068558f670f9a11dde5e
The code in mkfs correctly extends the command line with a '-n' for
vfat but does not currently do it for fat. This means that mkfs for
fat ends up with a '-L' which is what you'd do for everything like
ext[234].
The change just treats fat like vfat in the one place where this check
is required.
Change-Id: If65dfd949acdadff33a564640fb42ea73026a786
Closes-Bug: #1703063
The purpose of the openSUSE element is to build openSUSE distribution
based images, so an additional community repo shouldn't be pulled into
the image. In addition the dkms dependency is blacklisted for SUSE
in the dkms element anyway, so this should be a noop.
Change-Id: I0aa06d9f4f110546032f910e3361840693d02de7
On Power systems console should be added the kernel command line
in the following order: 'console=tty0 console=hvc0'.
The first one is the graphical console. The last one is the serial
console. The kernel enables all the consoles pointed through the
kernel command line. However, only the last one will receive
input/output during kernel boot. All the other consoles will be
enabled after the boot.
Change-Id: I0069f608e0ab104d3778954e033fb82ed5ea7693
We replace the base resolv.conf with an "outside" copy so that
resolving works when we're in the chroot.
Installing resolvconf package modifies the in-chroot resolv.conf to a
symlink (to /var/run) which it wants maintained in the final image.
We have the existing "immutable" check for a created resolv.conf file,
but no eqivalent for a symlink.
This adds a check to see if the resolv.conf is a symlink and leave it
alone if it is, assuming it has been re-created in the chroot.
I have tested this with ubuntu-minimal+resolvconf with
dhcp-all-interfaces and the system seems to work with resolvconf
working correctly.
Change-Id: Idd5a26e9d55979bd951577d5b098ed4bfba91ad3
The python3 package actually contains some core modules (like the xml
one) which are not present in the python3-base on which is pulled by
the python3-devel package. As such, it's best to have it installed
similar to python-xml for python2.
Change-Id: I5cd5d1127ae62d6753c2ace44965179c5400bb9a
In order to support {CentOS,RHEL}7 for building cloud images we need to
handle the differences in grub packaging from Ubuntu. We also need to
populate the defualt location for cloud images for CentOS builds.
Change-Id: Ie0d82ff21a42b08c4cb94b7a5635f80bfabf684e
When a download redirector redirects to a broken mirror, timeout
quickly rather than waiting until the overall job is being timed out.
Change-Id: If7eb63d406aaf61f71aa9203cf708c474aa63fd0
The 'packages' variable already contains the packages we need so
use it instead of duplicating the packages.
Change-Id: Id22e1862f9654e66252d03a0fed9839cf004d750
Several people have popped up in IRC recently with failures in these
elements. Without Python 2.7 available in the image they are
unsupported (OpenStack hasn't supported it for a long time). Remove
these to avoid further confusion.
The centos/centos7 DISTRO split that has happened with centos-minimal
is unfortunate but I don't think it helps to rename centos7/rhel7 ATM.
To summarise; DISTRO=centos7 means image based build,
DISTRO=centos && DIB_RELEASE=7 means the minimal build.
In the future, I think it is important that the minimal builds and
image builds set the same DISTRO. This reflects that "upper" layers
shouldn't care about the exact building of the lower layers. I see
CentOS 8 going one of two ways
1) the changes are so significant, we start separate centos8 /
centos8-minimal elements. They both set DISTRO=centos8 (and
DIB_RELEASE to point-release maybe?). This means we have to update
all "if DISTRO == centos || DISTRO == centos7" branches to also check
for "centos8". Evenually (!) "centos" goes away for versioned DISTRO
only
2) we restore centos element with DISTRO=centos and DIB_RELEASE=8, and
centos-minimal remains the same. This means we have to audit all "if
DISTRO == centos" calls to make sure they're appropriate for version 8
(stick a "&& DIB_RELEASE=7" on them all basically).
I'm not sure we can fully decide until we start to see excatly how the
distro switching/matching bits look, but (2) is consistent with Ubuntu
and probably the preferred solution.
Some "rhel" parts have been cleaned up. More could be done in
rhel-common, but given our lack of coverage of that I'd prefer to
leave it for now.
Change-Id: I6ea784116ef59ca22878c8512c963f29c815a00a
The image download tests have long been too unreliable for the gate.
We need to cache the base images similar to how devstack caches it's
testing images. Let's move them to non-voting jobs for the time
being.
This means that the gate jobs are now all based on "-minimal" and are
using infra mirrors. Unfortunately, there is still some unreliability
because we currently have issues with infra mirrors being very slow
after AFS updates, leading to job timeouts. But we're on the right
path...
Also, I noticed we don't have tests of the "ubuntu" image-download
based tests, which were tacitly being tested by apt-sources before we
moved that to -minimal. Add simple tests for these.
Change-Id: Ie33ee49656872467ef68d753210032156bb6b2cb
In a system where python2 is not installed and /usr/bin/python is not
linked then the cleanup process will fail trying to invoke the python
script. Use the previously determined DIB_PYTHON_EXEC if it's available.
Change-Id: I128292808ccef92cc1803988b35caae5aa6fa541
