Commit Graph

170 Commits

Author SHA1 Message Date
Zuul
d01d3d8832 Merge "Make python changes more reliable" 2017-11-30 01:49:07 +00:00
Zuul
71ca627d30 Merge "Add debian minimal requirement for arm64" 2017-11-29 23:47:12 +00:00
Zuul
c146d2f3b7 Merge "Fix grub2 dependency on arm64" 2017-11-29 23:47:10 +00:00
Matthew Thode
ade82904a1
Make python changes more reliable
We oneshot emerge without calculating dependencies a few things to solve
for possible dependency loops.

Python 3.5 also became stable, so don't need to do special things for
it.

Matched the uninstall with the install lines (no need for a full if
statement).

Change-Id: I7c5e546612ac47d659e73a46a52e34d39ca81949
2017-11-20 23:19:46 -06:00
Zuul
b879a03c75 Merge "Dont install python-pip for py3k" 2017-11-20 05:38:24 +00:00
Markos Chandras
da02f37de1 elements: Respect devpts mount options
This is a continuation for f2cc647dae ("diskimage_builder: lib:
common-functions: Fix options for devpts mount"). We also need to
respect the devpts mount options when the dib elements are mounting
this virtual filesystems themselves.

Change-Id: Iee44703297a15b14c715f4bfb7bae67f613aceee
2017-11-14 08:31:55 +00:00
Gregory Haynes
ab89c7d69c Dont install python-pip for py3k
We want to install python3-pip, not python-pip when we are building a
py3k image less we pull in python2. Once we stop installing python2 we
have to stop calling python2 during pip install.

Change-Id: I7d8ba9300039cce90965410a4e16ca9e711904c3
2017-11-13 23:00:52 +00:00
Zuul
bc6c928bb9 Merge "Move to a common lock-file directory" 2017-11-07 17:32:28 +00:00
Zuul
906a3f4a57 Merge "Use -t devpts for /dev/pts mounts" 2017-11-02 14:20:05 +00:00
Zuul
1b0631da84 Merge "Update proliant-tools to support Gen10 Proliant servers" 2017-11-01 07:22:03 +00:00
Ian Wienand
b25d0337b8 Move to a common lock-file directory
In a couple of places we use flock for critical sections, but we leave
lockfiles around in various locations which can be confusing.

Introduce DIB_LOCKFILES global (under ~/.cache/dib/lockfiles) and
write lockfiles in there.

Fix up removal of the lockfile in the yum path; we just want to make
sure we cleanup the .rpmmacros file, but we don't need to remove the
lockfile as well.

Co-Authored-By: Andreas Florath <andreas@florath.net>

Change-Id: Ie810b2836be521325afe923708d046112e1e1e20
2017-10-26 16:27:59 +11:00
Zuul
8f025691ba Merge "Dont install python-virtualenv for py3k in deb" 2017-10-24 06:33:45 +00:00
Zuul
c5f713b1ec Merge "Change to install a package in 'proliant-tools'" 2017-10-24 05:48:25 +00:00
Andreas Florath
cebfcf85f9 Use -t devpts for /dev/pts mounts
Currently a bind is used when mounting /dev/pts in chroot.
This leads to problems - especially when running DIB in parallel:
It was observed that the /dev/pts mount vanishes from the host
system.

This patch uses '-t devpts' - as it is done for /sys and /proc -
for handling /dev/pts.

Change-Id: Id7775ae6fca6502af800e7b73a00862ef320206b
Signed-off-by: Andreas Florath <andreas@florath.net>
2017-10-23 07:13:02 +00:00
Gregory Haynes
00d7c619e9 Dont install python-virtualenv for py3k in deb
On ubuntu we detect that in python3 we need to install
python3-virtualenv, but append this to the packages to install rather
than replace python-virtualenv which results in both being installed
(and therefore grabbing python2).

Change-Id: I422490ebe9a9c655552685bc2ff342d288335a9c
Closes-Bug: #1724656
2017-10-18 23:11:55 +00:00
Zhangfei Gao
6cc155fd66 Add debian minimal requirement for arm64
Debian system building fails, fixed by adding arm64 package.

DIB_RELEASE=jessie disk-image-create debian ironic-agent \
    grub2 devuser -a arm64 -o deploy-jessie
dib-run-parts Running /tmp/dib_build.v5FEtaKx/hooks/cleanup.d/99-extract-kernel-and-ramdisk
ls: cannot access /tmp/dib_build.v5FEtaKx/mnt/boot/vmlinu*: No such file or directory

Change-Id: I610d767785df49fed954f12854be5ae78ff9baa6
Signed-off-by: Zhangfei Gao <zhangfei.gao@linaro.org>
2017-10-16 13:39:50 +08:00
Zhangfei Gao
a8635d3a3b Fix grub2 dependency on arm64
Unable to locate package reported when
DIB_RELEASE=jessie disk-image-create debian ironic-agent \
grub2 devuser -a arm64 -o deploy-jessie

E: Unable to locate package grub-pc-bin
E: Unable to locate package shim-signed
E: Unable to locate package grub-efi-amd64-signed

Fix the issue via adding arch dependency and arm64 packages

Change-Id: I40650a887b575a9c2b00a8c5036c35354d548673
Signed-off-by: Zhangfei Gao <zhangfei.gao@linaro.org>
2017-10-16 13:39:34 +08:00
Matthew Thode
e29f031bec
Update Gentoo element for element changes
There have been a few changes over the past few months, here we make the
following changes.

* change from backtrack=99 to complete-graph as a more correct flag
* make python version selection more in line with what gentoo supports
* set up python before stuff gets pip installed
* ensure we have the proper pip so we can install pip packages as root
* ensure we have the proper use flags for the disk formatting changes
* set DIB_RELEASE like other distros
* fix openssh-server element for gentoo

Change-Id: I17202de3016616ce34c8cbead7d0fb047a64e96b
2017-10-08 12:02:46 -05:00
Anshul Jain
afb7084a4d Update proliant-tools to support Gen10 Proliant servers
This commits make update to ssacli version to point to latest
ssacli release that has support for HPE P/E-Class SR Gen10 controllers.

Change-Id: Ia9a0eaec78d601f56b4036e57601554b87f21acc
Closes-Bug: 1721185
2017-10-04 07:02:07 +00:00
Ian Wienand
df00e9adcb Add initramfs-tools for ubuntu-minimal
A small update was made to 4.4.0-96.119 that dropped the
initramfs-tools dependency from the kernel [1].  This had the
unfortunate affect of removing the initramfs from ubuntu-minimal and
making it unbootable, since we specify the root device via LABEL=.
Add the package explicitly alongside the kernel.

Also, small fix to pass unit tests

[1] https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1700972

Change-Id: I57a0f08cd5e082ecdf8dba0ab34fb3062c50836d
2017-09-21 10:42:11 +02:00
Aparna
7bc2b23290 Change to install a package in 'proliant-tools'
This commit adds change in 'proliant-tools' element to
install a package 'unzip' which is required to perform
SUM based firmware update for HPE Proliant servers.

Change-Id: Ib8f6d18402439edd93d100cc7a4fb2094c863715
2017-09-18 08:04:34 +00:00
Jenkins
22e03f9820 Merge "Add missing package dependency for yaml" 2017-09-15 13:52:35 +00:00
Jenkins
254875dbde Merge "Create /etc/machine-id for fedora" 2017-09-15 04:05:51 +00:00
Jenkins
6e266399fb Merge "Use latest Fedora .qcow URL" 2017-09-15 02:36:24 +00:00
Ian Wienand
768c5e188c Create /etc/machine-id for fedora
As described in the comment, we need to create the /etc/machine-id for
the image-based build when systemd isn't updated (as is usually the
case for a new distro)

Work on clearing this out continues, but this brings it to parity with
fedora-minimal.

Change-Id: Icbbbabb4114d4d95909648d8e39a6bae6d2a7b7b
Depends-On: I761e425f8a658669d9b8a70ce4260cec263ea51a
2017-09-15 11:54:01 +10:00
Ian Wienand
7774260b76 Use latest Fedora .qcow URL
The URL we are using seems to have disappeared.  Update this to
download.fedoraproject.org.  The new URL requires a "subrelease" now,
add it, along with a note on where it comes from.

Change-Id: I761e425f8a658669d9b8a70ce4260cec263ea51a
2017-09-15 11:06:22 +10:00
Yolanda Robla
da5c926fe9 Add missing package dependency for yaml
This element was assuming that yaml was included as package,
but there are systems not including it. So properly add yaml
as a dependency.

Change-Id: I72da2776674a3963657052b9a9715abcb4fab1e2
Partially-Fixes-Bug: #1715686
2017-09-13 14:16:41 +02:00
Yolanda Robla
3ff8d1e10b Move the ordering of the dracut regenerate command
When using combined with rhel7 image, the unregister of repos
has already happened, because it is executed under 60- ordering.
As dracut-regenerate may need to install extra packages for it,
it causes this step to fail, because it cannot find repos where
to pull the packages from.

Change-Id: I35e37df7990ad76a5004cb90fdd863ec743a5483
2017-09-06 12:52:05 +02:00
Ben Nemec
72d0d22cdf Remove nested quotes from TAROPTS
Per the bug report, these seem to be causing issues with maintaining
file capabilities.  They aren't necessary so let's just remove them.

Change-Id: I06c90fdc85655986142b936cadbe04d75dd27427
Closes-Bug: 1714604
2017-09-01 17:29:15 -05:00
Jenkins
39d84d2059 Merge "Use [[ for =~ matches" 2017-08-29 05:59:42 +00:00
Ian Wienand
c448864901 Use [[ for =~ matches
Avoid incorrect use of [ with =~ matching

I guess this doesn't trip "-e" because it's in an if-conditional.  I'm
looking at making bashate detect this; maybe we can run bashate over
things we know are scripts

Change-Id: Ia3fe2b978fae5bdaadbb1789058180d3ad950d00
2017-08-28 17:01:03 +10:00
Julia Kreger
6d64a2aee6 Fix cylical systemd config for dhcp-all-interfaces
In Ubuntu/Debian, the default dependencies cannot be relied
upon as we enter into a cyclical dependency relationship which
prevents the unit from starting.

Added the required configuration to the systemd unit file.

This issue has also been observed in glean[0], which has a nearly
identical unit file for interface start-up.

[0]: https://review.openstack.org/#/c/485748
Closes-Bug: #1708685

Change-Id: I23ac9510d1a21c7073bd33f76ba66fa04a8be035
2017-08-25 15:51:23 +10:00
Jenkins
ea23aa13a5 Merge "Add netbase to ensure /etc/protocols is placed for debian" 2017-08-15 00:37:27 +00:00
Julia Kreger
f19c45eb29 Add netbase to ensure /etc/protocols is placed for debian
Many programs rely upon /etc/protocols to be present
however the default debian image that is generated lacks
/etc/protocols. This is observable when building an image
for use with ironic via the ironic-agent element, since
the IPA agent fails to start as python needs /etc/protocols
to open a socket connection.

Added to debian-minimal as it is inherited into the debian
element.

Change-Id: Icc81635870961943707cf6b3f61a9ddbd51cb8fd
Closes-Bug: #1708531
2017-08-11 14:17:30 +00:00
Ian Wienand
a88a768e98 Clear up debian element documentation
There is some confusion in the readme's over what is happening.  The
original change (Iaf46c8e61bf1cac9a096cbfd75d6d6a9111b701e) split out
debian-minimal and made debian "... simply be a collection of the
extra things we do to make it look like a cloud-init based cloud
image"

Make this clearer in the documentation

Change-Id: Ibe6fad9c67b70a5e31e43e06419968135174fef3
2017-08-09 13:15:38 +10:00
Paul Belanger
7cbbee7ea3 Bump fedora/fedora-minimal DIB_RELEASE 26
Fedora 26 is now the latest release:

  https://fedoraproject.org/wiki/Releases/26/Schedule

We are building and using these in infra now

Change-Id: I012c2d28255be274e88abc2751d968bafaf76fbb
Depends-On: Ieba5f69020a13681074f72cfca2955071801b63a
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-08-04 14:18:05 +10:00
Ian Wienand
818b75af41 Move selinux packages to redhat-common
Change I008f8bbc9c8414ce948c601e3907e27764e15a52 has shown that we
build redhat images without the "semange" tool available, which comes
from the policycoreutils-python package (see also
I3f9e2c322d042a5dddba33451c0fc21a4d32a88a).

I403e7806ae10d5dd96d0727832f4da20e34b94c7 added some of the selinux
libraries to yum-minimal for ansible support, but not to others.

Given both these changes, it seems that selinux[-targeted],
libselinux[-python] and policycoreutils[-python] can reasonably
considered part of all base images.  Move the selinux related packages
into redhat-common.

This also adds it explicitly to install_test_deps.sh.  It was actually
being dragged in by the docker install, but is a required component
for building (should be in bindep, but not there with that yet).

Change-Id: Idd4ae71ee6deee84604823b6b5dc4a845f316e01
Related-Bug: #1707788
2017-08-01 11:08:54 +10:00
Jenkins
308783d382 Merge "Switch openSUSE to 42.3 by default" 2017-07-28 05:14:23 +00:00
Alfredo Moralejo
b1961e14ea Use SELinuxfs to check selinux status
Currently, the cleanup script is using existence of
semanage binary to check if selinux is enabled. However
this is misleading and can lead to problems when selinux
is disabled in a system where the binary exist.

This patch changes the detection logic to use /sys/fs/selinux
directory which is a in-memory filesystem created only when
selinux is really enabled.

Change-Id: I008f8bbc9c8414ce948c601e3907e27764e15a52
Related-Bug: 1706386
2017-07-26 18:57:25 +02:00
Dirk Mueller
1c4c4fd734 Switch openSUSE to 42.3 by default
This is the latest stable release, so we should default to it.

Change-Id: I05643787002d339ccbf7a718847fe4ed6f39eacc
2017-07-26 08:56:02 +02:00
Jenkins
609bcee27b Merge "zypper: Clean caches and don't cache packages locally" 2017-07-26 02:25:40 +00:00
Markos Chandras
81e72d4045 elements: zypper-minimal: Install tar package
tar is an essential package but nothing pulls it explicitly. This causes
some issues in the openSUSE CI jobs like the following one

"Failed to execute tar: No such file or directory", "Failed to write
file: Broken pipe", "Failed to retrieve image file. (Wrong URL?)",
"Exiting."], "stdout": "", "stdout_lines": []}

Just like 'sed', add 'tar' to the list of packages for the openSUSE
minimal builds.

Change-Id: Ia36e3d9fd6b78862a6831ba80b43d4614a349ca0
2017-07-25 16:27:25 +01:00
Jenkins
a6da39acb8 Merge "Move setfiles to outside chroot with runcon" 2017-07-24 02:04:21 +00:00
Ian Wienand
5089e4e541 Move setfiles to outside chroot with runcon
As described in the comments inline, on a selinux enabled kernel (such
as a centos build host) you need to have permissions to change the
contexts to those the kernel doesn't understand -- such as when you're
building a fedora image.

For some reason, setfiles has an arbitrary limit of 10 errors before
it stops.  I believe we previously had 9 errors (this mean 9
mis-labeled files, which were just waiting to cause problems).
Something changed with F26 setfiles and it started erroring
immediately, which lead to investigation.  Infra builds, on
non-selinux Ubuntu kernel's, would not have hit this issue.

This means we need to move this to run with a manual chroot into the
image under restorecon.

I'm really not sure why ironic-agent removes all the selinux tools
from the image, it seems like an over-optimisation (it's been like
that since Id6333ca5d99716ccad75ea1964896acf371fa72a).  Keep them so
we can run the relabel.

Change-Id: I4f5b591817ffcd776cbee0a0f9ca9f48de72aa6b
2017-07-24 10:14:07 +10:00
Dirk Mueller
bfeb9d9e99 zypper: Clean caches and don't cache packages locally
For builds inside the infra, we don't want to pack the cache
inside the image (as it might be different at the time the image
runs). In an opensuse-minimal image this saves about 10MB of image
size.

Change-Id: I5ecabd46f0a662798bda3e4468395ad8308d0055
2017-07-23 17:24:24 +02:00
Jenkins
55971717b6 Merge "elements: openstack-ci-mirrors: Use openSUSE mirrors for gating jobs" 2017-07-22 05:22:34 +00:00
Jenkins
e029af993b Merge "Remove DIB_[DISTRO]_DISTRIBUTION_MIRROR" 2017-07-22 05:22:04 +00:00
Jenkins
7a70299668 Merge "Enable console during kernel boot on Power" 2017-07-20 03:55:19 +00:00
Ian Wienand
7ffe6856d6
Add -m flag to setfiles for Fedora 26
As described in the comment and associated bugzilla, the behaviour of
setfiles has changed in Fedora 26 to require "-m" situations where
labeled file-systems are mounted below non-labeled file-systems.  Our
loopback/chroot system appears to trigger this nicely, leading to a
setfiles call that does nothing without this.

Change-Id: I276c6f6a4fb44f4bea5004f6b4214f94757728ae
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-07-19 08:01:19 -04:00
Ian Wienand
6ffde2e596 yum-minimal: pre-install coreutils
As described in the referenced bug, the dependency solver in yum
doesn't handle weak dependencies well and in some cases, such as
Fedora 26, can end up choosing coreutils-single (the busybox-esque
single binary) instead of actual coreutils, which then causes problems
with conflicting packages later.

Change-Id: I2907bf3b74c146986b483d52cc6ac437036330b4
2017-07-18 14:51:18 +10:00