As motivation for this; we have had two breakouts of dib in recent
memory. One was a failure to unmount through symlinks in the core
code (I335316019ef948758392b03e91f9869102a472b9) and the other was
removing host keys on the build-system
(Ib01d71ff9415a0ae04d963f6e380aab9ac2260ce).
For the most part, dib runs unprivileged. Bits of the core code are
hopefully well tested (modulo bugs like the first one!). We give free
reign inside the chroot (although there is still some potential there
for adverse external affects via bind mounts). Where we could be a
bit safer (and could have prevented at least the second of these
breakouts) is with some better checking that the "sudo" calls
*outside* the chroot at least looked sane.
This adds a basic check that we're using chroot or image paths when
calling sudo in those parts of elements that run *outside* the chroot.
Various files are updated to accomodate this check; mostly by just
ignoring it for existing code (I have not audited these calls).
Nobody is pretending this type of checking makes dib magically safe,
or removes the issues with it needing to do things as root during the
build. But this can help find egregious errors like the key removal.
Change-Id: I161a5aea1d29dcdc7236f70d372c53246ec73749
This is breaking deployments that use registration. I
believe os-refresh-config scripts don't source environment.d
files, so the variable is unset there.
Given that this a blocking issue for RHEL deployments, I'm
pushing a quick revert and we can come up with a proper fix
when this isn't stopping other work.
This reverts commit 71bd8b3a33.
Change-Id: I87504660352220d45f5445bb933edc1c28885fcb
Hardcoding subscription-manager to use rhel-7-server-rpms causes
users building rhel6 with Satellite6 to fail. This setting cannot
be overridden with environment variables, therefore needs to be
smarter. Setting RHEL_MAJ_VER in the rhel/rhel7 environment fixes.
Change-Id: Ifbd88bc76ef8b38a739272ba6e045a12849d68df
Closes-Bug: 1404364
The loopback handling in the Linux kernel limits the filenames of
files associated to loopback devices, see also linux/loop.h.
This is reflected also on userspace, as kpartx will silently do nothing
(exiting with 0) when requesting to remove a filename longer than 64
characters, as that name will obviously not match the truncated
filename. The result of this is that, when extracting qcow2 images for
the first time, if the qcow2 filename is long enough then the loopback
device will not be removed, remaining as stale in the host.
As a workaround, use a temporary file name when convering a qcow2 image
to raw, instead of using the base name of the qcow2 file.
While this still will not fix the issue when manually using a long
temporary directory (e.g. TMP_DIR=/very/long/etc...), at least should
avoid it in other cases.
Change-Id: Ibf46cd313a9d89412c0e1068fa0993be6c5a29db
There is a wide variety of tracing options through the various shell
scripts. Some use "set -eux", others explicity set xtrace and others
do nothing. There is a "-x" option to bin/disk-image-create but it
doesn't flow down to the many scripts it calls.
This adds a global integer variable set by disk-image-create
DIB_DEBUG_TRACE. All scripts have a stanza added to detect this and
turn on tracing. Any other tracing methods are rolled into this. So
the standard header is
---
if [ "${DIB_DEBUG_TRACE:-0}" -gt 0 ]; then
set -x
fi
set -eu
set -o pipefail
---
Multiple -x options can be specified to dib-create-image, which
increases the value of DIB_DEBUG_TRACE. If script authors feel their
script should only trace at higher levels, they should modify the
"-gt" value. If they feel it should trace by default, they can modify
the default value also.
Changes to pachset 16 : scripts which currently trace themselves by
default have retained this behaviour with DIB_DEBUG_TRACE defaulting
to "1". This was done by running [1] on patch set 15. See the thread
beginning at [2]
dib-lint is also updated to look for the variable being matched.
[1] https://gist.github.com/ianw/71bbda9e6acc74ccd0fd
[2] http://lists.openstack.org/pipermail/openstack-dev/2014-November/051575.html
Change-Id: I6c5a962260741dcf6f89da9a33b96372a719b7b0
We check python files with dib-lint rather than flake8 which have
conflicting opinions. This means weve been (forcibly) writing non pep8
python.
Also fixing pep8 issues so tests pass.
Change-Id: Idc9db40334f6e15738a7802c06697270df68741c
This patch writes a warning out to stderr to notify element authors
that may be using map-packages to migrate to pkg-map. This patch
also prints out a warning during image-create that lists the specific
elements that are not using pkg-map.
Change-Id: I7e2a7611dd5650815736ce998aa94a7833193d06
When running inside a Docker container, we cannot rely on devices in
/dev/mapper to be automagically created by udev, because we probably
don't have a udev at all. To work around this, run dmsetup mknodes
after every kpartx run.
Change-Id: If7e30579224ce54c5ed26d08974d8293c144719a
Just remove globally /lost+found after the root and install phases, so
distribution elements don't need to do that by themselves.
Change-Id: Ic783e613bd8794aefd3f40c9a7c308d14cd04b8d
The default value points to old image name
which results in 404 error. So updating the
value to point to correct name.
Change-Id: Ia66f0cf4f4167926892a0786467412d277ebffc3
Rename rhel 10-yum-blacklist to 10-rhel-blacklist to
avoid future name conflict with a yum element blacklist.
Change-Id: Ic2136a06e1ec8f19908ada978782733d5fa7cacc
There is a substitution of suffix in rhel element, like
rhel-guest-image-6.5-20140121.0.x86_64.qcow2 to
rhel-guest-image-6.5-20140121.0.x86_64.raw
The convert is correct but the substitution of the suffix is wrong.
${PARAMETER#PATTERN} will match the string from the beginning and generate rhel-guest-image-6.5-20140121.0.x86_64.qcow2.raw
We should use ${PARAMETER%PATTERN} instead, which is from the end.
Change-Id: Ic45a3f800058ea9e5d746ff543c068a54632407d
Updates the various operating system elements so that we
set the DISTRO_NAME variable for each distro.
This env var is used by bin/pkg-map to set the default
distribution name for package name mappings.
Change-Id: Ib4b05eb7191dd50d25799a0bac51fd15c01b74cb
If 'DIB_RHSM_USER' isn't set, the image build with rhel
element fails because of check 'set -u'. So, we should export
'DIB_RHSM_USER' in script to avoid error during build
Change-Id: Ic63a20e3d91a25bbe804440a5919b035489c60a3
Closes-bug: #1318717
As with the previous similar changes, this is intended to catch
problems as they happen instead of ignoring them and continuing on
to potentially fail later. Setting this on all existing scripts
will allow us to enforce use via Jenkins.
Change-Id: Iad2d490c86dceab148ea9ab08f457c49a5d5352e
Using set -e in all of our scripts will prevent some subtle bugs
from slipping in, and will allow us to enforce use of set -e with
tooling.
This change also adds -u and set -o pipefail in the less complex
scripts where it is unlikely to cause problems. A follow-up change
will enable those options in the complex scripts so that if it
breaks something it can be reverted easily.
Change-Id: I0ad358ccb98da7277a0ee2e9ce8fda98438675eb
Provide package name mappings for openjdk-7-jre-headless:
RHEL/Fedora: java-1.7.0-openjdk-headless
SUSE: java-1_7_0-openjdk-headless
See https://review.openstack.org/#/c/86314
Change-Id: I3a351d0f9854f6f320ba145a71ffc66fd02c231c
Unlike the redhat-common and opensuse map-packages maps, the RHEL one
appears to be unsorted. Fix that.
Change-Id: I75c5d2694525731660c2a3b5b58ee696c61923ba
Use "operating-system" to denote that an element provides that to the
image being created. This can then be used to specify an abstract
dependency in other elements to assist in build-time image verification.
Change-Id: Ic125803da65e0f8f8c20ef90b08815f4f7d09762
Some of the tripleo-image-elements were failing for RHEL due to lack of
build essential, python-dev and libz-dev.
This commit should fix those failure.
Change-Id: I080f9601faedb3259deada5cfbda7d00d3f13eb9
The RHEL 6 Guest Image has been updated with a new release that
addresses a number of bugs. The diskimage-builder element that
references this file needed to be updated to reference the correct
image name. In addition, some notes to help users download the
image from RHN have been added to the README.
Change-Id: I3c9e64a1887afd9c0b370d70eff4b7f9598cebd1
Closes-bug: launchpad-bug-1286568
All of the yum-based install-packages scripts are doing essentially
the same thing, so let's use one script for all of the elements
that depend on the yum element.
Change-Id: I49e8c9b44e41bcf4cb9fa820e8a9179754694a97
A Red Hat Satellite URL may be provided as the source for
Red Hat packages. A Satellite key may be provided for a pre-
defined set of RHN channels to be automatically applied. Support
for installation of a Satellite certificate also provided.
Change-Id: Iae5b97d6f4732a28efa08ab778a56c2f9b6b6bd9
When registering to RHN or RHSM a list of space-separated
channels or repositories may be provided. This allows control
over which packages are available to the system during build.
Note, the optional channel or repository is added by default
since diskimage-builder requires access to its packages. This
list is for additional channels or repos.
Change-Id: I9cf90d47ab1d09f5491f574132410438f9e294e3
This provides basic support to register to the Red Hat Network
(RHN Classic) for packages. The image is unregistered from RHN
before completing image build. The rhnsd service is prevented
from starting since it is not needed for packagage install.
Change-Id: I2dc124cdc6ae59a63fd94d82d77108acccc3f668
Red Hat periodically updates the qcow2 guest image available
for download. This sets the default image name to the latest.
Change-Id: Iba3075bbee3b41918d5cd3da9721fcbf98ff3bcd
install-packages is an executable script, it should be +x. The install
command that was installing it is setting permissions to 755, so we were
fine there. However, I was trying to test the script outside of an
image build to test a change, and it's helpful to have it +x in the
repository for that context like all the other scripts under bin.
Change-Id: Ic300cc56d463640d119db4e310d375c66ed133ca
tgt maps to scsi-target-utils on RHEL. This mapping already exists for
the Fedora element, so no need to add it there.
Change-Id: Ifc9a8976299458cdef50c2fbe00ba6a0c8d22f00
Closes-Bug: #1252975
The key 'RPM-GPG-KEY-redhat-release' may not exist in image
e.g. when you use non-default image (for example, if you build CentOS
image using rhel element).
Fixes bug #1252697
Change-Id: Iae642abb7a7f76c5528750dafedd4fd4bda56fe2
Since we are using bash syntax in some of the element fragments,
we should make sure we use bash for all of them, so that things don't
break on systems where /bin/sh != /bin/bash.
Change-Id: If2f043c57aa4e1492b7f9839213ef6123f683612
The command `sudo rmdir $TARGET_ROOT/lost+found` will fail
if `$TARGET_ROOT/lost+found` directory doesn't exist,
e.g. when you use non-default image.
Fixes bug #1245856
Change-Id: I48c8f2f201b29912a726249023ca7d20893cc958
