Set the pbr option 'warnerrors' to make build_sphinx turns warnings into
error. Fix all warnings.
`tox -edocs` will thus abort whenever someone introduce a new error.
Change-Id: Id6d09768a241866e1fdc1a1e2bf90336f5c5087d
debootstrap is not debian or ubuntu specific. We can make a debootstrap
element that knows how to do all of the things, and then a
debian-minimal and ubuntu-minimal image that use it. Finally, make
the debian element simply be a collection of the extra things we do to
make it look like a cloud-init based cloud image.
Change-Id: Iaf46c8e61bf1cac9a096cbfd75d6d6a9111b701e
glean is now moved into the openstack-infra repos, so the reference to
the originally temporary home can be discarded.
Change-Id: Ie89fff85e264a36d9bab15801314d5195b45031c
In some cases, like linux-image-* on debian, we need to only install
packages for a specific target architecture.
Change-Id: Ic0009d0c1e121d6f3f1f21345c544e2d98f080f9
This change uses blkid to identify the fs type during redhat-common
extract-image. The image is mounted with -o nouuid for redhat/rhel
images that have XFS filesystems.
This is required when building images from the same base image
as the host VM to avoid "Filesystem has duplicate UUID" mount
failures.
Change-Id: I066289fbb27733a5a555242a0e2c363d58dd27d0
Closes-Bug: 1443706
Now that we have a generic yum-minimal element, just use it in centos
instead of rinse. Adding base as an element-provides of yum-minimal
because this element conflicts with the base element.
Co-Authored-By: Gregory Haynes <greg@greghaynes.net>
Change-Id: I15275d821781171c118f21aa0c0bca55f65a65b3
The loopback handling in the Linux kernel limits the filenames of
files associated to loopback devices, see also linux/loop.h.
This is reflected also on userspace, as kpartx will silently do nothing
(exiting with 0) when requesting to remove a filename longer than 64
characters, as that name will obviously not match the truncated
filename. The result of this is that, when extracting qcow2 images for
the first time, if the qcow2 filename is long enough then the loopback
device will not be removed, remaining as stale in the host.
As a workaround, use a temporary file name when convering a qcow2 image
to raw, instead of using the base name of the qcow2 file.
While this still will not fix the issue when manually using a long
temporary directory (e.g. TMP_DIR=/very/long/etc...), at least should
avoid it in other cases.
Change-Id: Ibf46cd313a9d89412c0e1068fa0993be6c5a29db
This commit changes Ironic deploy ramdisk to find out
the virtual media device by using labels instead of
looking at the model of block device. This helps in
finding out the device irrespective of the hardware.
Corresponding Ironic change is
If5b78d9af7048f2631d050ee5ce01ab7a67e2354.
Closes-Bug: #1429340
Change-Id: I5e8a706989bad13051eb47db0b1e762e6c672318
Commit b4a1f1c190 wrongly changed the format of the available
images. There was a missconfiguration on the buildservice which produces
the images so no static links were available. That's fixed now so use
the correct names again.
Change-Id: Iac4cbc8672da67f5a89ac2f1be8bb9530215ea19
The centos-minimal approach of using rinse does not, it turns out, work
on centos. That's a bummer. It's also rather heavyweight. Instead, with
minor machinations, we can just use yum itself pointed at a chroot.
Also adding fedora-minimal element which creates a fedora image using
the new yum-minimal approach.
Co-Authored-By: Gregory Haynes <greg@greghaynes.net>
Change-Id: I026fd9d323e786dae5bb67824c6501067e1ceaa3
If you don't want cloud-init, you may need to get a few things
from config-drive because you may be operating on a cloud with no DHCP.
In that case, simply reading some values from config-drive and writing
out either DHCP or static network info, in addition to grabbing ssh keys
is helpful. Both Infra and bifrost want this for their images.
Co-Authored-By: Gregory Haynes <greg@greghaynes.net>
Change-Id: I2746ed256b9783eab058b803130d3ccac484eaeb
We support building elements without depending on the base element.
Breaking install-types out into its own element while making base depend
on it so elements can depend in it without base.
Change-Id: I104543d5482c76f60902e9fc32d91e196eeab51a
Turns out that updating packages last causes some pretty
non-intuitive behaviour if you are trying to pin a package
to a specific version. Lets just update the base RPMs first...
subsequent installations should install the most updated version
anyways (unless they are pinned).
Also moves the package-installs script from the 00 step to 01 so
we can do the update first.
Co-Authored-By: Ben Nemec <bnemec@redhat.com>
Change-Id: I962046cc6048e852e6582fbc579f88bb73e23fdd
This fix prevents loading of unsigned ubuntu kernel in UEFI secure
boot environment when image is created using 'iso' element.
'iso' element uses 'linux' and 'initrd' modules of grub2 to load
kernel and initrd respectively. The grub2 implementation of Ubuntu
can load unsigned kernel when these modules are used.
Ubuntu has Grub2 modules 'linuxefi' and 'initrdefi' which exits
boot process if unsigned kernel is used in UEFI secure boot mode.
The 'iso' element should use these modules in grub.cfg to prevent
loading of unsigned kernel when node is booted in the UEFI secure
boot environment.
'linuxefi' and 'initrdefi' works seamlessly when node is booted in
normal UEFI boot mode (non-secure).
Fedora do not have this issue. This fix has been tested in Fedora
environment. It works fine.
Closes-Bug: 1443114
Change-Id: If256ba1f7d7c149482d0f37fabcdfa8ed22e3f91
ubuntu-signed element would install 'linux-signed-image-generic' that
provides signed kernel that can be used for deploy in UEFI secure boot mode.
Package 'linux-signed-image-generic' ships signed kernel with extension
'.efi.signed' (Ex. '/boot/vmlinuz-3.13.0-49-generic.efi.signed').
The kernel modules directory for signed kernel and unsigned kernel is same.
It is without 'efi.signed' extension to its name. This is different from normal
practice of directory naming in '/lib/modules' (Ex. For signed kernel
'vmlinuz-3.13.0-49-generic.efi.signed', modules directory is
'/lib/modules/3.13.0-49-generic').
This needed some changes in '/lib/ramdisk-functions' and 'ramdisk' element to
copy kernel modules.
The signed kernel package contains both signed and unsigned kernel. The
unsiged kernel is without extension '.efi.signed' (Ex.
'/boot/vmlinuz-3.13.0-49-generic'). This required change into
'/lib/img-functions' and 'baremetal' element to pick up signed kernel version
when this element is used.
Closes-Bug: 1443076
Change-Id: I60061cbea847b47fa752b9463cfd387e8e7f0635
The targetcli element was triggering a bunch of errors from dracut
when we installed all of Python. It turns out this is because there
were filenames with spaces in the find output and the loop didn't
handle that properly. This switches to a while loop that can
handle odd filenames.
Change-Id: Iacbf16f26f2bc9991840250dc8ae7990db54d811
Currently, calling the troubleshoot function in a ramdisk script
doesn't work as expected on dracut ramdisks. This adds an alternate
troubleshoot implementation that will behave as intended.
I did not make it conditional on a kernel param as was done in the
original because dracut can behave strangely if you allow it to
continue after an error. Always dropping to a shell immediately
should be less confusing.
Change-Id: I98000f4ac6d7890b1f44fe4d10394ac0ea332fcb
Do not rely on environment changes (like exporting REG_HALT_UNREGISTER)
to persist between different hooks run. This helps when the hooks are
run in different new environments every time.
Instead, in 99-unregister redo the same checks on REG_METHOD as done in
00-rhel-registration, still respecting REG_HALT_UNREGISTER in case the
user does not want to unregister the image generated.
Change-Id: Id594dcd72334f38a2fa96da21206da77a83d7a1a
Closes-Bug: #1434431
Cleaning up the apt-sources README to be easier to consume. Also
removing some tripleo references from the README.
Change-Id: I6937fd5cd51288b36890dde214701bcef1d61381
We don't want to trace the RHEL registration scripts because that
is likely to log things like passwords and activation keys. To
still allow for debugging failed runs, add sanitized logging of
the arguments passed to the registration commands, since that is
the part of the process where problems are most likely to manifest.
Change-Id: I0f661e9c152f43b814fda61211bd56ba93e3b9dc
The default locale set by cloud-init is now generated to prevent the
warning printed when the user is logged in.
Closes-Bug: 1440728
Change-Id: I2faff6c9d3ab8bb5f66d58e77bcf37f186bf501d
Make sure that the target directory for 50targetcli exists already, in
case there is no dracut installed at extra-data.d run time.
Change-Id: I85ade9e85e823b7564a5839c8b6181548a15ad41
This commit changes the 80-deploy-ironic script of
deploy-ironic element to report back the status of
boot loader install (when boot_option == "local")
using a newly introduced vendorpassthru.
Closes-Bug: 1422723
Change-Id: I9c1d8643be7cb9e273d65ddd791715a5c271fd93
Copy all of the necessary parts for a Fedora based dhclient to work. This
includes a number of network scripts. Also grab the ip command supplied by
the iproute package, the busybox "ip addr" command was missing the valid_lft
and preferred_lft options.
This will allow the dhcp to work in the ramdisk instead of getting passed the
PXE net config.
Related-Bug: #1417026
Change-Id: I8feee9a740855dab7b47162c5727bf91db77fcc6
The listing of *-$INSTALL_TYPE-install files currently uses ls, which
errors out when the glob matches no files, thus using true to not fail
it.
Instead, use find to collect the file list, so there is no need to
ignore the command errors.
Change-Id: Ic6888106858df320a1c90a84f1b9ec74d436b9e6
The wrong APT config name is used to disable download of translations.
It's Acquire::Languages, not APT::Acquire::Languages.
Change-Id: Ie0c12d444bab19b4486845944ef51031e9133470
Closes-bug: #1436523
Not all operating-system elements install cloud-init, but the base
element assumes its existence. Create the directory if it does not
exist.
Change-Id: I4bda8dc5d200825ea0c8163a4e5c44050a45083f
it may happen that if the system where disk-image-create runs is busy,
then the kpartx -l run may leave a stale autodelete loop device.
This is because kpartx -l first adds a new loop device, then does the
listing and removes the loop device. The latter may not end before the
end of the kpartx run, leaving a loop device marked as autodelete.
Such kind of loop device will automatically delete itself, so the
rm -r $WORKING
after
sudo umount -f $WORKING/mnt
in the EXIT trap will fail because $WORKING does not exist anymore.
To prevent this situation, just ask udev to finish its operations,
properly removing the (temporary) loop device.
Change-Id: I12246f3dbe6b5669e698767682a5a142f803823b
RHEL 7 does not ship tgtadm or tgtd so they cannot be used in the
deploy ramdisk. This change separates the tgt-specific parts of
the ramdisk into their own element, and adds a new one that supports
targetcli instead.
For now, the tgt implementation can only be used with traditional
busybox ramdisks and the targetcli one can only be used with dracut.
This is because dracut is primarily used for RHEL right now so it
makes sense to keep the dependencies simple. If there is a future
desire to mix and match the implementations that could be done, but
it would require users to explicitly select between tgt and
targetcli.
Change-Id: I4f99c91016287e08d836095c2f2261de8b45abdc
Co-Authored-By: James Slagle <jslagle@redhat.com>
It is reasonable that elements may need to include additional
kernel modules in a dracut ramdisk. This is done with the
--add-drivers option to dracut, but previously the value passed
was hard-coded.
This change allows an element to put a file containing its desired
drivers in a dracut-drivers.d directory, and the list there will
be added to the list of drivers added. This functions in
essentially the same way as the binary-deps.d directory that
already exists for including additional executables in a ramdisk.
Change-Id: Ie892b908d36c175a469f7cde7dd803ad4b1942b6
This is required on Fedora 21 in order to build some
packages via source. Includes files like:
/usr/lib/rpm/redhat/redhat-hardened-cc1
Specifically this fixed MySQL driver compilation issues on Fedora 21
for source builds.
Change-Id: I459f2203fa145049dda185da952813118193d573
Official MariaDB repositories offer the package : MariaDB-Galera-server.
This package has been now ported within Fedora (and also RDO), the
package is now called mariadb-galera-server. Yum install being case
sensitive hence this change.
Change-Id: Icd03877f17d01708b3916578991e42eef30a69e4
As part of the blueprint root-device-hints Ironic will pass some to the
deploy ramdisk some hints about which disk device it should pick to be
root device (the one where the image will be deployed on).
Before the deploy ramdisk would pick the first device it finds, but as the
machine could have more than one SATA, SCSI or IDE disk controllers the
order in which their corresponding device nodes are added is arbitrary
causing devices like /dev/sda and /dev/sdb switching around on each
boot time.
Plus, as people are adding support to build RAID arrays in Ironic we need
a way to tell it to use the just created device to be the root device.
The list of hints that could be passed to the deploy ramdisk so it finds
the right disk is:
* wwn (STRING): unique storage identifier
* serial (STRING): disk serial number
* model (STRING): device identifier
* vendor (STRING): device vendor
* size (INT): The size of the disk in GB
If not hints are passed, the deploy ramdisk will continue to do what it
did before to find the disk.
Change-Id: I8425f593e1a610af5a3697988702603ff218f2de
This commit adds support for uefi localboot in
deploy-ironic element. The change is to mount the efi
system partition (created by Ironic) in /boot/efi.
The corresponding Ironic change is
I00ac31da325676ea4ea1ac4185f5ac3a52c5809a
Implements: blueprint local-boot-support-with-partition-images
Change-Id: Idf7ac5987e14e1d31311834196ca7283deec15c6
Commit 36b59c001c introduces
DIB_DEBUG_TRACE, to be checked in element scripts for enabling tracing.
In the aforementioned conversion, few scripts were left with
unconditional "set -x" calls: remove them, changing the default value
for unset DIB_DEBUG_TRACE from 0 to 1, to retain their older behaviour
(as it was done in 36b59c001c too).
Change-Id: I3d1a9290021bf63de7d4e7752e809852e784ac8b
Previously, this code was not checking for the proper environment
variable for an element's installtype. There was a line replacing '-'
with '_' as is required, but that value was not actually used when
searching for the environment variable.
Change-Id: I0bbd56969188389db81844d9276269464870f776
/tmp does not contain anything useful anyway, and excluding its content
makes the initramfs smaller too.
Change-Id: Ia72867e0cdebacf668ac1a1f551a965da0d69694
This adds support to UEFI secure boot by copying signed shim and
grub bootloaders into ramdisk image.
Closes-Bug: 1419707
Change-Id: I1193cd3a9011855a6804966a31c7c0e28da90ada
The newest stable Fedora splits out kernel modules into
a separate package. By default this is not installed in
the Fedora cloud image... and it contains some things we
need for Ironic (iscsi_tcp module) among other things that
might be very useful.
Change-Id: I3374ea278fecfeb6552e4664717ef3646d382c17
Closes-bug: #1429504
The other distro elements set DIB_RELEASE which allows the other
elements to know what distro release is being built during the
extra-data or environment.d phases.
Change-Id: I00bf13410ded5b678ebc66ff191891ed3cc80f4f
This is the centos element, which builds CentOS 6.
There are a couple of modifications to redhat-common because
the version of tar in el6 doesn't support --xattrs-include.
There is a change to both pkg-map and svc-map to add 'centos'
to the 'redhat' family.
Also explicitly have to install cloud-utils growroot and
dracut growpart for proper fsresize at instance launch.
Also sets the DIB_EXTLINUX variable because there is no grub2
for this distro.
Change-Id: Iffd57bce1484c43c2cffcbdb37b602185216e63a
The construction of the combined svc-map-services file uses the host's
/tmp directory to store a work-in-progress file. That file's left behind
after it's done with.
There's a potential race here; together with problems that may arise
if two users attempt to run devtest on the same system.
This patch fixes this by creating the temporary file specifically under
$TMP_MOUNT_PATH.
Change-Id: Iecbdc583e37bed542249c316919c3712c28c7440
This allows them to install packages already specified in .yaml files
for them, which are not installed otherwise since the migration to .yaml
files.
Change-Id: Iac8c7eb55116938616e5299b84487cd52e0cbf6f
openSUSE appears to have redone their fs layout on the repo site
and also changed the naming scheme of their image file.
Change-Id: I75afe54b88ea531d3c1f3b7d85e5435cdacabed2
subprocess.check_output() has been introduced in Python 2.7, so the
script will fail when trying to install stuff in guests with Python 2.6
and older (like RHEL 6 / CentOS 6, for example).
Thus gracefully fallback to subprocess.Popen() when
subprocess.check_output() is not available.
Change-Id: I335148397932177810f095a942b993b249991107
Closes-Bug: #1415240
A recent commit (I29c73355a85112840b57f93b39a9eeda421875e6) updated the
epel element to rely on wget, but wget is not always installed. Since
wget is used in pre-install.d, we must install it in that phase.
Change-Id: Iad2a9c4f0081835a1aa78e6fd2f721f6afd44dc2
Also make the apt-sources element exit with a non-error status code
when the DIB_APT_SOURCES is not set.
This allows including the apt-preferences and apt-sources elements
even if DIB_DPKG_MANIFEST and DIB_APT_SOURCES are not exported.
Change-Id: I8507dd9c69d1371eab38f720dcfdd89aa5345e8b
When there is no pkg-map file for the specified element, list the passed
packages if --missing-ok is specified.
This mirrors what is being done already later, when a pkg-map file
exists but it does not have the mapping for the requested package.
Change-Id: I50287f8e6a5e4335deb3de2252075b8bfdd53ce5
epel-release 7.2 has been superseded by 7.5, so update accordingly.
this time, detect the available epel-release package from the repo
instead of hardcoding version number which will break build each
time the version number gets bumped up.
Change-Id: I29c73355a85112840b57f93b39a9eeda421875e6
Pip now supports retries on failure. If setting multiple pypi mirrors a
fallback will not occure until the failures have finished for the first
mirror. This can cause a substantial delay if mirror fallback occurs a
lot.
Change-Id: Iad37a9015a2d5c861a345a111bd1725b965a42d3
We don't export DIB_RELEASE in the base distro scripts, so many
times it will be an unset variable in 51-bootloader from the vm
element. Currently that script doesn't handle the situation
properly and we get errors like:
line 160: [: =: unary operator expected
This change quotes the references to DIB_RELEASE so that won't
happen anymore.
Change-Id: Ic8f91804334f8862c217f465be80e96d3116f5c8
We have a good pattern of namespacing env vars with DIB_. Add support
for DIB_PYPI_MIRROR_URL* and maintain backwards compat support.
Change-Id: I434c9d1b14cd571b20754c9ad7cd64c936f8399a
As we started using jq it makes sento to use it everywhere when
building a resulting JSON.
Also removes some unneeded code.
Change-Id: Ib1391dc9f4e1463a9a3d0c13909ff60e3c993e82
When setting IPMI credentials is requested, ironic-discoverd
will return ipmi_setup_credentials=true, ipmi_username and ipmi_password
in response. Use ipmitool to set credentials in this case.
Change-Id: I8ace9e817e5ce9dbfb8564feec4a3ce6c0e1b233
There is a wide variety of tracing options through the various shell
scripts. Some use "set -eux", others explicity set xtrace and others
do nothing. There is a "-x" option to bin/disk-image-create but it
doesn't flow down to the many scripts it calls.
This adds a global integer variable set by disk-image-create
DIB_DEBUG_TRACE. All scripts have a stanza added to detect this and
turn on tracing. Any other tracing methods are rolled into this. So
the standard header is
---
if [ "${DIB_DEBUG_TRACE:-0}" -gt 0 ]; then
set -x
fi
set -eu
set -o pipefail
---
Multiple -x options can be specified to dib-create-image, which
increases the value of DIB_DEBUG_TRACE. If script authors feel their
script should only trace at higher levels, they should modify the
"-gt" value. If they feel it should trace by default, they can modify
the default value also.
Changes to pachset 16 : scripts which currently trace themselves by
default have retained this behaviour with DIB_DEBUG_TRACE defaulting
to "1". This was done by running [1] on patch set 15. See the thread
beginning at [2]
dib-lint is also updated to look for the variable being matched.
[1] https://gist.github.com/ianw/71bbda9e6acc74ccd0fd
[2] http://lists.openstack.org/pipermail/openstack-dev/2014-November/051575.html
Change-Id: I6c5a962260741dcf6f89da9a33b96372a719b7b0
Both ccache and linux-image-generic were being installed by
install-packages rather than using a package-installs.yaml.
Change-Id: Ib54c5b51d605278bb9d046a897be989c17c6fb0f
Patch adds support to create PowerPC image with vm element.
It creates 2 partitions, one for PReP boot and other for root and
installs grub-ieee1275.
Change-Id: I4675ef2b82aa69b63e63a1cc7db01b0c0e6f9fff
Closes-Bug: 1418739
If there are no old-style package-installs and the package-installs
element is included we fail because we call install-packages with no
arguments.
Change-Id: I3c78b51d0a2673ecc581f9faded078974dbc27b7
Since the package-installs-squash script relies on the defined
environment variables for figuring out what the values are for an
element's install type and the default install types, sudo -E must be
used to preserve the environment.
Change-Id: Ice5cf0bca65d8078f84daedbef123fc73d5a7b63
Working on host systems without selinux, where the guest image
does have selinux, creates a situation where the instance will
have about a 1 minute delay on first boot because it must relabel.
The previous check for sysfs assumes that the host system has
selinux, which is not needed for the guest setfiles to work.
Change-Id: Ic186a45991b6d80880ad635e9c80985612f53a05
Closes-bug: 1414200
This patch extends the deploy-ironic element to make it install a
bootloader on the disk in case the boot_option parameter passed in the
kernel command line is set to "local".
This parameter is set by Ironic as part of the
blueprint local-boot-support-with-partition-images
Change-Id: I5ebe6f364ae0ac408939399e5f28728b41c4766e
Patch adds support for PowerPC Big-endian and Little-endian local
fedora VM images and using VM images with two partitions, PReP
boot and root partition. Since PowerPC requires PReP boot
partition to be present.
Change-Id: Iba03226e187609df898732c13b1aa5f895b156e3
Closes-Bug: 1413487
ramdisk-image-create assumes the presence of vmlinuz file in OS image.
But ubuntu PPC LE images have only vmlinux file. This patch adds
support for using vmlinux file.
Change-Id: I5dd0b8ceb46b73be57d4c15b39b96f99b524fa3f
Closes-Bug: 1413362
When set to true this cloud-init setting overwrites the /etc/hosts
file at reboot, breaking stuff in TripleO where controllers rely
on resolving each other by name. We still want cloud-init to add in
there a mapping for the local fqdn, so we switched to 'localhost' [1]
1. http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/doc/examples/cloud-config.txt#L470
Change-Id: I85468dece3b67f39841aa826f9b7ca4bf113027a
Closes-Bug: 1411594
This commit adds http_proxy=$http_proxy to use apt-get update through a
http-proxy.
Change-Id: Iee589f6d11b84991512493d22f0e3dbfe60736ec
Closes-Bug: #1412314
The rhel-common element has a script that uses subscription-manager
to register an instance of RHEL with the customer portal or Satellite.
If os-refresh-config runs this script more than once,
subscription-manager returns a non-zero value causing os-refresh-config
to fail. Adding --force to the subscription-manager calls doesn't
properly unregister the image from Satellite and the result is multiple
content host registrations. This patch changes the registration to only
run once.
Change-Id: If372e7f59bbdc77e2b233024c1342f005004f2e5
Previous to the Sattelite 6 release, the katello-agent and it's
dependencies needed the rhel-7-server-rh-common-beta-rpms repo
enabled. Since the the release of Satellite 6, the "beta" can
be safely dropped. This fixes an error where registration to a
satellite instance fails due to a 404 for the beta repo.
Change-Id: Ia9879498c7db0c912bb58ea084d448b8b3915569
The dpkg element was hardcoding ~/.cache/image-create as the cache path,
but elements should use DIB_IMAGE_CACHE as the root for the cache to
allow users to override the path. Replace the hardcoded path with one
that is rooted at $DIB_IMAGE_CACHE.
Change-Id: I76030654e7dfc7b7d4dfa87e082c14e7cee479b9
The correct workflow for building RHEL6 images includes not using
the rhel-registration scripts in this element, but rather the 00-rhsm
script. This patch updates the return value from 1 to 0 for the case
when the REG_METHOD is left unset. This will allow the RHEL6 images
to build without needing to set REG_METHOD. This patch also improves
the note about RHEL6 image building.
This patch is the result of the discussion in the comments of [1].
[1] Iff7b9fc30d5a36231598a977a9edcd55229766c5
Change-Id: I2f35b8d7d8749d44d88f06e9e2c3116ff93b88fe
Closes-Bug: 1404364
When using the rhel element, it can be uesful to disable registration.
For instance, if using an internal mirror repository that does not
require registration/authentication.
Change-Id: I42fa34e7f146137e5b1b9c5d7785702311e1dca1
Copy all of TMP_MOUNT_PATH into the ramdisk so that
ramdisk-install.d actions are respected.
Includes a workaround for a bug in Dracut that caused the copy
to fail.
Also moves init instead of copying it so we don't conflict with
the Dracut init.
Change-Id: Ia8d7b210369fe7eb7ab239cbdb7f96841104a35d
Closes-Bug: 1403121
These scripts are littered with references to passwords and keys,
so rather than try to disable tracing in every one of those places
I propose we simply turn it off for the scripts. This will likely
need to be revisited after the global tracing discussion is
resolved anyway.
Change-Id: I805015300bf01b2f8eec9f70ccfdf533dd9386b3
Related-Bug: 1298205
The latest update to package-install captures both stderr and stdout
from pkg-map, unfortunately, pkg-map has a 'missing-ok' option
which causes it to print an error message on stderr.
The result is that package-install tries to look for packages named
"Missing", "package", "name", etc.
Change-Id: I86b3b71a64b29d533b42fd0cae020e8ecf22cac2
Closes-bug: 1402085
The element was prepending the work directory forcibly, which will have
problems when the image name has an absolute path, which is allowed.
Change-Id: I0cb7b96e24daab8ee73611936af72074c70ac1aa
Closes-Bug: #1400405
Ubuntu has recently release their "Snappy Ubuntu Core" which is a
minimal base-layer for running docker applications. Seems like the sort
of thing someone might want to use for tasks.
Change-Id: I6cb724451d1862121dee4ccf1f599ab8938f0b7f
VMs on Rackspace Cloud need nova-agent installed to get networking
information set up properly. Add a nova-agent element so that dib can
be used to create images to use on Rackspace cloud.
Change-Id: I075a50e9a2d7ae7641905b66bbbc627ad708b5ea
There are times when a much more stripped down base image is desired
over the distro cloud images. For instance, Infra would like some base
images that do not have cloud-init or really much of anything else. This
is easy to accomplish with debootstrap and rinse.
Change-Id: I44ff22457165afb048fdaea469210ae47d83dd3f
Instead of doing the work in the image of parsing through the element's
package-install declarations, we can squash it on hostside, where we
have both YAML and JSON available to us, and then emit a single
pre-processed file into the target to be used later.
Change-Id: I3f182aa3aae0a79b2f3ea4e66c1878ad12878b0a
Depending on bootloader and configuration, ramdisks may receive
a ip= kernel parameter that is used for manual network bring up
in the ramdisk. This parameter contains the address of the boot
server. In the absence of ip=, we want the ramdisk to DHCP and
proceed as normal. However, in this case need to know the address
of the boot server. This falls back to a 'boot_server' parameter
for this address if it is not provided via kernel cmdline.
Change-Id: Id1ed15ba125d91fa0a13d8e19d1385082ee5d9e9
Closes-bug: #1401289
Related-bug: #1401298
The docker build folks obsess about image size and speed a bit. Grab a
few of their optimizations from their debootstrap build process and
apply them to ours.
Change-Id: Ic3d3565423b0ae090896d99fd3bf1145eca6303d
The username and password combination is considered insecure to store in
the metadata passed to the stack as they can easily be obtained and
possibly used in an unathorized manner by logging into one of the
registration systems. The use of an activation key is more desirable
as it can only be used in conjunction with subscription-manager to
register a RHEL system.
This patch deprecates the username and password support from the script
that registers RHEL with either Satellite or RH Customer Portal during
boot-time. This patch also adds a warning if the username and password
combination is used in the stack metadata. The documentation and examples
have also been updated to warn operators of the deprecation of username
and password.
This patch does not affect the username and password support for
registration activities while building images with diskimage-builder.
Change-Id: I05b7a18e910d31ad2273042409f8657ad9dee36a
Previously the REG_REPOS value was a space delimited list of repo
names, but the spaces were incompatible with passing through json.
If you pass a list in json through the heat parameter, the output
is the string representation of a python list of unicode vaules.
This patch changes the rhel-registration scripts to process the
REG_REPOS value as a comma delimited list which is more easily
passed through json. Both scripts have been updated to remain
consistent.
Change-Id: I21b3fd115e53aa3b0fa4af9bbfb1f08b6fe163ab
Allows specifying the path to a yum repo configuration file to be used
during the image build process. The repo configuration file is removed
at the end of the build. This is useful in that it allows using any
arbitrary repo configuration during an image build.
Change-Id: I7d5c67d74a0bb4722ffc60aacfd9fa7e80fb59d5
Using lsb_release to figure out what distro and release an image is
is redundant, as the same information is available in DIB_ vars.
Further, it causes errors when creating new rhel7/centos7 images
like this: http://paste.openstack.org/show/144729/
Change-Id: I213ee581b8abfe119ffe3b783e6a3236e58e3e47
The file extention claims rst format, but it's plain text
as such, the online documentation doesn't render properly
Change-Id: I24accb45ab8c7803a25f2642ce1b2d479d5a6e9c
This patch adds support to register with the hosted customer portal
using activation keys. If an activation key is present in either
the REG_ACTIVATION_KEY or rh_registration.activation_key, it will
use that value instead of username and password credentials when
registering with Satellite or the hosted Customer Portal. This
patch also enforces that an org must be set in either the REG_ORG
or rh_registration.org to use the activation key.
Change-Id: If40dd78ba793d508afb1a5ab345470ee5929afb0
We have a new package-installs file format. Migrating existing
package-installs files to the new format.
Change-Id: I57d4a007efee9624e60c41357cefa627d8c7373f
We currently support package-installs definitions which has some
limitations and oddities. This new format requires only one definition
which does not reside in our run-parts directories and follows a
consistent naming scheme (package-installs.yaml).
Change-Id: Ie51a7c4fdc15634ae8e069728e5e07cc1dc36095
Fedora sets requiretty globally by default. This makes sense for
interactive machines with lots of password typing, but is pretty
attrocious for machines that might need users who remotely sudo to do
things. Just remove the setting.
Change-Id: Ic32bd92061a73f854683cc0d2d8919071dabe8cf
We immediately remove this, but the -grub2 causes a later attempt to
remove it again which fails. Don't install it in the first place.
Change-Id: I31d64abc9596707bdb8a0505d0dcbf7b82e1b82d
sudo is not needed, since in-chroot elements are run in the context
of the root user. Furthermore, sudo in pre-install is problematic as
sudo may not have been installed yet (imagine a debootstrap build)
Change-Id: Ib5c7e176a90fe3b8fa9c3cd702d3d815df54f472
ironic-discoverd [1] is a StackForge project for conducting hardware properties
discovery via booting a special discovery ramdisk and interrogating hardware
from within it. It aims to be one of the official means of hardware properties
discovery for Ironic in Kilo release [2].
The ramdisk collects hardware information from the machine
it's booted on and posts it to the URL provided via
kernel argument 'discoverd_callback_url'.
[1] https://pypi.python.org/pypi/ironic-discoverd
[2] https://review.openstack.org/#/c/135605/
Co-Authored-By: Lucas Alvares Gomes <lucasagomes@gmail.com>
Change-Id: Ic81fe8b3bd0884971bb522b48658c7ee538a31f2
Using only a local filesystem mirror could lead opaque errors.
Print a warning message in this situations.
Change-Id: I5f77151ea928868f4c441e8a1bb2eb0966b21832
Closes-Bug: #1297948
An app using pkg-map (like package-installs) might want to distinguish
between a hard error (invalid pkg-map file) and a soft error (no
mapping found). Currently this is not possible because we only return
with error values of 1.
Also printing error messages to stderror so we can still make use of
stdout data during a soft error.
Change-Id: I8bef56d3a74e8530afb8c58ac50ca3e9f7700639
We check python files with dib-lint rather than flake8 which have
conflicting opinions. This means weve been (forcibly) writing non pep8
python.
Also fixing pep8 issues so tests pass.
Change-Id: Idc9db40334f6e15738a7802c06697270df68741c
This patch updates the 00-rhel-registration script to not
not fail a stack when metadata isn't found.
Change-Id: Ie305fed79e4baadf1a03c4a3d06a23cf36e92f77
This patch writes a warning out to stderr to notify element authors
that may be using map-packages to migrate to pkg-map. This patch
also prints out a warning during image-create that lists the specific
elements that are not using pkg-map.
Change-Id: I7e2a7611dd5650815736ce998aa94a7833193d06
The source-repositories element puts source in place in the generated image
for later installation. It is possible for two elements to define the same
source to later install, and as long as the remote location and reference to be
installed is identical, we should not error.
Change-Id: Ic7dfb1961486c59d2005b5c1e7587267ea42e999
This allows extra arguments to be passed to the debootstrap
command when creating an image using the debian element.
Change-Id: I87eb856365ff6a17f051e2e56587235648328d57
Co-Authored-By: Ghe Rivero <ghe.rivero@gmail.com>
It was depending only on deploy-baremetal. Changing it to 'deploy' could
be used with both baremetal and ironic deployments
Change-Id: Ia2600080b488f7d259a9baf111767e82b6cc933d
These are already listed in the respective deploy-* elements, and
if we include them here then they get added to every ramdisk built
with dracut. That causes issues for elements that don't need
something like tgtd because it won't be installed and the
binary deps script will fail.
Change-Id: Ibdcf7d200d4f7136396b63404cd966f7557043e0
This element creates kernel and ramdisk files based on Fedora,
example:
disk-image-create -a i386 -o test fedora ironic-agent disable-selinux
Change-Id: Ifa133d1680b81cb87d32a405aa7d7b40fe91f835
We don't actually need a real value for the root kernel param, and
requiring one causes problems for things like the discovery ramdisk
that don't pass in a disk= parameter. Dracut seems to be happy to
take /dev/zero as the value, so we can just always use that.
Change-Id: Ia724f0214c26aa18c6f8f41f2c48d7f25b52ee6c
This patch writes a warning out to stderr to notify element authors
that may be using map-services to migrate to svc-map.
Change-Id: Ic80db16c607958d025e89b3a4058a9cbb568938e
The URL we were using to download lsb_release from no longer works,
install from package so this is not affected by disappearing URL's
in future.
This was originally installed from a URL because the package dependencies
caused a 87M increase in the size of the base image. For fedora the
increase is now 27M which is a little more tolerable.
Change-Id: I6e56a4a81786b33c5c6b92df2bd8236cd3f19670
This patch documents the 00-rhsm script as <= RHEL6 as it doesn't
work with the latest release of Satellite. The Red Hat Network
(RHN) method of registration is being phased out in favor of Red
Hat Customer Portal and Satellite 6. The subscription-manager
command line tool is the preferred method of registration.
The registration of RHEL is required to enable repositories for
software installation during image creation time.
This element provides functionality for registering RHEL images
during the image build process with diskimage-builder's
disk-image-create script. The RHEL image will register itself
with either the hosted Red Hat Customer Portal or Satellite to
enable software installation from official repositories. After
the end of the image creation process, the image will unregister
itself so an entitlement will not be decremented from the account.
Boot-time registration is supported through metadata. Please see
the configuration in the README for more information.
Change-Id: Ia9ef377cc4ed9595633888bfb248a1224e04b542
Adds new disk-image-create --install-type option which
can be used to enable alternate install types. The
default install type is 'source'.
The motivation is to eliminate elements like
enable-package-installs which require coupling and also
don't work with elements in multiple element repositories.
This patch does not prevent you from using the previous
DIB_INSTALLTYPE_ variables to customize the install type,
rather it just changes the default so you don't have to
set it so often when using things like packages.
Change-Id: Icee98440fc2251728f2dca30e7c4789a0fd89b93
I would like to recommend to use + instead of \; in the find
command. As this will ensure the removal of all selected
directories in a single invocation.
Hence improve the speed of deletion.
Change-Id: I409fe11aae217afb6f790491591005c679264ed4
Our package-installs script fails when installing a package which does
not have a pkg-map but a pkg-map file exist for the element.
Change-Id: I3dab802e23bccfc916efcc1c70c6ce6c4a9ccf67
A similar change was needed in the normal ramdisk element as part
of ae928057bd to avoid running
cleanup immediately after the ramdisk build completes.
Change-Id: Ia96e2d8011b88ed96cc6727914c5a5d2dea59757
Current iso element uses 'search --file' to find the
grub root. This is a wrong approach as it may find
some other partition containing /vmlinuz. Instead modify
it to search for grub root by the label of ISO image.
Closes-Bug: 1384826
Change-Id: Id4217be3420597bed2f80a96788928259ec91582
After bootstraping a Debian image, the repository keys
are installed to verify the packages signatures, but the
Release signature file is missing. Updating the repo,
will retrieve a new InRelease file (inline signed).
Change-Id: I14f0d22cc9c72be9b07f3708270359bc8cff112d
Instead of hard-coding a list of binaries to include in the dracut
ramdisk, use the existing binary-deps.d functionality to provide a
list. This will allow other ramdisks (such as discovery) to add
the binaries they need.
Change-Id: Ib7ffa15e08db1cc45e93a8f2a5c01369772c93ff
Busybox should be installed and managed from the ramdisk element,
since that is common to all elements that need busybox. In
addition, we do not want busybox installed when building Dracut
ramdisks, so it can't be in the deploy-* elements.
Change-Id: I2656d20b466138f7f6dfcf558ba90c6909151d3c
This commit adds a new element named 'iso' to build a bootable
ISO image for the kernel/ramdisk emitted by the 'baremetal' or
'ramdisk' element.
Change-Id: I89d175a29e2d0bc64b47fe527f0d0f6875f6849a
During internal testing we are getting the following error from
merge-svc-map-files
File "/tmp/image.K2MYCphY/hooks/extra-data.d/10-merge-svc-map-files", line 54, in main
with open(data_path, 'r') as dataFile:
IOError: [Errno 21] Is a directory: '...diskimage-builder/elements/svc-map'
Somehow IMAGE_ELEMENT contains a extra white space that manifests itself
as an element without a name. It is very hard to find where this is coming from so
instead this patch makes merge-svc-map-files more robost to this situation
Change-Id: Id1500ead8a77d691408617dcdc4e095bc5775be8
The element builds dracut from source on Ubuntu because the
Ubuntu dracut package is broken and very old, so it can't be
installed properly and causes a number of other issues that
are fixed by using a newer version of Dracut.
This initial version should work in virtualized environments.
Further validation of its suitability for real baremetal
deployments will need to be done in the future, but this should
be sufficient to enable that work.
Regarding Dracut specifically, in order to limit the changes
needed in the existing scripts this element continues to use a
cut down version of the /init script that we were building for the
existing ramdisk. However, instead of running it as pid 0 it is
run as a Dracut pre-mount hook. This allows Dracut to set up all
of the hardware and system bits, while falling early enough in the
Dracut sequence to complete the deployment before Dracut would try
to boot off the hard disk.
bp tripleo-juno-dracut-ramdisks
Change-Id: I144c8993fe040169f440bd4f7a428fdbe3d745cf
This reverts commit 4bf38a829a.
This variable is pointless since we can always replace it with
/lib/systemd.
Commit I459f7514ab35082d31607968252a9005fa25de2a will fix the issue
in tripleo-image-elements.
Change-Id: I524badb6836a0d04a5f6e6a5b5d95920fd923ef6
We are autodetecting and configuring devices in dhcp-all-interfaces
so having one configured in /etc/network/interfaces by default
is redundant and slows boot down.
Change-Id: Ic4e8a0668c793d21ed2dd96908649c9a77264f67
Closes-Bug: #1239480
After being deprecated two releases ago, finally remove any reference
for the support of first-boot.d
Change-Id: I08d67404ef48cad61db3b18fb86e970abfa5d2b6
In our official image builds we are only allowed to use resources
that are "blessed" by the build system. This means that external
things like git repos and tar files are not allowed. Currently,
even in offline mode source-repositories expects those things to
be available in the cache, so we need a way to disable it entirely.
This change adds an environment variable NO_SOURCE_REPOSITORIES
that does so. It can be set in an environment.d script so elements
that might rely on a source repository will know it's not available.
The 02-lsb script in redhat-common is one such example and is
updated to handle this case.
Change-Id: I0de63bee6ad79733d6711478c707a9b41593e85f
This is the final released image; it is the same one deployed in many
cloud provers such as HP and AWS. It is also safe from all known
vulnerabilities up to its release
Change-Id: Ib8ba561662d2a98da55eb292b850dc019226dd59
Each distro had its own copy of the file, but they were all the
same and it's actually installed in redhat-common so it belongs
there.
Change-Id: Iad15f39fd28d1c4e20d30dcb2eb0ae6f2fa2b015
When uploading images to multiple clouds it is possible that the same
image will be needed in multiple formats to accomodate hypervisors
across clouds. Update disk-image-create's -t flag to take a list of
desired output image formats so that a single disk-image-create can
output all of the desired image formats.
Change-Id: If121b2342ae888855ba435aa3189f039e985b812
The centos7 images come with the base version of cloud-init already
installed.
---
$ virt-inspector CentOS-7-x86_64-GenericCloud-GA-7.0.1406_01.qcow2
...
<application>
<name>cloud-init</name>
<version>0.7.5</version>
<release>10.el7.centos.1</release>
<arch>x86_64</arch>
</application>
...
---
I think this is a hang-over from early rhel7 images that didn't have
it.
Change-Id: Idf488942222d9e6bca4d66f04f5af42e6707944d
instack runs this script on a system where epel has already been
installed, which causes the yum command to fail. Only installing
the epel-release rpm when it isn't already present addresses the
issue.
Change-Id: Ia179d7c7ed69ea5f785b5dcd16394c7663a145cc
rpm-distro element was trying to install bin programs into the
image but it doesn't provide any. Move that logic into the yum
element which provides the install-packages bin.
Change-Id: Id5339fc7ffbef6327fe41f50a3e39ab30b6d0320
disk-image-create processes a DIB_IMAGE_CACHE variable and exports it,
but there are several elements that ignore the value and wrote out
the base location themselves. Use the variable everywhere so that it
will get overridden everywhere.
Change-Id: I00fff354d6c931ad67cf3052d055f0e4604dfdc8
The way redhat-common's extract-image script was creating the base
tarball caused file capabilities to get dropped, which meant that
things like ping in RHEL 7 images was unusable for regular users.
This change adds the necessary options to the tar call to maintain
as many extended attributes as possible.
--acls and --selinux are intentionally omitted, and the selinux
xattrs are filtered out because all of those items cause issues
in our chroot environment. We restore selinux attributes at the
end of the build anyway so that shouldn't be a problem.
bz reference: https://bugzilla.redhat.com/show_bug.cgi?id=1144149
Change-Id: Ibff99ce9bde01bc5ecf95dc3a5d3e2cebe5015b9
EPEL 7 was released September 30. The beta release rpm was removed
from the repo and was replaced with the final release rpm.
Change-Id: I103a2f7abd7563869b9e39ccc1fbb3cfbd9ff6ae
In both of the changed files in this patch, the $DISTRO_NAME usage
is checking against "centos". The valid value for the centos7
element for $DISTRO_NAME is "centos7"[1].
[1] http://git.io/8IvuTw
Change-Id: I0c4a4b2ecf87c7a9283d10071f6feb494a6c716a
A sanity check that cache-url does the right thing when presented
with the appropriate curl return codes. This change brings in the
test code for scripts from tripleo-image-elements, which needs to
be factored out into a place both projects can use it.
I'm stubbing out curl so we don't have a dep on curl in our unit
tests, and because I've seen some strange behavior out of curl in
the gate that caused random failures in this test.
Change-Id: I31e6b06b45415feec7285511d07e65eb78d0d045
The openstack-selinux package contains policy updates for OpenStack.
Updates are accrued in openstack-selinux before they are merged into
the upstream selinux-policy package.
Change-Id: I7c00e5cae17489202fe7aee3e656fca27ab630ba
As some finalise steps can install packages we need to generate the
dpkg manifest after that has occurred.
Change-Id: I2177db2e64d4d9c21deeac7cf017919888a2d524
Per [1], our current root label of cloudimg-root does not work with
XFS because XFS only allows 12 character labels. This change
addresses that by allowing a custom rootfs label to be specified
in the call to disk-image-create. There should be no backwards
compatibility concerns as the default label is unchanged. Any
external elements dealing with the label would need to be updated
to support this new feature, but should continue to work as before
as long as a custom label is not specified.
[1]: https://bugzilla.redhat.com/show_bug.cgi?id=1139584
Change-Id: I596104d1a63b5dc6549e8460a1ae3da00165ef04
Adds a new svc-map element which can be used to install
services based on an in element 'svc-map' YAML config
file format.
This change is intended to decouple elements from DIB
and allow new elements to support multiple distributions
(with various service naming schemes) without having to
constantly maintain DIB's various bin/map-services files.
This patch ensures all of the YAML files are compiled into
a single file without namespaces. The YAML compilation
process occurs during image creation time and errors if
conflicting mappings are found.
Change-Id: Id11433ea342aace71a358936a7ca3151ec11d506
In preparation for enabling Dracut-based ramdisks, this change
factors out functionality that is common to both busybox and Dracut
ramdisks. Said functionality is moved to a ramdisk-base element
which is added as a dependency of the ramdisk element. ramdisk now
only contains the functionality specific to building busybox-based
ramdisks.
bp tripleo-juno-dracut-ramdisks
Change-Id: Iad2907c8be491c88727d87ed5e5a720e5beb66c3
This patch adds a new EPEL element for use with EL-based distros
in accordance with the official docs[1]. This patch also
modifies the rhel7 element to not use the hard-coded repo file.
[1] https://fedoraproject.org/wiki/EPEL
Change-Id: Ib2fa24b2d519837271a0e213a2dc5a012cc31f3c
Update to newer Centos7 images as announced at [1].
Changes included:
* update to GA image
* respect TMP_DIR for image working directory
* use suffix-match rather than prefix-match to build RAW_IMAGE name
to be more robust against weird names
* partition layout work-arounds not needed any-more, all one
partition.
Resulting image booted locally with kvm and virtualbox (after
conversion to vdi)
[1] https://www.redhat.com/archives/rdo-list/2014-August/msg00045.html
Closes-Bug: #1363146
Change-Id: Ie24033468b78587ea87188ee1b843b26895798ff
Adds binaries specified by user to chroot environment
allowing to build images on amd64,i386,armhf,arm64
architectures.
Closes-Bug: #1332458
Change-Id: If6e63a472ee85559b93b5e6b96d9c3ddcf7bcc98
