Diskimage-builder currently writes cloud-init config file which adds a
host entry mapping the hostname and FQDN to 127.0.0.1 into every image
built. This is probably useful for some use cases but not for all, so we
now allow customizing the manage_etc_hosts value via
DIB_CLOUD_INIT_ETC_HOSTS variable and also not writing the config at all
if that variable is explicitly set to an empty string (currently the
default is 'localhost' but in the future the default will be empty
string).
Particular description of the problem this causes in TripleO follows:
We get hosts files like this:
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
127.0.0.1 ov-rl5i5saoc6h-1-hj5tzsbrdv4c-controller-dy6nuyarqy5z.novalocal ov-rl5i5saoc6h-1-hj5tzsbrdv4c-controller-dy6nuyarqy5z
# HEAT_HOSTS_START - Do not edit manually within this section!
192.0.2.17 ov-rl5i5saoc6h-0-wfzcsrqo34p6-controller-m3hy26lhxavl ov-rl5i5saoc6h-0-wfzcsrqo34p6-controller-m3hy26lhxavl.novalocal
192.0.2.15 ov-rl5i5saoc6h-1-hj5tzsbrdv4c-controller-dy6nuyarqy5z ov-rl5i5saoc6h-1-hj5tzsbrdv4c-controller-dy6nuyarqy5z.novalocal
192.0.2.16 ov-rl5i5saoc6h-2-a6v7saxnivm5-controller-7jboskte34r7 ov-rl5i5saoc6h-2-a6v7saxnivm5-controller-7jboskte34r7.novalocal
# HEAT_HOSTS_END
The duplicate hostname/FQDN entry for 127.0.0.1 and 192.0.2.15 confuses
Corosync, which then fails to start a cluster when using hostnames in
the config file instead of IPs.
Change-Id: Ia8582883f737548e2911d3f36a1943e5b236281b
Partial-Bug: #1447497
We support building elements without depending on the base element.
Breaking install-types out into its own element while making base depend
on it so elements can depend in it without base.
Change-Id: I104543d5482c76f60902e9fc32d91e196eeab51a
Turns out that updating packages last causes some pretty
non-intuitive behaviour if you are trying to pin a package
to a specific version. Lets just update the base RPMs first...
subsequent installations should install the most updated version
anyways (unless they are pinned).
Also moves the package-installs script from the 00 step to 01 so
we can do the update first.
Co-Authored-By: Ben Nemec <bnemec@redhat.com>
Change-Id: I962046cc6048e852e6582fbc579f88bb73e23fdd
The listing of *-$INSTALL_TYPE-install files currently uses ls, which
errors out when the glob matches no files, thus using true to not fail
it.
Instead, use find to collect the file list, so there is no need to
ignore the command errors.
Change-Id: Ic6888106858df320a1c90a84f1b9ec74d436b9e6
Not all operating-system elements install cloud-init, but the base
element assumes its existence. Create the directory if it does not
exist.
Change-Id: I4bda8dc5d200825ea0c8163a4e5c44050a45083f
There is a wide variety of tracing options through the various shell
scripts. Some use "set -eux", others explicity set xtrace and others
do nothing. There is a "-x" option to bin/disk-image-create but it
doesn't flow down to the many scripts it calls.
This adds a global integer variable set by disk-image-create
DIB_DEBUG_TRACE. All scripts have a stanza added to detect this and
turn on tracing. Any other tracing methods are rolled into this. So
the standard header is
---
if [ "${DIB_DEBUG_TRACE:-0}" -gt 0 ]; then
set -x
fi
set -eu
set -o pipefail
---
Multiple -x options can be specified to dib-create-image, which
increases the value of DIB_DEBUG_TRACE. If script authors feel their
script should only trace at higher levels, they should modify the
"-gt" value. If they feel it should trace by default, they can modify
the default value also.
Changes to pachset 16 : scripts which currently trace themselves by
default have retained this behaviour with DIB_DEBUG_TRACE defaulting
to "1". This was done by running [1] on patch set 15. See the thread
beginning at [2]
dib-lint is also updated to look for the variable being matched.
[1] https://gist.github.com/ianw/71bbda9e6acc74ccd0fd
[2] http://lists.openstack.org/pipermail/openstack-dev/2014-November/051575.html
Change-Id: I6c5a962260741dcf6f89da9a33b96372a719b7b0
Both ccache and linux-image-generic were being installed by
install-packages rather than using a package-installs.yaml.
Change-Id: Ib54c5b51d605278bb9d046a897be989c17c6fb0f
When set to true this cloud-init setting overwrites the /etc/hosts
file at reboot, breaking stuff in TripleO where controllers rely
on resolving each other by name. We still want cloud-init to add in
there a mapping for the local fqdn, so we switched to 'localhost' [1]
1. http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/doc/examples/cloud-config.txt#L470
Change-Id: I85468dece3b67f39841aa826f9b7ca4bf113027a
Closes-Bug: 1411594
Adds new disk-image-create --install-type option which
can be used to enable alternate install types. The
default install type is 'source'.
The motivation is to eliminate elements like
enable-package-installs which require coupling and also
don't work with elements in multiple element repositories.
This patch does not prevent you from using the previous
DIB_INSTALLTYPE_ variables to customize the install type,
rather it just changes the default so you don't have to
set it so often when using things like packages.
Change-Id: Icee98440fc2251728f2dca30e7c4789a0fd89b93
This reverts commit 4bf38a829a.
This variable is pointless since we can always replace it with
/lib/systemd.
Commit I459f7514ab35082d31607968252a9005fa25de2a will fix the issue
in tripleo-image-elements.
Change-Id: I524badb6836a0d04a5f6e6a5b5d95920fd923ef6
After being deprecated two releases ago, finally remove any reference
for the support of first-boot.d
Change-Id: I08d67404ef48cad61db3b18fb86e970abfa5d2b6
disk-image-create processes a DIB_IMAGE_CACHE variable and exports it,
but there are several elements that ignore the value and wrote out
the base location themselves. Use the variable everywhere so that it
will get overridden everywhere.
Change-Id: I00fff354d6c931ad67cf3052d055f0e4604dfdc8
For package installs we need to be able to fix up permissions in
/var/log at install time, and if we delete all of the files then
those permission fixes get lost. To deal with that, we can just
empty the files instead of deleting them.
Change-Id: I70abb354e4dc726ec7a176694c7c6e9455d89d08
The $DISTRO_NAME var is now set in all of the OS elements
in environment.d. This patch removes the call to
lsb_release and instead uses the $DISTRO_NAME variable.
Change-Id: I7088eb88b6a3611fef5a21a6b62975876549465f
With this change package name to distro mappings used
within the base element are now stored in the top
level pkg-map file.
Additionally, all bin/install-packages commands are updated
to use the -m option so that the base element pkg-map
namespace is used by default.
Change-Id: I3bdbf987c8525f9ef67ca0c263651d6613f29ef0
export $SYSTEMD_SYSTEM_UNIT_DIR to know where .service files should be
moved.
So far those files are moved directly in /usr/lib/systemd/system whereas
Debian use /lib/systemd/system.
We load the value from the systemd pkg-config file.
The patch don't use pkg-config directly for three reasons:
- environment.d files are run before install.d and the other similar
targets, making hard to install an additional dependency
- the .pc file can easily be parsed using awk
- pkg-config package name is not standard across the different
supported distributions
$SYSTEMD_SYSTEM_UNIT_DIR default value is /lib because, as pointed by James
Slagle, /lib is actually a symlink to /usr/lib/ on Fedora and will work
for Debian and Fedora.
See: https://bugs.debian.org/719634
Change-Id: I413a92284e6a79e7fcdde99c5138bc3ce8e85e80
diskimage-builder/elements/base/cleanup.d/99-tidy-logs added
to tidy up (remove) initial logs files on disk images.
Makes for slightly smaller images, and easier times tracing post
boot issues.
Change-Id: Iee98b5d818e080bcd9635482bc29349a71938087
As with the previous similar changes, this is intended to catch
problems as they happen instead of ignoring them and continuing on
to potentially fail later. Setting this on all existing scripts
will allow us to enforce use via Jenkins.
Change-Id: Iad2d490c86dceab148ea9ab08f457c49a5d5352e
Using set -e in all of our scripts will prevent some subtle bugs
from slipping in, and will allow us to enforce use of set -e with
tooling.
This change also adds -u and set -o pipefail in the less complex
scripts where it is unlikely to cause problems. A follow-up change
will enable those options in the complex scripts so that if it
breaks something it can be reverted easily.
Change-Id: I0ad358ccb98da7277a0ee2e9ce8fda98438675eb
The phrase is no longer needed as of August 23, 2000 with Nicaragua's
joining of the Berne Convention.
Additionally, in at least one instance,
elements/cache-url/bin/cache-url, its existence in the file between
Copyright lines is just weird and feels misleading, even though it is
not.
Remove all of the lines, because sanity.
Change-Id: I24fd76c2b4f66b8036010b5079db39ead729abee
Fixes problems found by set -eu and pipefail, including:
-Many unset variables
-Commands that can fail under normal circumstances, which breaks
with set -e. This change swallows those expected errors to allow
our existing error code to handle them.
-The dkms element was not finding Fedora kernel versions correctly.
This may be an issue for other distros too, but since Fedora was
working fine without this functionality I only changed it to print
a warning message rather than failing the build when it happens.
-The ramdisk init script will not be set -eu because if it fails
the result is a kernel panic, which can be tricky to debug.
However, in testing with set -e a few failing commands were found
and have been fixed in this patch.
Change-Id: I44cf98dfc80cfcaec54b88cc83be80a3dbf2cec3
Move whether the package or source install type is enabled out of the
source-repositories element and into the base element.
source-repositories wasn't a great fit for this functionality to begin
with.
Specify the install type for an element via the
DIB_INSTALLTYPE_<install_dir_prefix> environment variable.
Additionally with this approach, any new install type can be added
in an element, without having to update source-repositories or anything
in dib.
You could just write your install type implementation under
nova-foo-install, then define DIB_INSTALLTYPE_nova=foo in your
environment, and the scripts under nova-foo-install would get run during
the image build.
Source installs (any scripts under <install_dir_prefix>-source-install)
is the default install type for all elements.
Change-Id: I9414aca360c41e030e27d3d0c0a52d9d8e13d8b1
dib-init-system is installed in 04-dib-init-system. Before it is
installed, 10-dib-init-system.bash may be sourced so command not found
issue found.
The patch invokes "/tmp/target.d/dib-init-system" directly in
10-dib-init-system.bash.
Change-Id: I9843224497c7dab8392261b26d62b6d79d1795ee
Since we are using bash syntax in some of the element fragments,
we should make sure we use bash for all of them, so that things don't
break on systems where /bin/sh != /bin/bash.
Change-Id: If2f043c57aa4e1492b7f9839213ef6123f683612
dib-init-system script is installed into $PATH. Called
without arguments it will print the name of init system
used to stdout.
Additionally, set DIB_INIT_SYSTEM environment variable to
the init system used.
Tested on ubuntu+upstart, centos+upstart, fedora+systemd,
debian+sysv.
Closes-Bug: #1251610
Change-Id: I29668079091f6060dab66d8259890384d3bbd653
Commit c7d80dd (Cleanup mount points automatically) removed the unmount
of $TMP_MOUNT_PATH/tmp/ccache in run_d_in_target() and moved the
"rm /tmp/ccache" to elements/base/finalise.d/02-remove-ccache. There
are two problems with this:
1) Not unmounting at the end of run_d_in_target() results in tmp/ccache
being bind mounted muliple times on top of itself (three times, if you
just run `disk-image-create base`). It is eventually unmounted, but
somehow the auto unmount code is confused, and tries to unmount it
one more time than it was mounted, which results in an error like
"umount: /tmp/image.THQkZxQa/mnt/tmp/ccache: not mounted".
This doesn't actually break anything, but it's a little messy.
2) "rm /tmp/ccache" in elements/base/finalise.d/02-remove-ccache never
succeeds in removing /tmp/ccache, because that hook is invoked by
run_d_in_target(), *while* /tmp/ccache is mounted.
This present commit solves the above by moving the ccache setup glue out
of img-functions and into the base element's root.d. This has the
following implications:
1) lib/img-functions is a little cleaner.
2) /tmp/ccache is available in the chroot during the root, extra-data,
pre-install, install and post-install stages. It is not available
during block-device, finalise and cleanup stages as it will have been
automatically unmounted by then.
3) /tmp/ccache won't be setup if you're building an image that doesn't
include the base element.
Change-Id: Ief4c0a6f4ec622db6c6f652776215684178d8943
Changing the grub config makes no sense in a build not heading for a vm
and may fail because grub is removed from images not including the vm
element. Forcing textmode for those images would be better done in nova.
Change-Id: I1c5b89e551e62df2463200b1889cb2342498c7dd
The dynamic kernel module system is not available on RHEL, CentOS,
Scientific Linux, or SUSE. Make it part of the distro post-install
rather then base post-install.
Change-Id: Ic2c345bf9f0738dadae611194e263d3a5d424a3e
Manually listing the mount points that need cleanup is not
maintainable and makes it hard to write elements that use bind
mounting to inject resources into the build environment.
Change-Id: I7c9ade444f4ebe42552f8e321f257a7ec0a674ef
In the 52-force-text-mode-console hook, gracefully exit when the
/boot/grub2 directory does not exist on the Fedora 19 cloud image
filesystem. By default the Fedora 19 cloud image is using extlinux to boot
the image instead of GRUB. The decision was taken because GRUB is quite
big (would pull in ~30MB of dependencies) and there's very little benefit
in using it because cloud images don't need to cover all the corner cases.
Change-Id: I52723d4d14c21b1787b4c7e0dd21a09cdbdae0d1
In the 51-grub hook, after the GRUB installation, the script will look
for a GRUB configuration file and in case it's not present one will
be generated using the grub-mkconfig command. The reason why it have
to be done is because the new Fedora 19 cloud images is using extlinux
by default.
Change-Id: I80b15b3122698d98ac4d47dc06faf5909a90ab00
Many python extensions are built from source each time an image is
built. Repeating these is wasteful, so ccache is employed to eliminate
that waste. A cache dir outside the chroot is used to speed up
subsequent image builds.
Change-Id: Ib73563ddbe5f3be7454bfc54ab91cedb559a1304
