When the cache tarball gets fail the output doesn't indicate
what url is failing. Adding '-x' displays the urls attempted.
Change-Id: I71b2dbfd712e72c591410ca9b68edc76d493ad6e
As with the previous similar changes, this is intended to catch
problems as they happen instead of ignoring them and continuing on
to potentially fail later. Setting this on all existing scripts
will allow us to enforce use via Jenkins.
Change-Id: Iad2d490c86dceab148ea9ab08f457c49a5d5352e
Updates the fedora element so that DIB_DISTRIBUTION_MIRROR can
be used to configure a custom Yum mirror on Fedora. This
implementation replaces the baseurl in each of the default
Fedora Yum .repo files so that a custom mirror URL/path
can be supplied. The URL should point to the directory
where the updates and release directories exist. Example:
http://download.fedoraproject.org/pub/fedora/linux
As part of this change we also remove any existing
metalink settings in the .repo files to cover the case
where an end user runs their own createrepo command
to create the mirror... which means your metadata won't match that
of the normal Fedora mirrors and your baseurl won't get used.
Change-Id: I1a49a2941b1cc489e91a72b87430393cd32e865d
DIB's map-services is currently used in TIE's os-svc-config
binaries to support service name lookups. Many of the systemd
specific functions require bin/map-services to exist.
This change adds bin/map-services files that are noops
to both the debian and ubuntu elements so that as those
distros move towards using systemd they can make use
of the functions we already have in place.
Change-Id: If8d61f3858b11de86c3292e840d033e5e3cecedb
If scripts are not set -e then errors can be ignored, causing more
confusing failures later.
Also adds an exclusion comment to the ramdisk init script since we
don't want that to exit on failure.
Change-Id: Idf43993bd10b1ef16c1d3b0d9df8d0ad94c46458
Adds a new element, package-installs, that provides an interface for
declarative package installs and uninstalls. Packages to install can be
added to an install.d/package-installs-<element-name> file. The set of
packages listed across such files are installed in a single transaction
at the beginning of install.d.
Prefacing the package name with a "-" indicates that the package should
be uninstalled at the end of the install.d phase. Again, the full set of
uninstalls are done in a single transaction.
An element providing a package-installs file should add package-installs
to its element-deps file.
Change-Id: I5b540388eff1079c8dee933b869463371481152b
This commit fixes the if/else block around Debian stable/unstable which
was badly merged during the last rebase.
Change-Id: Ia2eb806da3d7c8d28afffb026f35600c7125d8d7
This makes it easier to work with temporary files - less traps
scattered around higher layer code.
Change-Id: I2fdd93115a7b0d289c2e28f8c133d4059de75b87
This patch adds a debian-systemd element to get systemd support with
Debian.
A backport of systemd is used on Debian Wheezy.
Change-Id: I095c81b2c32844fccf940c97b79054923c25b8c2
Upstart cannot be installed by debootstrap on Wheezy because of a
conflict with sysvinit. sysvinit is flagged a being “essential” and
apt-get will refuse to remove it.
See: https://bugs.debian.org/668001
This patch uses another strategy. We install a standard sysvinit
Debian chroot with debootstrap and during a second step, we
replace sysvinit with the wanted init system.
Change-Id: I081c81b2c32874fccf940c97b79054923c25b8c2
Cloud images expect the primary interface to run dhclient in order
to come up and acquire an IP address. Although we have technology
to manage without this for the undercloud and so on, the seed
still relies on this machinery in order to bring up its first
interface.
Change-Id: I079a679779de760ee9ca6f93cbb22e69a65e03af
Using set -e in all of our scripts will prevent some subtle bugs
from slipping in, and will allow us to enforce use of set -e with
tooling.
This change also adds -u and set -o pipefail in the less complex
scripts where it is unlikely to cause problems. A follow-up change
will enable those options in the complex scripts so that if it
breaks something it can be reverted easily.
Change-Id: I0ad358ccb98da7277a0ee2e9ce8fda98438675eb
In the mailing list discussion, everyone seemed to agree that we
should just be explicit that these are bash scripts since that's
where we're testing them and there's no good way to ensure they
work on any other shells right now.
Change-Id: I86c41d2cddf6e91e7deeb4c2ab51635c80a1ce0c
The phrase is no longer needed as of August 23, 2000 with Nicaragua's
joining of the Berne Convention.
Additionally, in at least one instance,
elements/cache-url/bin/cache-url, its existence in the file between
Copyright lines is just weird and feels misleading, even though it is
not.
Remove all of the lines, because sanity.
Change-Id: I24fd76c2b4f66b8036010b5079db39ead729abee
Trusty ramdisks do not boot, failing to exec init. This reverts
the default DIB_RELEASE back to Saucy until Trusty ramdisks have
been properly tested.
Change-Id: I5ac81822b78805275268cecc021a74a62c22ef58
Nodepool needs to be able to install kernel headers, which means we
need a mapping for redhat systems here.
Change-Id: Ic83d642b5ebe85b042cf7b23b295be74bbe185fa
Provide package name mappings for openjdk-7-jre-headless:
RHEL/Fedora: java-1.7.0-openjdk-headless
SUSE: java-1_7_0-openjdk-headless
See https://review.openstack.org/#/c/86314
Change-Id: I3a351d0f9854f6f320ba145a71ffc66fd02c231c
Unlike the redhat-common and opensuse map-packages maps, the RHEL one
appears to be unsorted. Fix that.
Change-Id: I75c5d2694525731660c2a3b5b58ee696c61923ba
This patch adds support for sysv init to the dhcp-all-interfaces element.
Specifically, it adds a sysv init script and the relevant install/config steps.
Tested with Debian jessie/testing.
Change-Id: I4ae1d5365bafda188cb65ea1fb72d8ffe2408910
I3166951c6fbb2faec53ffc339edd2fd0c3512300 recently landed without two
fixes that I suggested, namely using rm -f rather than testing for
file existance, and a comment fix.
Change-Id: I68e187e20802f3482d4c524a6c435fec7526178e
This change adds extra tests to the
elements/manifests/cleanup.d/01-copy-manifests-dir script to prevent it
from exiting with an error due to non-existant files.
The premature exit condition only seems to happen when an image is being
created without including the "base" element.
Changes
* adding file detection tests for dib_arguments and dib_environment
files
Closes-Bug: #1308224
Change-Id: Iaa8bafb6765241408b9f02d4896913d15f2f401d
This is dependent upon change in tripleo-incubator
I1ff4bb98c99dfe87ccc4fb19767b93e27707d3a7 and concludes the
renaming of deploy to deploy-baremetal
The rename was performed so that each element had a more descriptive
name, and so that each element could accurately set an element-provides
of "deploy", given that the elements enable tripleo deploys via the
ramdisk image.
Change-Id: Ic2ca5f95b8895fa0d6d7314cab9de30a7204541c
Use "deploy" to denote that an element provides the deploy
capability for a ramdisk for tripleo installation purposes.
This will be a 3-part change to enable renaming deploy to
deploy-baremetal so that the generic "deploy" capability can
be used for the provides file, as a dependant change will have to
be made in tripleo-incubator.
The tripleo-incubator change is
I1ff4bb98c99dfe87ccc4fb19767b93e27707d3a7 and that swaps to using
deploy-baremetal in place of deploy for ramdisk creation when using
baremetal.
Change-Id: Ibb4026e8fc8ba6378061461c4796a91ab2fa991f
/etc/lsb-release doesn't always exist in cloud images and doesn't
serve any significant purpose in this element so removing the
dependency entirely.
Change-Id: I4d972074e6e10fea09a875cbc4ed4e60248d46d9
DIB_OFFLINE has wide-ranging effect. This patch adds a tunable that
only turns on the tarball-caching behaviour specifically for the
debian element.
Change-Id: Idd646d7909df6907afc0eeeec1129c3f1a1f92bc
Moves the code that removes default cloud interfaces
into the dhcp-all-interfaces element. Having it live in
stable-interface-names won't work right now because we
need this for virtualized testing.
Change-Id: I3166951c6fbb2faec53ffc339edd2fd0c3512300
“debian” element now source the ${TARGET_ROOT}/.extra_settings file. We
can use this file to adjust its behavior and specifiy what init system
should be used.
Change-Id: Id57ce879341184a57851d2578cf70978f409a48e
Fixes problems found by set -eu and pipefail, including:
-Many unset variables
-Commands that can fail under normal circumstances, which breaks
with set -e. This change swallows those expected errors to allow
our existing error code to handle them.
-The dkms element was not finding Fedora kernel versions correctly.
This may be an issue for other distros too, but since Fedora was
working fine without this functionality I only changed it to print
a warning message rather than failing the build when it happens.
-The ramdisk init script will not be set -eu because if it fails
the result is a kernel panic, which can be tricky to debug.
However, in testing with set -e a few failing commands were found
and have been fixed in this patch.
Change-Id: I44cf98dfc80cfcaec54b88cc83be80a3dbf2cec3
Mariadb package in RDO repository is mariadb-galera-server, but
this name is mapped to MariaDB-Galera-server because of
mariadb installed from mariadb.org.
To avoid mapping conflicts mariadb-rdo uses virtual package name
mariadb-rdo-galera-server which is mapped to mariadb-galera-server.
Change-Id: Idf77d91876baeb81d89eb688808b892ece2bf19a
This was supposed to have been moved to the redhat-common element,
and having it here too is both confusing and may break builds
depending on which version gets used.
Change-Id: Icae5cc633e9e49a9ee63bd9dfc94657c6795c6f1
Use "operating-system" to denote that an element provides that to the
image being created. This can then be used to specify an abstract
dependency in other elements to assist in build-time image verification.
Change-Id: Ic125803da65e0f8f8c20ef90b08815f4f7d09762
* Create a standard element to copy manifests out of the image and
into the build area.
* Ensure all current manifests are copied into a standard manifests
directory
Change-Id: I37eff5a13a14564e1adc33eb4f0144d571267617
The version of depmod in busybox does not work if two versions of
the same kernel version are available, as search paths are not
honoured correctly. However, we don't need to actually call
depmod as it's done during image building anyway.
Change-Id: Idd3cd5854e7c86e7ca1f2b0e50758d7db79ab6e7
Co-Author: matthew.gilliard@hp.com
Closes-bug: #1302856
In the bash script "99-override-default-apt-conf" 'set -u'
is being used to check for uninitialised variables.
This resulted in generic 'unbound variable' messages rather
than the intended error messages.
Tweaked the check for DIB_APT_CONF to work if the variable is unbound;
so that we get the intended error message rather than the `unbound
variable` message.
Change-Id: Ib8e287060f32312ab80dd7f44e2792ecca48edbb
Adds an environment variable DIB_ADD_APT_KEYS pointing to a folder of
apt repo GPG keys. Each key will be verified and installed with apt-key
to the apt keyring.
Change-Id: I4b15347a76aa3241e741931bb94f17598641ae42
The ${TARGET_ROOT} chroot is owned by the root user.
Without this change, the build will fail for a standard user
with a “Permission denied” error.
Change-Id: I6eb87d3a14292da5620bdfd623ea7ef846fe2af6
There is a known issue in legacy grub. When using splashimage with
serial console, Legacy grub wont start properly. This commit is to
ensure that when serial console is used, legacy grub should not
contain splashimage. It also adds serial support to legacy grub
configuration file.
Change-Id: I2007916998dfe7c66ef51bf8c57c1698dc0fcb75
When the repository references are sha1s the fetch command used will
not work. This will ensure that all of the objects are still fetched,
and all of the local cache references for branches and tags will be
updated rather than just the specified reference in use.
Change-Id: Iad2554b3087a5da37dac6e9f6a72432d49fce83d
Closes-Bug: #1301435
--target has been introduced in grub2. This patch ensure we use
the minimal version before adding the --target XXX flag.
Debian Wheezy and Precise share the same limitation regarding the
root device.
Change-Id: I5b3ad42b9fb536d13fb65d84339bc15bc23d99a3
The grub install scripts on Fedora have hardcoded checks for
/sys/firmware/efi and if the directory exists, it uses efi
specific instruction like initrdefi, linuxefi etc. vm element
should fix it once the configuration is generated.
Closes-bug: #1231946
Change-Id: I888304cb9167b8e074b04ca6c5b2f9143b70d37d
This was calling it for all interfaces previously, making the fixes
we'd put in place be ineffective.
Change-Id: I16bf81225a942e3b19a7e745d949e6ddea2fd50e
When high VM churn is encountered dhcp-all-interfaces got tangled up
and leaked instances until upstart run out of filedescriptors.
This happened because we were locking on every invocation, and the
lock logic was disabling the single-interface code - so we always
probed every interface every time - and this was serialized across
every instance of dhcp-all-interfaces.
Change-Id: I4ef1fb83d978cc99d4cb9cc2666bde302c6db7d1
To create an auditable log of packages installed in an image use
dpkg-query to create a json data file with the package names, version
and install status.
Change-Id: I59018221edcb944a876ad47b402538c63c426bc7
The cloud-init package on Debian Linux overwrites the system
hostname to "(None)" unless a hostname is specified in one
of the /etc/cloud/cloud.cfg.d/* files. This change applies
a default hostname of "debian" using cloud-init.
Change-Id: I9b411df44cfa05f899e708900f95719c0bcd6267
Corrects an issue in the config_exists function that could
cause duplicate interfaces to be configured on Ubuntu.
Previously we used 'return ifquery' directly which caused
a silent 'numberic argument required' error to go unnoticed.
This would also return 1 meaning a new interface would get
configured.
The new logic should handle things properly.
Change-Id: I625225e15113d7e184e3bcb5054df1616dec008a
Closes-bug: #1298430
Virtual baremetal instances have their NICs show up as ens4
which doesn't play nicely as a default in devtest at
the moment.
Ideally we'd want it to be em1 like real baremetal
(which is why we put this in the baremetal element
to begin with). Turns out the baremetal element is
required in devtest so we can extract the ramdisk and kernel
so lets leave it alone for now and move this elsewhere
until we get a better solution for fake 'baremetal'
testing.
Closes-bug: #1298152
Change-Id: Ia71e1d32b93db0c4c844a6dc1ebcd04ab0c13f05
Currently the dpkg element fakes start-stop-daemon, initctl and
invoke-rc.d to stop dpkg from starting a bunch of daemons in the
chroot. This is problematic when packages use service, start, stop or
restart commands.
This patch uses a policy-rc.d instead of faking invoke-rc.d to achieve
the same thing. This approach now aligns exactly with debootstrap.
Without this patch DIB runs on some debian distributions fail
when trying to umount the chroot loop device as there are daemons
running.
The log will now show "invoke-rc.d: policy-rc.d denied execution of
start." instead of "Warning: Fake invoke-rc.s called, doing nothing."
Change-Id: I6db192127aca19b5b7915179b781f5192078bfc7
Related-Bug: #1211564
cloud-init packages are not available in the Debian Wheezy repository.
However, they are available in wheezy-backports.
With this change, the cloud-init packages are installed from
wheezy-backports if DIB_RELEASE is wheezy.
Change-Id: I3daa3b0ba185ba3f7c1bf6b8f9b1f8c167fcb340
Those parameters allow to access the console for VMs using
extlinux (CentOS 6 for example). They are already added to
Fedora images.
Change-Id: I035170bd101062e004fb3782b5f8e5a7944a4f03
Closes-Bug: 1296899
pypi-mirror creates a separate mirror index for wheels (one per OS
that mirrors are built on). To be able to use it one then needs to be
able to export multiple mirrors for inclusion in pip.conf. As a drive
by I made it possible to disable the use of the pypi.python.org index
without using --offline (as --offline has larger impact).
Change-Id: I3e85a8069b18cafd7eae4cd0591821acc3b5a739
Remove the correct ifcfg-eth0 file on Fedora.
This will require an associated incubator change as well
to ensure the undercloud/overcloud now use em1.
Change-Id: I4b4bb7b6a25b058eebe889fd8ce1a403aaf193fd
libmariadb-dev is named mariadb-devel on Fedora. mariadb-devel
package is needed for upcoming mariadb element.
Change-Id: Ib62071a964a4449c4e32ef1646ab7a0fb0f5f7b7
Update the systemd serial port element implementation
so that it uses udev rules. Much nicer than
hard coding ttyS0 and ttyS1.
The udev rule only adds real serial device types
(i.e. == 4). See:
define TTY_DRIVER_TYPE_SERIAL 0x0003
from include/linux/tty_driver.h.
Change-Id: I5d7de4815fd09c01693b7da4b8949f22fee20745
Updates the dhcp-all-interfaces element to fix a race
with the recent udev rules implementation on Fedora.
With the new approach we make the udev rule want (require
to startup) a generic dhcp-interface@.service template which
can be started individually for each interface that is
discovered.
The dhcp-interface@.service is setup such that it:
1) It calls dhcp-all-interfaces <iface> directly with
a pre-exec script. This creates the ifcfg file right
before we need it but avoids the case where network.service
might get greedy and try to start it itself.
2) Only runs if the ifcfg script doesn't already exist. This
is important because we only need to bootstrap the DHCP configs...
Once they exist the network.service will take care of starting them
on reboots, upgrades, etc.
3) On initial boot ensure that the initial DHCP interfaces come
up after network.service. Since we really only want
dhcp-all-interfaces to help bootstrap that haven't already
been configured this seems reasonable.
4) We also try to ensure that cloud-init
comes up after the DHCP interfaces. Cloud init has a decently
long timeout that this wasn't a functional problem but it keeps
log file spew down.
Change-Id: I71b026f027182aad49c3435bb903e5e38e524685
Closes-bug: #1294803
This is being added as a separate element because RHEL 7 is too
different from either RHEL 6 or Fedora to share all of the
configuration they do. It uses the redhat-common element for
pieces that are duplicated.
Some current limitations of the element:
-RHEL 7 is currently still in beta, so final image locations are
not available. This currently defaults to the RHEL 6.5 image,
but that will need to be overridden using the DIB_* env vars.
-There are additional tripleo-image-elements changes that will be
needed to be able to use this for building tripleo images.
-Open vSwitch packages are not currently available in the official
RHEL 7 repos.
-RHEL does not support qemu without kvm acceleration, so at this
time it cannot be used as a compute node for virtualized testing.
Change-Id: I705fd475303576525a27ce6801c673b5721792c9
Add an element intended for use in both Fedora and RHEL. This
allows them to share install steps that are common to both.
Change-Id: Ie4e820a7b777b8701514351b1f802cfe57c3812e
Refactors dhcp-all-interfaces.sh so that if an optional
INTERFACE argument (the first argument) is passed to the script
it only inspects that single interface. If no argument is
passed then the previous default behaviour is used which
causes all interfaces to be inspected.
To avoid a collision with the previous $1 we move to using
$FLOCKED for the exec flock command which runs on distributions
using ENI.
Also sets PATH so that the commands within the script
can all be found if it isn't set properly (/sbin/ip, /bin/cat, etc.)
This is a move towards using udev rules to add these types
of interfaces automatically.
Change-Id: I3ec8fd2cc2071bfc6943c744ca619e31b71146fc
Move whether the package or source install type is enabled out of the
source-repositories element and into the base element.
source-repositories wasn't a great fit for this functionality to begin
with.
Specify the install type for an element via the
DIB_INSTALLTYPE_<install_dir_prefix> environment variable.
Additionally with this approach, any new install type can be added
in an element, without having to update source-repositories or anything
in dib.
You could just write your install type implementation under
nova-foo-install, then define DIB_INSTALLTYPE_nova=foo in your
environment, and the scripts under nova-foo-install would get run during
the image build.
Source installs (any scripts under <install_dir_prefix>-source-install)
is the default install type for all elements.
Change-Id: I9414aca360c41e030e27d3d0c0a52d9d8e13d8b1
Since file to be fetched from the root tftp directory it shouldn't
contain any hardcoded path. Additionaly it eliminates bearing with
map-file containing regular expressions for tftp path replacement.
Change-Id: Iee1672834f735d7dfefce43000540522e1d053b9
Closes-Bug: #1291533
Some of the tripleo-image-elements were failing for RHEL due to lack of
build essential, python-dev and libz-dev.
This commit should fix those failure.
Change-Id: I080f9601faedb3259deada5cfbda7d00d3f13eb9
By default most cloud images have disabled the
udev rules which give us stable network interface names. While
this is a reasonable default for something like EC2 (the target
for these images) this is very bad behavour on bare metal
and can cause a "musical NICs" situation to occur where
on reboot the NIC names get swapped.
In this commit we re-enable stable interface naming by
removing the dev nulled rules softlink and installing
the biosdevname package. Once this is done interface
names should again show up as em1, em2 instead of
the old eth0, eth1.
We also remove some the common, statically configured
eth0 config files on some distributions. Equivalent
files (if needed) may be auto-generated by using
the dhcp-all-interfaces element.
Change-Id: Idd67977342719e2f295e2fa2c0ed6cfa8602171a
As git repositories are copied into the image make a note of their
details and add them to a manifest on the build system.
This allows the code used in building an image to be found after
the image build is completed without needing to open the image up
for inspection.
Change-Id: I7dbe9e163ad38a418cf2869a81e720de2c27dfb1
In 8b2325118f we added a line
so that 98-source-repositories tries to return from the top
level script (not from within a function).
Also, remove the return 1 within the function as well
because we just want to continue in these cases.
Change-Id: I66eddc12208b278594a0a8d8676c38d72045ca75
NFS client utilities for Debian are found in nfs-common, and the
Fedora equivalent is found in nfs-utils. The NFS server components
are already accounted for in the nfs-kernel-server mapping.
Change-Id: I2a43208f54b7fb625c239ce22231c43e194a97ed
Looks like shells only support alphanumeric characters and "_"'s in
environment variables. We were substituting "-" characters but in
order to be able to set overrides in projects with names containing
other characters we need to substitute more, I'm looking at you
oslo.config.
Change-Id: I3e2b1b0bc5871e4ec4ffd8117906cd077aa2cb0d
Co-authored-by: James Polley <jp@jamezpolley.com>
This will allow us to add environment variables to source-repository
scripts. Useful for things like arch-specific download urls.
Change-Id: I4d046825a317dc88db3d7673ad261cedacd1a104
This got removed accidentally in a recent consolidation of
install-packages. Adding it back so that -d support works again.
Closes-bug: #1287142
Change-Id: If17e613bc9bc0f808517edfa1506858d3b9dce54
The RHEL 6 Guest Image has been updated with a new release that
addresses a number of bugs. The diskimage-builder element that
references this file needed to be updated to reference the correct
image name. In addition, some notes to help users download the
image from RHN have been added to the README.
Change-Id: I3c9e64a1887afd9c0b370d70eff4b7f9598cebd1
Closes-bug: launchpad-bug-1286568
This patch alphabetizes the openSUSE packages dictionary in
map-packages. The openstack mappings were left
at the bottom as it looked deliberate.
Change-Id: I3661fba4d4ed68c836893e21fb4a8728c691af20
This patch alphabetizes the Fedora packages dictionary in
map-packages. The openstack mappings were left
at the bottom as it looked deliberate.
Change-Id: I4874c19f2a6826062874698e8f61bc6e2685ab19
Extlinux bootloader configuration in "vm" element requires vmlinuz and
initrd.img files in /boot/ with "generic" suffix. When these don't exist,
bootloader configuration should fall back to using whatever does exist.
Change-Id: I79d426e8f008e9f23db899f0b8f8cf2f23643e28
Testing showed that show-ref did not accurately determine whether a
git repository contained the given reference. Testing with name-rev
showed that it did.
Change the HAS_REF determination appropriately to avoid unnecessary
cache updates when using fixed references.
Change-Id: Iece213348516358af84417807f8dd36e959139eb
Reference to ${GRUB_OPTS:""} in 'vm' element causes syntax error
if GRUB_OPTS is set. Fix by changing to ${GRUB_OPTS:-""}.
Change-Id: Iae3ee8b7c4064c92d0cfc1b5a8caa22e53d6d112
We were applying proxy settings after apt-get install runs are done
in the Ubuntu element, which made it ineffective. Secondly, we were
not applying no_proxy settings to apt, which caused it to fail in
corporate settings. More explicit control may be facilitated in future
but local-config is a DTRT tool, and so we should help it DTRT.
I think I've correctly setup Zypper for this, and I have no idea
how to make the same improvement for Yum (and also Yum currently
doesn't set HTTPS proxies, which may lead to similar reports there).
Change-Id: Idbb9781d64d44b717ea22a25429383fa45d84148
Debian builds fail due to missing rsync in the baseline set
of packages that debootstrap defaults to. Rsync is used by a
variety of tripleo-image-elements and should be included in
disk images by default.
Change-Id: I8d83a53dfab279075e11f9825f15ee960395dd17
Add the DIB_GITREPOBASE variable to the source-repositories element
to enable global changing of git repository location.
Change-Id: Icfac5e103fce6e5a3f1a2db094144a0ed89fcc33
* Rename the cache directory for sources
Make the cache directory name match that of the element that creates
it. This makes it easier to find and more logically implies what
element created the cache directory.
* Add REPONAME to repository cache directory
For better usability add the reponame to the directory created
to cache it.
* Add migration code
Move pre-existing cache directories of the old name to the new name.
Change-Id: Ic62f19acb1746888ccbdf95eebf6e653f3e66357
The current method to rewrite the mirror URL in the ubuntu element will
only rewrite archive.ubuntu.com, which leaves the security URL alone.
This is probably not intended for people who are creating images behind
firewalls.
Change-Id: I54a867e0e9d5a7c94324a5c00ff1a83b4e6e4524
All of the yum-based install-packages scripts are doing essentially
the same thing, so let's use one script for all of the elements
that depend on the yum element.
Change-Id: I49e8c9b44e41bcf4cb9fa820e8a9179754694a97
We need to update vim-minimal, otherwise when rabbitmq-server gets
installed and pulls in vim-common, we run into:
https://bugzilla.redhat.com/show_bug.cgi?id=1066983
Change-Id: I910cd6c5c5fcfc6f6780b86029b1b4e660dd8d05
The pypi element is cool, but some folk have local network mirrors
which we should permit them to use.
Change-Id: Ie840ad1184e72b0e01966eee0298cfd6511b6c19
The ubuntu element supports simple replacement of sources.list by using
the DIB_DISTRIBUTION_MIRROR environment variable. Drag the debian
element into line by supporting both DIB_DEBIAN_MIRROR and the new shiny
DIB_DISTRIBUTION_MIRROR.
Change-Id: I9d5f8d1e3251965b34e55929182aa601a524fe8f
We can now support file:// URIs in source-repositories files and other
places, by ignoring curl's inability to determine an HTTP status code
for them.
Change-Id: Ie226ce05ecb912b23cedc73fd7ba53c0b9c0fe35
Updates the dhcp-all-interfaces element so that
we add new DHCP interfaces using a udev rule
instead of a systemd service.
The previous approach (using the systemd service)
was proving to be problematic when used with
the new Fedora constistent network naming
convention rules (em1, em2, etc.). We aren't using
these device names as a default yet... but we will
need to soon and when we do udev rules seem to be the
best way to avoid racy behaviour when calling
dhcp-all-interfaces.sh from the systemd service.
Change-Id: I87e0aed1f34718e896bd414388886a1f1073b0c0
Refactors dhcp-all-interfaces.sh so that if an optional
INTERFACE argument (the first argument) is passed to the script
it only inspects that single interface. If no argument is
passed then the previous default behaviour use used which
causes all interfaces to be inspected.
Also sets PATH so that the commands within the script
can all be found if it isn't set properly (/sbin/ip, /bin/cat, etc.)
This is a move towards using udev rules to add these types
of interfaces automatically.
Change-Id: Ia482c1d3ddce0f0d8d77f9bc3ac76d6924640715
Don't assume environment.d is always under /tmp/in_target.d. This makes
dib-run-parts more flexible, and less specific to diskimage-builder. For
instance, dib-run-parts is already used by os-refresh-config, but
/tmp/in_target.d/environment.d is never going to exist when it's called
by os-refresh-config.
This is useful b/c elements can now install os-refresh-config scripts
under /opt/stack/os-config-refresh/environment.d to influence how
further scripts are run. Specifically, the OpenSuse packages prepend the
OpenStack user accounts with "openstack-". Instead of checking for which
users exists with an if/else in all the os-refresh-config scripts that
need to do user related things, we can just update the scripts to use
$NOVA_USER, etc. The OpenSuse element can then install a file under
/opt/stack/os-config-refresh/environment.d to set NOVA_USER to
openstack-nova. We could have files that declare defaults as well within
each specific element, or just set a default variable at the top of each
os-refresh-config script.
Change-Id: Iadbfad995da657e2965fd55fc4ba3a88138b4cfc
Updates dhcp-all-interfaces so that we avoid using
ifquery on Fedora to determine if an interface exists.
Adds a new config_exists function which simply looks for
the ifcfg config file on distro's that use netscripts.
Change-Id: Ie55524b83820fe4fab28cc9d3f7e08ca2e42c182
With https://review.openstack.org/#/c/70721 installing
the dhcp-agent subpackage was introduced, which is needed
for openSUSE. It seems on Fedora this is part of the main
package, so we need to map it back to that one.
Change-Id: I80f8412d291b3571d220fd26cfece86a03479440
Just adding an element shouldn't make a disk image build crash, even if
we're not using the element for its intended purpose.
Change-Id: I2ec91ce4975b3b0deee7c85bb223f854c694553f
Adds the ability to set $DIB_LOCAL_IMAGE to use as the base cloud image
for a Fedora image build. There are many repetitve tasks that are done
every image build. With this change you can build an image with the
fedora and vm element and then reuse the resulting image as input to
future image builds. This greatly reduces future image build times.
For instance 99-up-to-date is already taking almost 2 minutes (even with
rpm downloads already cached) for Fedora 20, and is only going to keep
taking longer. By having a local up to date Fedora cloud image, this
time can be saved on each image build.
There is one minor change to support this in that /tmp/grub needs to get
cleaned up at the end of image builds so that the image can be reused.
Plus, there is no reason for it to stick around anyway. (didn't think
this was worth a seperate commit).
Change-Id: Ic74d138da922ecc99c38c27f105170d90009a84a
Otherwise some dependend libraries might be missing in the ramdisk. E.g.
on SUSE bash is dynamically linked against libreadline.
Change-Id: I8b36bee6aa5a1e1da419e748e8bba046c29f3ae2
/usr/bin/env is more portable (/bin/env does not exist on SUSE).
Also address minor nit on review about python 3.x compatibility.
Change-Id: I6bc1a579bb43664f4d2be35a96f45189a6df0e27
Rename generate-interfaces-file.sh to dhcp-all-interfaces.sh
so that it is easier to find/remember in the $PATH on a deployed
image.
Also, on some distros the script actually generates many files
so the previous name was a bit confusing.
Change-Id: I7152fa7c28e8ade251311da2cd5f75972423b66c
Updates the generate-interfaces-file.sh script in
the dhcp-all-interfaces element so that we only
add interfaces that have real MAC addresses.
The generate-interfaces-file.sh script is run early enough
in the boot process (before OVS is initialized) that this
isn't usually a problem unless you execute it manually by hand
after booting. Then you'll end up with network/DHCP
configs for all of your OVS bridges, etc. This
patch avoids configuring all of the virtual interfaces
which have generated MAC addresses.
Change-Id: I7a705084aa5b11305ac0ec5ca37fd2e87a2ae8b7
Closes-bug: 1239479
map-services is used by os-svc-restart from Change-Id:
I15eb2a5db4b0a08e1fb40bda640cd8f224939a92
map-services is not only distro specific, it also needs to be able to
handle different service names for a given distro based on source vs
package installs, since the service names may be different. This is done
via a simple check if the requested service is defined, if it is, we
print the requested name, otherwise, we print the mapped name.
Currently only implemented for Fedora. Support for other distros is
easily implemented just like map-packages.
Change-Id: Ie7b2dcfa7cabd887d5c212df110d90f9d00a7f65
The SELinux relabel of the filesystem is taking almost 2 minutes and
isn't needed unless you actually plan to run with SELinux enforcing.
Plus, it appears to "leak" out of the chroot, referencing filesystems on
partitions that aren't even mounted in the chroot.
Note you just can't use getenforce or selinuxenabled here to get the
state of SELinux because those commands are not accurate inside a
chroot.
TBH, a downside of this is that if someone goes to try to enable SELinux
in an image where it was built with it not enabled, the file contexts
are going to be wrong. So they'd need to relabel themselves at that
point. However, this saves me quite a bit of time during image builds,
so I thought I'd submit to get other folks opinion on it.
Change-Id: I2132060d573fc93cf974f3560fdc651ff8ba38b4
If creating an image of a disto without grub2 extlinux is instead used,
extlinux was being configured to mount the root filesystem as
read/write which in turn causes e2fsck to fail to test the filesystem,
on RHEL 6.5 this causes the boot process to fail.
Change-Id: I07859e0df9a2bc1ddd8c5f2ed4509ba14312e140
Delete stale symlinks created by current openSUSE 13.1
for kpartx loopback partitions. Those break bootloader
installation otherwise.
Change-Id: I13a7387e341b4426b459610fd1d3c49ce5730f46
openSUSE's ccache package doesn't include symlinks to enable ccache
for gcc by default, so we set them up manually.
Change-Id: I79dbe814ff2dfad8158fb1f06d68cc07eae19a41
A Red Hat Satellite URL may be provided as the source for
Red Hat packages. A Satellite key may be provided for a pre-
defined set of RHN channels to be automatically applied. Support
for installation of a Satellite certificate also provided.
Change-Id: Iae5b97d6f4732a28efa08ab778a56c2f9b6b6bd9
When registering to RHN or RHSM a list of space-separated
channels or repositories may be provided. This allows control
over which packages are available to the system during build.
Note, the optional channel or repository is added by default
since diskimage-builder requires access to its packages. This
list is for additional channels or repos.
Change-Id: I9cf90d47ab1d09f5491f574132410438f9e294e3
This provides basic support to register to the Red Hat Network
(RHN Classic) for packages. The image is unregistered from RHN
before completing image build. The rhnsd service is prevented
from starting since it is not needed for packagage install.
Change-Id: I2dc124cdc6ae59a63fd94d82d77108acccc3f668
Red Hat periodically updates the qcow2 guest image available
for download. This sets the default image name to the latest.
Change-Id: Iba3075bbee3b41918d5cd3da9721fcbf98ff3bcd
Rather than using a script to mount the image using nbd to extract the
kernel and ramdisk, make a new element called baremetal, which contains
a cleanup.d script that will copy them out to <image name>.{vmlinuz,initrd}.
Closes-Bug: 1224669
Change-Id: I8f3569aa12148d18b1c8242b6fbbd8857894b26f
Add a script to ubuntu's pre-install.d that will change
archive.ubuntu.com to a mirror specified in $DIB_DISTRIBUTION_MIRROR.
Change-Id: Idfd5a40ec8a1912a5de5dcdcaf9795946b0b53b8
Update the default version of Fedora to 20 for the cloud image. A few of
us have run through devtest using Fedora 20 and it's working.
To continue to use 19, just specify DIB_RELEASE=19 for your image
builds.
Change-Id: I3d48272dedcd424747db5485ae0d8e9953f04191
We need to use the eval command together with curl so that the
$TOKEN_HEADER variable is seems by curl as two arguments: "-H" and
"'X-Auth-Token: ...". Without the eval bash would interpret the variable
as only one argument "-H 'X-Auth-Token: ...'" making the curl command
to not understand that parameter and fail to pass the auth_token to the
Ironic API.
Change-Id: I4dcfc323d6ab9b7fa207328386ef65a146a93617
The source-repositories element will now symlink the install scripts
corresponding to the set install type for an element into the install.d
directory.
Different install types are implemented by elements by writing scripts to do
the install type under install.d/<element-name>-<install-type>-install
directories.
For example, the nova element would provide:
nova/install.d/nova-package-install/74-nova
nova/install.d/nova-source-install/74-nova
source-repositories will create the following symlink for the package
install type:
$TMP_HOOKS_PATH/install.d/74-nova -> nova-package-install/74-nova
Or, for the source install type:
$TMP_HOOKS_PATH/install.d/74-nova -> nova-source-install/74-nova
Change-Id: I1bfaf39e5a98b2af904fbc6d674dbada30b27ed9
dib-run-parts should dereference symlinks, and if they point to an
executable file, execute that file as a script. This can be accomplished
by using the xtype predicate in the find command instead of the type
predicate.
This change is needed so that we can dynamically symlink hook scripts
into hook directories such as install.d at runtime to support different
install types.
Change-Id: I933e7f4b4dcf16956841d8c14aa63b0f9a18fc5d
Upstart support in Debian is somewhat experimental, but having it would
be especially useful for TripleO which already generates upstart scripts
which will work fairly well on Debian based systems.
Change-Id: I267da5ae7bc4a484fac152a14963d96ca2ad2693
Also allows setting the "distro name" so that things that change the
debootstrap signature will have a different tarball.
This is being done in advance of a debian-upstart element to allow
experimenting with debian and upstart.
Change-Id: I1a558fdaf5924c65f1cced9f9bf29d73643c30af
tgtd returns execution control and backgrounds itself almost immediately
and before it has made it's listening socket available. This can cause a
race condition as the tgtd socket is not available when tgtadm is run,
resulting in an error:
failed to send request hdr to tgt daemon
Add a function to check if the socket is available before moving on to
calling tgtadm, and a wait_for helper function we can use. We'll
check for the socket every 0.5 seconds, for up to 5 seconds.
I'm seeing this issue on almost every deploy using a ramdisk built from
Fedora 20. I'm not sure if something has changed in tgtd, but this
behavior is documented since Fedora 18 at least. In the systemd script
for tgtd, there is actually "sleep 5" to work around the problem.
See Also: https://bugzilla.redhat.com/show_bug.cgi?id=848942
Change-Id: Iffa9fc63393309ca653d592dff17316ecbea3e09
This element can be used to provide a custom list
of modules via DIB_MODPROBE_BLACKLIST which
will be disabled via modprobe.d/blacklist.conf.
I'm using this to disable network adapters on the
Red Hat TripleO rack where the devices can't be
explicitly disabled via the BIOS.
Change-Id: I8a0a8ee05fa62628434d7f6422577dbf5cdd7a2e
Updates the dhcp-all-interfaces element so that the link
check tries up to 10 times.
On some of my machines 3 times (seconds) doesn't appear to
be quite long enough...
Change-Id: Ibf7015162fc890d4de8a417b868b0301146944c0
This is pretty much a duplicate of t-i-e/elements/os-apply-config, isn't used
in tripleo and the upstart script is out of date. We don't need to carry this.
Change-Id: I2b23d5930afae42cff28e2d5ce3d6c84224afa4a
Adds a new "package" type to source-repositories. When the package type
is specified in an origin-repository-* file the package name and type
"package" need to be specified like so:
nova package
The existing map-packages mechanism can be used to map the specified
package name to the actual distribution package names. The
pre-configured package repositories on the image will be used to install
the packages during the install.d phase of the image build.
If the repo type is package, an environment.d hook script is updated
with an environment variable which is sourced by dib-run-parts. This is
so that other hooks, particularly in install.d, will know what repo type
was used and can make the correct decision about doing a source install
or not.
Change-Id: Ief4e524664cd32d2087aeb3b1766f05e017de91e
Before downloading and caching the latest version of grub2, we need to
first remove all old versions. Otherwise, the find command that writes
the grub2 version to /tmp/grub/install to install during the finalise
phase could pick one of the older versions of grub2 in the cache. This
will cause the install during the finalise phase to fail since the older
version of grub2 will conflict with the newer installed version of
grub2-tools.
Change-Id: I5b43e359db3ba81ed6283dfb41c7e89629516d38
Includes:
* Enhancements to the generate-interfaces-file.sh script
so that it generates ifcfg network-scripts for
distros like Fedora/RHEL.
* Includes a new dhcp-all-interfaces.service systemd
script which ensures network interfaces get generated
before the network service starts on Fedora.
* Add a new disable_interface function to
generate-interfaces-file.sh which (on Fedora)
will delete the ifcfg config for an interface which
isn't plugged in.
This is important because ifcfg-eth0 exists by default
and we want to remove it if the NIC isn't connected.
Previous behaviour on Ubuntu is unchanged and the
generate-interfaces-file.sh just logs it as skipped.
* General doc updates...
I tested this on a multi-nic machine w/ Fedora where the first NIC
exists but was not connected.
Change-Id: Ia99e312539da43caefd72aa60398d43dac5dcc8f
Closes-Bug: 1239880
Tcpdump has a fairly small footprint and should be installed to all
images (it currently is on the ubuntu image by default). Sometimes the
reason you need it is also the reason its to late to get it.
Change-Id: I4849157515b389d534bdee755cdac6b0768fcb29
install-packages is an executable script, it should be +x. The install
command that was installing it is setting permissions to 755, so we were
fine there. However, I was trying to test the script outside of an
image build to test a change, and it's helpful to have it +x in the
repository for that context like all the other scripts under bin.
Change-Id: Ic300cc56d463640d119db4e310d375c66ed133ca
If "kpartx -av" is called and there is no unused /dev/loop* device,
then kpartx fails. Calling "losetup -f" finds first unused device
and creates it if it doesn't exist.
Fixes bug #1195264
Change-Id: I5d59b519fd8e2a7380b71a788f9d3f2331b6567f
dib-init-system is installed in 04-dib-init-system. Before it is
installed, 10-dib-init-system.bash may be sourced so command not found
issue found.
The patch invokes "/tmp/target.d/dib-init-system" directly in
10-dib-init-system.bash.
Change-Id: I9843224497c7dab8392261b26d62b6d79d1795ee
This is a dependency for the libvirt-python Python package that
ceilometer-agent-compute uses. libvirt-python is already installed globally in
the system via apt/yum, but that's not available to ceilometer-agent-compute's
virtualenv.
Change-Id: I07a12226889c6cda3bd04021240ee9714a396522
If we don't recognise git/tar/etc as source-repositories repotypes, we
now log the repotype, for debugging purposes.
Change-Id: I7b870807658441c928c7880291eb0284b6f97778
We now unconditionally cache the repositories being fetched with
source-repositories.
Additionally, by improving the way we handle repository refs, we are now
able to build images with code taken directly from gerrit reviews.
Change-Id: Ifd4c5691f6761eb5551663e6d0aa1c0e42afced3
Add apt-sources element for Ubuntu OS to override the default
/etc/apt/sources.list in cloudimg.
Set DIB_APT_SOURCES with a proper sources.list file to replace
the default one.
If you want to use this element in tripleo project, set NODE_DIST or
EXTRA_ELEMENTS / UNDERCLOUD_DIB_EXTRA_ARGS / OVERCLOUD_DIB_EXTRA_ARGS
to make it take effect at build time and run time.
e.g. before running devtest.sh:
export DIB_APT_SOURCES=/etc/apt/sources.list
export NODE_DIST="ubuntu apt-sources"
Change-Id: I9ce0d03b506c2948b96382e4d6e85f0aff906450
Some of the tripleo image elements rely useradd a for creating a group
with the same name as the user that is added. On openSUSE is feature
is turned off by default. So enable it here.
Change-Id: I120067e5e522dc6e704e7f09b724d60dee0c7f22
52-force-text-mode-console, running after 51-bootloader, calls
grub2-mkconfig (on Ubuntu that will happen via the update-grub call).
This call will overwrite all the changes that 51-bootloader did to the
grub.cfg. By merging the 52-force-text-mode-console code into
51-bootloader we avoid that. This also means that 51-bootloader will now
call grub2-mkconfig regardless of whether there is already and existing
grub2 configuration or not (52-force-text-mode-console did that in the
past anyway).
Additionally this commit enables the force to textmode code for openSUSE
(was previously only working correctly for Ubuntu and Fedora).
Closes-Bug: #1260323
Change-Id: Ida8e1a69df22d41ad70eb7154707c50613b536a8
This provides some customizations for zypper based distributions (e.g.
openSUSE and SLES). It is heavily inspired by the yum element and sets up
zypper to cache downloaded package outside of the chroot so that they can be
reused to speed up subsequent builds.
Change-Id: I775c921ee19cf7d1180fb68c1b7857ea6245a77d
Remove the comment about the rpm-distro element. The only tiny bit we copied
from it is 01-install-bin. 01-override-zypp-arch was unneeded and has been
removed from the opensuse element before the initial merge.
Change-Id: I9886bb6e7d723690f2d2ac476d6d3aca5a367607
Fix the test for $TARGET_ROOT in root.d/50-yum-cache.
Don't use $TARGET_ROOT in pre-install, it's only set for "root" and "cleanup".
Change-Id: I048364ea08ef503a4466f3494f18f72ebf99b5c2
This is currently experimental/incomplete. See
elements/opensuse/README.md for details.
Change-Id: Ie5bcab15c9703cf15ceb642fb986a8afc34f96fb
Co-Authored-By: Ralf Haferkamp <rhafer@suse.de>
Make the source-repositories element log which repo it's cloning from
so it's easier to tell what's going on.
Change-Id: I224f7bd98bc3d0ba94313dae9063d4ba4e25bdaa
If you build a ramdisk with ramdisk-image-create and forget to specify
-o, you end up with a .kernel and .initramfs file as the image outputs.
This commit defaults the $IMAGE_NAME value to image so you'd end up
with image.kernel and image.initramfs
Change-Id: I10f3ac9ad178c32119523e4da930070e3ade4f73
Ironic requires callback from deploy ramdisk via API with json data.
Auth token also required if Keystone auth enabled for Ironic API.
This patch adds separate element 'deploy-ironic' for Ironic deploy
ramdisk creation.
Change-Id: I5ef96711763e50307cfa17c406a9e16bffb937bc
When `/etc/selinux/config' does not exist, although selinux has
already been disabled, the image creation will fail.
Change-Id: I9e4a9a006073fd3f708049407ef98f82c3f399d1
In tests, saucy has been able to match raring for the ability to bring
up clouds, so we can roll forwards and switch the default to saucy.
Change-Id: I4df35d40f902c0daa7b2761d7afea8db25526753
tgt maps to scsi-target-utils on RHEL. This mapping already exists for
the Fedora element, so no need to add it there.
Change-Id: Ifc9a8976299458cdef50c2fbe00ba6a0c8d22f00
Closes-Bug: #1252975
We need to update openssl before proceeding as part of the image build
because the version on the Fedora cloud image is no longer compatible
with new python environments installed by virtualenv.
Change-Id: I3ed889f7382e7ca0de052ef5bdd6afeefc8bbdd8
Closes-Bug: #1254879
We can no longer rely on the name of the udevd binary as a hint for its
version, so instead we query the binary and behave accordingly.
Change-Id: Id4c869dfeeca5ed970169343b6b5f0f2c4499ca3
The Ubuntu build for ARM breaks when the pre-install step attempts to remove
grub components. This change gates those package removals based on whether
they are currently installed.
Change-Id: Ie169dbf12213e69b3713b3b1f9a68ae224f9fd31
The key 'RPM-GPG-KEY-redhat-release' may not exist in image
e.g. when you use non-default image (for example, if you build CentOS
image using rhel element).
Fixes bug #1252697
Change-Id: Iae642abb7a7f76c5528750dafedd4fd4bda56fe2
We can now actually override the $TROUBLESHOOT environment variable in
ramdisks, by not declaring it as readonly. Yay for insufficient
end-to-end testing.
Change-Id: I026c9dbdd411bf3eb945d5b18188eb1eedf3850a
We now allow operators to drop into a troubleshooting shell while
booting a deploy ramdisk, without having to have pre-configured their
desire to do so ahead of time, by means of a 10 second pause for them to
press a key.
Change-Id: I05ffa1ebaa95c83dee7bd2a2f52ba4c08928bb10
If we are booting pxe booting using syslinux, and it has IPAPPEND 2 in
the boot stanza, then it will append the mac address of the device we
are booting from to the kernel parameters where we can get at it
pretty easily.
If we are booting physical hardware via UEFI over the network, we can
rely on the BootCurrent EFI variable to point at the boot entry for
the NIC we booted from, which will include the MAC address of that
nic.
If neither of those cases are in play, we can just fall back to the
all-physical-devices-with-links code.
This currently uses the Bash 4 support for associative arrays to handle
the netboot-in-UEFI case, if needed I can rewrite it to be Bash 3 compatible.
Change-Id: I5e50e30c60d6d732a09ab61251cbb9be08bb6113
Since we are using bash syntax in some of the element fragments,
we should make sure we use bash for all of them, so that things don't
break on systems where /bin/sh != /bin/bash.
Change-Id: If2f043c57aa4e1492b7f9839213ef6123f683612
We now run the network bringup portion of the ramdisk init from an
init.d script, thus allowing image builders to inject code into the init
script before that point (e.g. configure network hardware).
The environment variables used by the network bringup code are retained
in the base init script so they can be overridden by init.d fragments.
Change-Id: I1ef0bb21e7f26c0ff3f02266f853ce5402bcb94d
Closes-Bug: #1252023
Check if Upstart exists before blocking the daemon. Not all dpkg based
OS's have Upstart installed so this should help increase compatibility.
Closes-Bug: #1251949
Change-Id: I2dcb1ff3641778b5653ca5762a728398adb57da1
We assemble the init script of ramdisks with the fragments from included
elements, alphabetically.
We now place leading numbers on all of the fragments we ship, to make it
more obvious to element authors and downstream users, how the process
works.
Closes-Bug: #1251706
Change-Id: I56b0d42971c8c462eddcfe1769f8124405e1233c
dib-init-system script is installed into $PATH. Called
without arguments it will print the name of init system
used to stdout.
Additionally, set DIB_INIT_SYSTEM environment variable to
the init system used.
Tested on ubuntu+upstart, centos+upstart, fedora+systemd,
debian+sysv.
Closes-Bug: #1251610
Change-Id: I29668079091f6060dab66d8259890384d3bbd653
This is needed to run scripts from devtest, in particular so that
swift ring files can be distributed.
Change-Id: Icbe8c5b24a0d494730357983f230a37a7f078de7
ramdisk-defaults file is sourced before chroot to an image being created
is executed. Busybox path should be set inside chroot instead.
Change-Id: Ic0e1d3093ca9d489dd96b14157e93b336cf0baee
The command `sudo rmdir $TARGET_ROOT/lost+found` will fail
if `$TARGET_ROOT/lost+found` directory doesn't exist,
e.g. when you use non-default image.
Fixes bug #1245856
Change-Id: I48c8f2f201b29912a726249023ca7d20893cc958
These are different tftp implementations, tftpd-hpa and tftpd-server are
equivalent, this mapping was added in a previous commit.
Change-Id: Idd1f554d4ee8e1d3c515d5f2f191e90abe0ff396
Now that we have moved onto neutron-dhcp-agent. dracut-network is no
longer needed, infact it was getting in the way. Dracut wasn't
requesting the classless-static-routes dhcp parameter but was
configuring networking. Removing this causes the network service to
redo dhcp and set the appropriate static route (specifically so
overcloud nodes are routed to the correct metadata server). This element
is still required for undercloud nodes so needs to be included there.
Change-Id: Ic23560164aff7791dd87ed2c3ad008efd248ae93
This will be needed by a patch I'll be submitting to t-e-i to
change the tftp implementation used by ubuntu, so that both
Ubuntu and Fedora can use the same implementation.
Change-Id: Ie9581265dfa2c37342e01162fe580b1948dd3e03
We need this for the Horizon element (which should work both on debian and
fedora-based distros).
Change-Id: I3a1dacbd1823abe051ba977b92b67ed27423f405
Signed-off-by: Tomas Sedovic <tsedovic@redhat.com>
I am getting a lot of http 404's from Fedora mirrors, in particular in
the US. Alternatively we could get cache-url to use previously existing
files if they exist and there is an error but this may not be the
correct thing todo in all cases.
So for now we do a retry if HTTP 404 is returned to the request for a
Fedora image.
Change-Id: I7634aeac3bdfe6930fafe6d276186c09fc8587b5
When picking the latest version of the kernel on i386 we should prioritize
PAE kernels. Debug kernels will be ignored.
Change-Id: Ic0fc5907074ee2a5ddfbbb1db2f1c8a6060cae9f
Related-Bug: #1240873
If the /etc/init drectory gets created on Fedora, it causes
os-svc-daemon to add upstart init scripts instead of systemd.
As a result none of the openstack services can start.
Partial-Bug: #1239880
Change-Id: Iec317baa3eb9ff651fa66c582d2f614993cde45e
For some reason we are adding CR's to/etc/network/interfaces, but it
does not require them. They are a bit annoying to see in an editor and
serve no known purpose there.
Change-Id: I9aeeff5533f418f09fcf33edd42e5d85cd486d23
Previously dhcp-all-interfaces was only blocking the first network
interface to come up. We add an instance stanza to make it run one
instance of generate-interfaces-file per interface. We then use flock to
serialize runs of generate-interfaces-file.
Fixes bug #1233577
Change-Id: Ib16bed6b37ce0789e315ef57e05ac561470a6f2a
Ifquery does not print anything for interfaces which only have a single
iface line. It does, however, return non-zero if the interface is not
configured at all, so we can use that to indicate whether or not there
is a configuration.
Fixes bug #1233579
Change-Id: Ia2fdafbea57e806eba99ae8ddaf395ebdcc306e1
The fedora element downloads the latest available image so presumably
will jump to F20 once available. Probably causing several days (weeks?)
of busted stuff. Of course it will be impossible to know when all users
of the elements are ready to switch but the least we can do is allow a
little time as a buffer.
This commit ties it down to a specific version which can then be updated
when the consumers of this element are ready. This allso follows the
same pattern as the ubuntu element.
Change-Id: I15c8e15a66e8af1bd152c27144acbc55af9da88e
Commit c7d80dd (Cleanup mount points automatically) removed the unmount
of $TMP_MOUNT_PATH/tmp/ccache in run_d_in_target() and moved the
"rm /tmp/ccache" to elements/base/finalise.d/02-remove-ccache. There
are two problems with this:
1) Not unmounting at the end of run_d_in_target() results in tmp/ccache
being bind mounted muliple times on top of itself (three times, if you
just run `disk-image-create base`). It is eventually unmounted, but
somehow the auto unmount code is confused, and tries to unmount it
one more time than it was mounted, which results in an error like
"umount: /tmp/image.THQkZxQa/mnt/tmp/ccache: not mounted".
This doesn't actually break anything, but it's a little messy.
2) "rm /tmp/ccache" in elements/base/finalise.d/02-remove-ccache never
succeeds in removing /tmp/ccache, because that hook is invoked by
run_d_in_target(), *while* /tmp/ccache is mounted.
This present commit solves the above by moving the ccache setup glue out
of img-functions and into the base element's root.d. This has the
following implications:
1) lib/img-functions is a little cleaner.
2) /tmp/ccache is available in the chroot during the root, extra-data,
pre-install, install and post-install stages. It is not available
during block-device, finalise and cleanup stages as it will have been
automatically unmounted by then.
3) /tmp/ccache won't be setup if you're building an image that doesn't
include the base element.
Change-Id: Ief4c0a6f4ec622db6c6f652776215684178d8943
When extracting the base image without --numeric-owner, user and group
names in the tarball are mapped to uid/gid by the host. This can cause
problems when building an image for some other distro than you're
running yourself. For example, building an Ubuntu image on openSUSE
ends up with /var/cache/man in the image owned by 'proxy' (uid 13)
instead of 'man' (uid 6), because the host (openSUSE) uses uid 13 for
the 'man' user. This particular man/proxy discrepancy results in
"fopen: Permission denied" errors when apt-get does its "Processing
triggers for man-db" thing in the Ubuntu system. I wouldn't be
surprised if there were other kinks caused by this uid/gid mapping
discrepancy too, but that's the one I found so far.
The same thing can also happen with Fedora, but seems to be less likely,
or at least less obvious to me when building Fedora images on openSUSE.
But, IMO, it's better to be safe and just use --numeric-owner on all
base image untarring outside the chroot.
Change-Id: I9da5ac66dd182e7278fe4fee932093f61d35673a
The code to handle unregister of RHEL subscriptions was buggy and
broke if no subscription credentials were supplied.
Change-Id: Iac29c45f207725e31eac6487a87367fcd3d34d49
My previous attempt at forcing the mellanox module to load was
completely bogus. This should not be (although I lack hardware to be
100% sure).
Change-Id: I22ff88181c9c9f0c024e021eeb7f16d79715241a
Closes-Bug: #1233949
In cases where servers ignore the Modified time, curl cancels the
download, outputs a http 200 and leaves the output file untouched, we
don't want this empty file.
Fixes bug #1234926
Change-Id: I05b0dd95dcd53ca50d88ec07f2f1ee9958b6adb7
Cloud-init needs to query the metadata server after the network
interfaces are configured. The upstart job "cloud-init-nonet" is
specifically in place to provide a hook to block cloud-init from
running while we rearrange network interface configurations.
Fixes bug #1233577
Change-Id: Ib5cf75d858fdb670b2abcc082e912c4644d6b169
When LC_NUMERIC is set to a format that doesn't use a decimal point,
`printf` will fail.
Change-Id: Ie6c4d075928f47b17cc413d537fc31c9d0734bdb
Signed-off-by: Tomas Sedovic <tsedovic@redhat.com>
DIB_IMAGE_CACHE will be a user override for the location where images
are cached. Default location is ~/.cache/image-create
Change-Id: I3e9b9f970864d555c9ec9436344b53f6d3d66dfa
This package recently caused us some very large headaches when it
was updated for a security issue. It is completely unnecessary and
should be removed.
Note that we have recommended that it be removed from the cloud images
in launchpad bug #1227425.
fixes bug #1227420
Change-Id: Ic0d4efa7b44c46271d19576f5191c9421d07c015
OpenStack runs git.openstack.org which is more reliable and responsive
when projects operate within OpenStack Infra. Replace all of the
references to github with referneces to git.openstack.org.
Change-Id: Ib3ece85aba6451801487b0bdbd83147e39d9e155
Changing the grub config makes no sense in a build not heading for a vm
and may fail because grub is removed from images not including the vm
element. Forcing textmode for those images would be better done in nova.
Change-Id: I1c5b89e551e62df2463200b1889cb2342498c7dd
Boots into the new image kernel once baremetal-deploy-helper signals
it is finished using kexec utilities.
Change-Id: I705787cc394ef14200d80404ee497762ab79b452
In some cases cache-url fails when downloading an image and leaves
an empty cached file. qemu-img then fails with "Wrong medium type"
error on next run.
Change-Id: I23e91c52094f27248cf8452f192ad63646051190
When uninstalling grub2, leave all its dependencies
including grub2-tools installed to minimise the number of packages
which need to be installed in the finalise stage.
Since the yum cache is unmounted during finalise, installing
grub2 in finalise is slowed by re-populating the yum cache.
This change copies the grub2 rpm out of the yum cache so it can be installed
from file during finalise.
This should prevent disk becoming full during finalise on Fedora.
Closes-Bug: #1217185
Change-Id: If095adc4abb52a19a3aa0b1caebfb3e4d8f605ef
This option does not exist on RHEL hosts and matches what is
currently present in elements/rhel/root.d/10-rhel-cloud-image.
Change-Id: I578233c1f37d035c67600fc60e7c4eb4ff75cbb3
Our custom dib-run-parts now supports --list for better compatibility
with some OS-provided run-parts tools.
Change-Id: I2feb615b2094d9d6522bfe9b422362223bb8e652
Closes-Bug: #1190521
Installing redhat-lsb-core adds 87M to the install due
to bugzilla #1002342
lsb_release is a platform agnostic script, so this change fetches
a single file via source-repositories from linuxfoundation bzr http
and installs it to /usr/local/bin
Change-Id: I4c979d4eb5e34a753d9143d9dcfe4e5d08f74a0d
Adds a file type to source-repositories to allow individual files
to make use of source-repositories caching.
REPO_DIRECTORY has been rename to REPO_DEST since it is now sometimes
a file.
This change also fixes the incorrect path to the cache-url in the
tar type.
Change-Id: Ib79e41969c982a02f0235318d9f254b39c3c6d93
patch is installed indirectly via redhat-lsb-core, so it
should really be installed directly.
Meanwhile, Daddy Shadowman waited for these patches to be
submitted to bugzilla.
Change-Id: I2107cf7be99f396c5dd489a2693b949022a512e7
If you want to have the installation update packages, you'll
need to register the system log in to rhn and subscribe to an
available subscription.
export DIB_RHSM_USER to your rhn username
export DIB_RHSM_PASSWORD to your rhn password
To get the qcow2 image, log into rhn.redhat.com and download the
image from
https://rhn.redhat.com/rhn/software/channel/downloads/Download.do?cid=16952
Then export DIB_CLOUD_IMAGES to whereever you're hosting the qcow2.
Change-Id: Idb547f4ffe75514b1e3f6b34f5f347493b132925
If GRUB2 is not available in the system a fallback to install Extlinux
will happen. This patch also allows you to enforce the use of Extlinux
exporting the DIB_EXTLINUX variable.
Change-Id: I88fa792a0bd19cacca167134f2054b11c807a451
This patch is part of a another big patch that will implement a fallback
to Extlinux in case GRUB2 is not available in the system. The reason why
it's being splitted in two patches is because git is not being able to
recognize the file changes when the rename happens in the same commit.
Change-Id: Ic2027dfa057fd6d62b532203b7ff50c3c739bccb
The element bind mounts a pip cache inside the image build chroot so
that pip downloads can be reused across image builds. While similar
in purpose to the PyPi element that sets up a mirror, this element
just allows for a reusable download cache and doesn't require anything
to be setup beforehand.
The pip-cache element is not concurrency safe, and that is indicated
in the README for the element. An upstream bug was file as well:
https://github.com/pypa/pip/issues/1141
Change-Id: Ibd1d4ea17c24923ed939357ada95b781e3179cfd
Rather than dublicating code to implement rhel or any
other derivitive, this patch introduces an rpm-distro
element that should be used as a dependency.
Change-Id: I8a92bb041764d03f430b438f0013704f79a8674c
If the loop partition isn't created as a device by the kernel, go ahead and use
kpartx to create the device.
Change-Id: I53290b7724d0cb45a1fc9225ec096025db8978dd
When ramdisk-image-create is called with the hwdiscovery element,
it failed to install hwinfo package.
The root reason is 10-hwinfo is not executable, the fix is to
set the file permission of 10-hwinfo to be executable.
Change-Id: I3a57147103f607b3975eb6ad23a44475f26c2f1e
IP address is passed as a kernel boot param when booting undercloud
machine. For Fedora 19, this IP is not set, instead IP is fetched
from DHCP.
The problem is caused by a bug in dracut-network version 029, which
causes that dracut network setup script is not executed properly. This
patch applies a fix for the bug when building undercloud image.
Fixes: bug #1213833
Change-Id: Ie94a86ed716044343791628973143c1a7d0778dc
The yum element provides for specific customizations
for yum based distros. The included customization is
a yum cache mounted outside of the chroot so that yum
downloaded packages are reused on subsequent image builds.
Change-Id: I6833c9fdbc83cb09debec6a789082e105c917800
The dynamic kernel module system is not available on RHEL, CentOS,
Scientific Linux, or SUSE. Make it part of the distro post-install
rather then base post-install.
Change-Id: Ic2c345bf9f0738dadae611194e263d3a5d424a3e
- pip install to a github URL without a git+ prefix fails.
- Need to close EOF heredoc when creating mirror.yaml
Change-Id: Ic555174881e9c06fede1cf731141cbe6eb0e0b02
Using a custom pypi mirror can be very convenient, making image builds
substantially faster - because we create multiple virtual
environments we benefit more than single-virtualenv users would.
Change-Id: I997daf1f9477c447e1fb30818aea9e80a49b31a6
Manually listing the mount points that need cleanup is not
maintainable and makes it hard to write elements that use bind
mounting to inject resources into the build environment.
Change-Id: I7c9ade444f4ebe42552f8e321f257a7ec0a674ef
The images produced by the diskimage-builder have their filesystem with
a label of "cloudimg-rootfs", The regex changing fstab on fedora didn't
match the F19 case.
This regex matches both
F18 :
LABEL=_/ / ext4 defaults 1 1
F19 :
UUID=f2da0fc8-2e12-4a6e-8e95-a31a474aa1a8 / ext4 defaults,noatime 0 0
Change-Id: Iafcf5e069199f1df76874309b2d5064f343fb5e5
We try to expose metadata to users about what dib included in the
image and source-repositories was not doing that, so this fixes it. It
also permits easy introspection from other elements (such as an
upcoming PyPI cache layer).
Change-Id: I1e48e65f69ebe1f4556cc3bf35ccf195e99bcc76
Ubuntu 13.04 has been released now for 3 months. The updated libvirt,
openvswitch and kernel are all beneficial to various OpenStack components,
and many other software is updated beyond the versions in Ubuntu 12.10.
Change-Id: I358aed8bf906c3ff5103f19b1f9e6ac689b5d5ee
source-repositories uses find to discover things it should import. If
someone happens to use emacs, and happens to have had an autosave file
in a directory with a wildly broken version of a source-repositories-
file, then dib will attempt to execute the contents of the autosave file
and the user will be sad. Adding a filter to the find command for files
ending in ~ will prevent pain and suffering.
Change-Id: I27c8a4b53d3c9b913ca926db16b4271941f6fb5d
devstack-gate needs to be able to make an image that has pre-downloaded
but not installed a sequence of packages. To support that, add a -d flag
to install-packages to allow downloading but not installing. -d passes
through on a dpkg install, and with this patch should be transformed
to --downloadonly which is the yum version.
Change-Id: Ia4e61fa4304df86afbee7dfc8067ea1bdf528f3e
Symlink /usr/lib -> ../lib for Fedora deployment ramdisk. This
symlink is needed for systemd-udevd. Otherwise, the network device is
not loaded and initialized when the deployment image is booted. This
logic was previously contained in
elements/deploy/root.d/50-redhat-prepare-fs, but now that ramdisk
images are built in a chroot, the change needs to be moved so that it
is applied to the ramdisk build.
Change-Id: Icea43230126956ccf5fb8a6a96ca706b75d5c32f
Create an install hook where core utilities can be installed
on Fedora. In this case we are installing which.
Which is being used by
ramdisk/post-install.d/01-ensure-binaries
vm/finalise.d/51-grub
The absence to which caused the ramdisk build to fail on F19.
On Ubuntu which is installed with the package debianutils, this pretty
much has to exist on Ubuntu images, so a corresponding package install for
ubuntu isn't needed.
Fixes Bug 1202612
Change-Id: I847c1619cdab021c9348caed3fa32db64e663e8e
- Ensures /sbin and friends are in $PATH when invoked (without this,
various sudo invocations fail in exciting ways).
- Use dib-run-parts in lib/common-functions instead of run-parts
(neither SLES nor openSUSE ship run-parts).
- Ensure dib-run-parts doesn't descend into subdirectories (same
behaviour as run-parts).
- Move dib-run-parts from root.d to bin (cleaner, consistent with
other elements with separate bin scripts).
- Tested by building Ubuntu image on openSUSE 12.3.
- Note: this doesn't add support for creating SUSE images, it just
lets you run disk-image-create on SUSE-based distros.
Change-Id: I906c6bc3cf51cdf2c4415adeae1ca250faac25e1
The search and replace that converted dashes in REPONAME only converted
the first dash. They should all be converted.
Change-Id: I76a48cf5bdfa8664f0a65d0238fa4da7d80305db
source-repositories does a git reset on the .git directory of cached
repositories. But doesn't specify the directory to reset. A working
directory needs to be specified so that the $PWD isn't used.
Without this change $PWD is polluted with the contents of repositories
being cached.
Change-Id: Ic37b702ac579bf766bb2204a988fa9468d308abf
Caching PyPI objects will require inspecting the content downloaded by
the source repositories, so needs to run after it.
Change-Id: I551c075e0bb6853b2ef79268b7f6455a4f06743d
Cloning large repositories over the internet can take considerable
time. Caching them locally makes repeated image builds significantly
faster, so lets do that.
When users override the element source they will often be using a
local repository, so in those cases I don't cache - but we could
easily change our minds on that in the future.
Change-Id: I9822362cf722b904d9806dbbb4bb07cfe2b33437
Similar to the Ubuntu element the Fedora element had not been updated
to honour --offline. Also similarly we only check for the existence of
the final cached file rather than the image we directly download,
because they are generated separately.
The -u change is just hygiene to detect future buggy changes.
Change-Id: Ib92a9d419bb485c8861da041066827e8d6aac0b7
When --offline is set elements should not revalidate cached data. The
ubuntu element had not been updated to match this. SHA checking is
also skipped as we only move a new cached file into place when the
hash matches, and we might download a new hash before updating the
image cache, which would cause persistent --offline failures.
Change-Id: If1a0366b51951a73b7a3ffe23a29a3d910b08938
In the 52-force-text-mode-console hook, gracefully exit when the
/boot/grub2 directory does not exist on the Fedora 19 cloud image
filesystem. By default the Fedora 19 cloud image is using extlinux to boot
the image instead of GRUB. The decision was taken because GRUB is quite
big (would pull in ~30MB of dependencies) and there's very little benefit
in using it because cloud images don't need to cover all the corner cases.
Change-Id: I52723d4d14c21b1787b4c7e0dd21a09cdbdae0d1
If you have an EFI host but the image doesn't have the EFI modules
installed the grub-install command will fail in case the --target
argument is not set. The problem is that the grub-install script will
check whether the /sys/firmware/efi file exists or not to determine if
it's an EFI installation, but this value comes from the host, so this
patch will look for the /sys/firmware/efi file and will also check if
the EFI modules are installed in the image, if not set the --target to
a non-efi platform.
Change-Id: I4481b43e4a8fe4144be9c7eb9d9c618bbb2df21e
In the 51-grub hook, after the GRUB installation, the script will look
for a GRUB configuration file and in case it's not present one will
be generated using the grub-mkconfig command. The reason why it have
to be done is because the new Fedora 19 cloud images is using extlinux
by default.
Change-Id: I80b15b3122698d98ac4d47dc06faf5909a90ab00
Ramdisks are now built inside a chroot which is built by the normal
image build process. Doing so improves our independence of the
precise state of the build host.
This fixes bug 1194055.
Change-Id: Ibc254fbb9e7b404b5f38c1b35bcde8a4136e8e28
Previously debconf was used and thus this only worked on dpkg based
systems. Now we can support any system that runs a reasonably recent
cloud-init.
Change-Id: I1397aefb3af400adcfd98f858c5a32fad2b584b5
Latest Fedora cloud images are available here
http://cloud.fedoraproject.org/
With this change, when a Fedora version comes up,
d-i-b will point to it automatically.
Fixes 1199237
(Also, thanks to Lucas Gomes for testing.)
Change-Id: Ifcb8a48f3baf3ef841f00e6a20c6fd9f79fea129
Many python extensions are built from source each time an image is
built. Repeating these is wasteful, so ccache is employed to eliminate
that waste. A cache dir outside the chroot is used to speed up
subsequent image builds.
Change-Id: Ib73563ddbe5f3be7454bfc54ab91cedb559a1304
Before each phase dib-run-parts will pick these up to setup the
environment for the commands it runs.
Change-Id: I56697d7bc370918d4063eee67a1c35d4988a5359
The URL to download the Fedora 19 images is not persistent anymore,
they are now download from http://download.fedoraproject.org where the
request will be redirected to a different location. Passing the -L option
to the curl command will make it redo the request on the new place and
download the image correctly.
Related to bug #1199237
Change-Id: Ib9b9d65734992e2bf1a6888c5f2eb9de366f8204
On Fedora 19, there is a delay between the time loop0p2
is created to the time it is ready to be mounted.
This patches waits for 5 seconds for /dev/mapper/loop0p2
to be accessible. If it is not ready then the script
exits with status 1.
Fixes: bug #1197340
Change-Id: Iff0cfd0566673799a77500e0e24954f5f3743a17
When downloading the base image for the first time, the response
test expression fails with '[: 200: unexpected operator'. This is
caused by using '==' in the test expression, which is not supported
in sh scripts.
Fixes bug 1195030
Change-Id: I66260814cb591371dc5c10f8436f90c2f18d78cf
Add "NONZEROCONF=yes" to /etc/sysconfig/network to prevent
169.254.0.0 route from being created. This route interferes
with fedora baremetal nodes' access to the nova metadata
server at 169.254.169.254.
Change-Id: Iad77abfc97c0eba944680772f233389481a270ff
On fedora 18 the package pyOpenSSL is conflicting with one of the
packages needed to build the bootstack image. The problem gets sorted
if you install pyOpenSSL at the beginning of the image building process
and this is what this patch is doing.
Change-Id: I681683b81ad638dfeb690fde1bc7d08a2116aaed
Use the new source-repository interface to git clone
source repository.
Also there was a few thing in this element that needed to be fixed.
o url to the git repo wasn't curren't.
o pip and git are also dependencies that needed to be installed.
Change-Id: Ieb7be6576fc909aa4bba0233b7b81cab03395d76
Git isn't a dependency of this element, but is a dependency of pbr
if you are doing pip install on a git cloned directoy, because this
looks to be a fairly common pattern it makes sense to have git on the
target image.
Change-Id: I49ee3fa4eac493b9dfa379a674ab9b3390cfc89a
Commit cc5ecb4e9c switched from using
run-parts in favor of dib-run-parts.
/usr/local/bin isn't part of PATH on Ubuntu causing
"/usr/sbin/dib-first-boot: line 8: dib-run-parts: command not found".
Change-Id: Ia7e401221add197edf880af82e0148f3ba7d2b9c
The fedora element downloads images too, so we should re-use the caching
code from the ubuntu element.
There doesn't seem to be other examples of code shared between root.d
scripts. In the fedora and dpkg elements we copy install-packages into
the chroot, but that model doesn't apply when we're running scripts
outside of the chroot. Seems sane to just run it directly from the bin/
dir in the temporary hooks directory.
Change-Id: Iaa6aca660042fea323cab4271633a4bdbbc271b8
If a cached copy of the file doesn't exist, cache_url() passes a
non-existent path to -z/--time-cond and you see this warning:
Warning: Illegal date format for -z, --timecond (and not a file name).
Warning: Disabling time condition. See curl_getdate(3) for valid date syntax.
It works just fine, but the warning is ugly.
Change-Id: Ic6f13a2c596b988308d7fca9cd1745e5d48ae5fb
At least on RedHat the run-parts command is very precarious and doesn't
even check for the return code of the scripts it runs causing the
dib-first-boot to delete itself even when something went wrong with the
configuration of the first boot of the image. Using the dib-run-parts
instead will make the whole first boot routine to be more robust and
less platform dependent.
Change-Id: I0073fd52fc1fc8848ca9277a84603124215cdb6e
Adds a new err_msg function which is used to:
-echo feedback to the deploy ramdisk console
-keep track of the first error message we hit so
that we can send it along to the baremetal-deploy-helper.
Also, updates our wget request back to baremetal-deploy-helper
to include the first the first error message (if any) as
the 'e' parameter.
The err_msg uses a new simple safe_url_encode function to ensure
we don't send invalid characters in our HTTP post requests.
Change-Id: I5a623a6f66cde8d81ff1e75800dc2953ca2703a8
Adds a new 'troubleshooting' function which encapsulates
the things we might want to do when something bad happens.
Typically this is echo'ing a simple message and launching a
bash shell.
The new troubleshooting function also makes use of a new
'troubleshoot' kernel param which if specific will allow you to
hang the deployment process and interactively debug the deploy_ramdisk
via a bash shell on a console.
Troubleshoot is disabled by default.
Fixes LP Bug #1191043.
Change-Id: I2cba8a9674075ba7e420027d40ef8cbe756cf07e
Fixes bug 1184943
Adding an element that allows other elements to register
repositories it would like to be retrieved for it.
Doing this outside of the chroot allows locally cached
repositories to be used. It also gives the ability to d-i-b
to specify what revision to use if an alternative to the
most recent is required.
Effectively allowing a CI system to test d-i-b and elements
without being effected by unavailable git repositories or
breakages in actively developed source code being used by
the elements.
Change-Id: I1527facebaad256a357af680e017b34b1788575d
Updates our deploy element's init script so that it calls
find_disk in a loop until it returns true (meaning we have a disk).
Previously on some bare metal machines find_disk would return ""
which causes the subsequent iscsi target commands to fail due to
a bad backing store.
This commit also updates the error message when start_iscsi_target
fails to be a bit more accurate as well.
Fixes LP Bug #1190984.
Change-Id: I3cd535d6672c197c1c3c539c83bba36be7a14e18
Having all security enabled is not always interesting, e.g. when
developing or testing new elements.
Change-Id: I828ecedc805ce4f89d60d185994eaa9c651d436a
When the target directory is empty, the for loop printing our profile
data was failing because of a glob problem.
Change-Id: I34e612021c70d6b8f75423de0d8be969c52e58e2
This patch will append /usr/local/bin to the sudoers secure_path variable,
by doing so commands prefixed with sudo will then find some binaries
delivered by other elements in that path (e.g nova-rootwrap).
Change-Id: I1ebad8b1932620559a3d90aa39298f9b12e636fd
Bug 1124593 reported that ssh host keys were not being created, and the
lack of seed metadata files was the cause. Add those files and remove
the workaround.
Change-Id: I6fb55be7f8c7efd1f75aeba9123f495e312ff695
The images produced by the diskimage-builder have their filesystem with
a label of "cloudimg-rootfs", so we need to change the default /etc/fstab
on fedora to reflect that.
Change-Id: Id1bb00cb81cb200a114f500e26272624be577da0
This element will install the network modules for dracut and regenerate
the initramfs. The network support would allow us to configure the network
of the image via kernel cmdline instead of injecting files into the image.
Change-Id: I829405127e77aec80b99ac8dd2032cb0c6d81063
After the refactor of the first-boot routine there's no use for this
script anymore, in fact all it's doing is to create a broken link.
Change-Id: Ic09e7208bc950380341e8a073647950ac99a05fd
Fedora's partprobe does not create devices the same way it does on
Ubuntu. Instead of using partprobe, this patch uses kpartx, which
works on both Fedora and Ubuntu.
Change-Id: Iac5cbac083e423425693c8210cd5d058ce4bce72
Updates the new 99-dkms element so that it installs the package
before using it.
Fixes LP Bug #1183785.
Change-Id: Ia191ed3cae707a8c8591f849dc0c6293e849d64a
Work around ordering issues that cause dkms modules to not always
be built for all installed kernels, add a post-install.d script to
the base element that will force the issue.
(Note that any module/kernel pairs with a pre-existing valid
module will not be rebuilt, so this should not impact performance
unnecessarily)
Change-Id: Iccea14c6318014cc7e1fec77e71c14a224f7b76e
Per bug 1182648 we were not setting the mode and as a result couldn't
log in, defeating the purpose of having a local-config element :).
Change-Id: I4114b9536392f1517eac8f4bbe6f4b60ce1186a1
Apply kernel command line parameters to force Linux to remain in text
mode. Typical distro kernels quickly switch into a graphical screenmode
and this is incompatible with common Lights Out Management hardware
which is able to intercept text mode consoles to display them remotely.
Change-Id: Id553972c4fd87e78c9e6fe344331a399913d965e
Setting up a routine to run at first time the system boot may vary from
distribution to distribution. This patch will remove the logic from the
diskimage-builder code and put it into the elements.
The base element will now deploy a dib-first-boot script and each OS
element will ensure that this script runs at rc.local time. By doing
that we will put the OS specific stuff in the OS elements and remove
the embedded code out of the img-function file.
Change-Id: I24c5d1b1185de5693f145347fe912245f1ba7dfe
Boot time was 30 seconds shorter in an all-in-one devstack
environment.
Thanks to Ghe Rivero for a pointer to the solution
https://bugzilla.redhat.com/show_bug.cgi?id=208275
Change-Id: I90d0c96d5659326ba67d6119b96d9a4113adf7fe
With systemd rc.local lives in /etc/rc.d and is only run
if the rc-local service is enabled.
This change creates a /etc/rc.d/rc.local symlink, and modifies img-functions
prepare_first_boot to tolerate an initially missing /etc/rc.local.
Fixes bug #1179674
Change-Id: I7e6102b62224950c1de5d48205dc4feff60f9389
baseline-tools expects lsb_release (although tolerates it not being there)
Order needs to be 01-override-yum-arch, 02-lsb, 03-baseline-tools
hence the renames.
Possibly lsb_release isn't cared enough about to do this.
Change-Id: I03abffb1ab2f560e746ba4ffd407605de31f4930
The later pulls in most of a desktop. The former still pulls in a lot
but significantly less, and still provides lsb_release.
Change-Id: Ie27310f01c037b4a10feb5e380e6f5c4ecf21364
We need to support mellanox devices in disk images as well as ramdisks.
This adds the requisite element structure to provide that support.
Change-Id: Iaf383f5b32b84fbb69569ef42ec2460ab1b231e5
The PPA would have to be updated for each release we want to build for,
including raring. The PPA also gives the tripleo team root on every
built image, something we don't necessarily want.
A shallow review of the packages in the PPA and known elements shows
only one dependency in the tripleo-image-elements heat-cfntools element.
Change-Id: Ic514f929eba26014dd3a0d76d5bb3b8e9e0caf3c
This also switches to using curl which some people may not have
installed. However, curl is far superior for this type of download.
Change-Id: I7ac5a84b30eb8daad320c082f976931c41a24669
Qemu-nbd does not perform well with older versions of qemu due to
the lack of writeback caching mode. It also only builds qcow2 images
and there is a desire for raw image support. Finally, qemu-nbd makes
it very difficult to build images concurrently due to the somewhat
opaque nature of how it selects a /dev/nbd# device. losetup, on
the other hand, makes this process very straight forward.
Change-Id: I309fad8af4fd1e8d1720c17b65e1897a76d5e897
Co-Author: Clint Byrum <clint@fewbar.com>
Update audit in order to fix up a conflict with a file from the glibc
package on a fresh Fedora 18 install.
Change-Id: Ib44c8415bce4ec47e21626a697cf426b96a9061a
Post install scripts are useful because they can perform tasks you want
to handle after the OS/application install but before the first boot
of the image. e.g: Clean the cache left by the package manager reducing
the size of the image.
Change-Id: I03e77f602192bbdce29c02999d1b57fac8051ddc
Fixes: Bug #1145786
This switches $CLOUD_IMAGES and $RELEASE to the DIB_ namespace so
they will survive future changes to the sanitisation of the build
environment.
Change-Id: I7dc2aa82fb9ef452705b080cc404f41046014f20
The root user tries to `sudo -u stack`, without a tty,
causing fedora+devstack images builds to fail in Jenkins.
Change-Id: Ia0a7fb315cf9bd17cf250e70dba06363a697c97c
cloud-init is expected to generate unique hostkeys on first
boot, but sometimes it doesn't.
This change ensures that hostkeys are always generated.
This change should be removed if the cloud-init problem is solved.
Change-Id: I7d7e05fbd12727992cf1aebec547ae0488e002ff
ARM doesn't have a generic Linux image due to the soc-specific nature of Linux
kernels today, so we drop the manual installation of that package, replacing it
with a dist-upgrade instead. This involved tweaks to the dpkg and fedora
install-package scripts.
Change-Id: I97924b80ca87781307e1087b9fe4b18215770e84
Errors in mysqldump would have been missed due to not having pipefail
set. Also there was a type-o in the invocation of mysqldump that was
going undetected, that has been repaired. Finally we should not pass
--master-data if we don't have slave credentials, as the master data,
even if provided, is useless without a valid slave user/pass.
Change-Id: I0f80af34c747f465250ef17fae4a44dac919c9cd
Strict sql_mode settings are very application specific, many will simply
not work with them. This setting was likely inherited from some other
production MySQL config file and is not appropriate for a generic
configuration. Specifically, this broke the MySQL package maintainer
scripts.
Change-Id: I068fcc7b3c770541d3d1ecfb372384750f182584
This will write two files in /etc/ that contain the environment and
command line arguments used in the creation of an image. This should
assist with later efforts to repeat the creation of an image.
Change-Id: Icdbe6693380bed6c406feee10d2cb1a88a992932
Quantum-ovs was continually re-creating interfaces
every second or so, because it was configured to use tunneling,
and the default version of OVS included with ubuntu does
not support tunneling.
Change-Id: I1246181c4b9c96ebe2795b337454dc8ddcb35e8f
Use Heat Metadata to specify host and credentials to pull whole database
snapshot and setup Replication. Also include root credentials for use after
the snapshot has been applied.
Change-Id: Ie3aa92463d28db54a523f520c2b4cdfb528acf9d
There's no need to run os-config-applier at any time except when
os-refresh-config is in its configure state.
Change-Id: Id9758294c60469a976b61c68fc26d9b83130599a
'glance' element now results in a properly-configured,
working glance service on first boot.
While glance-api and glance-reg may need to be broken
out again in the future, the intention now is to focus
on a working service for an end-to-end PoC.
Change-Id: I4fb1cd5ad6defef5a5c4fddfded46d51e8097b5a
Best practice is to use start-stop-daemon rather than sudo, as sudo will
apply user-centric pam limits and create a wtmp entry. Also there is no
need for a script stanza, as we are only execing one command.
Change-Id: I0c2f12536b56d90fd43ab40e74424350efcc0b61
Future work is intended to allow this to serve
as a nova-virt element as well, by allowing the config
and which services are run (such as bm-deploy-helper)
based upon metadata config settings.
Change-Id: I59718a0b329e6adc59097492638cf02c76162c0a
Python package dependency conflicts have been observed
to occur for certain combinations of services at certain
revision.
Running all services in virtualenvs removes the issue.
Change-Id: I100817569b43a5af3427b0ae20cebdc7d55d03a5
Ensures that all network interfaces are present in
/etc/network/interfaces. Any interface not already defined there
will be added and configured for DHCP.
Change-Id: I27e0902e62804e8d719acd7288109bd0e294fd16
Enable templates to specify a list of users that need passwords assigned
in Metadata. These passwords are then communicated back to the template
via an optional wait condition handle using cfn-signal.
Change-Id: Iaaf4d4a9d0d757b7d44ea39e77eed3c55ffffd88
Do nothing if we are booted on a server which has no instance-id set,
such as a non-cloud VM or server.
Change-Id: I6df81edbea1b2b4e4f1ef11a9f2ddaf100d98eda
We didn't need to do most of this anyway, we just needed to configure
mysql before we installed the packages in the image. Some of the other
things were half-baked ideas which will come in later changes to add
os-refresh-config and os-config-applier support to the element.
Change-Id: Ib1b66a8b3b70fac10f3f826b1efd414c67f9bfc6
It is valid to request os-refresh-config without putting any scripts in
the element root. This fixes failing builds of heat-cfntools.
Change-Id: I4f70b33e453399ff5baa48919557995a6eb27520
The code which raises an error if two elements try
to install different versions of the same git repo
currently works correctly for SHAs, or if a branch
is specified first, followed by a SHA.
However it fails if two branches are specified, because
the shallow clone does not fetch all branch heads.
This change correctly handles any combination of
SHAs/branch-heads/tags.
Change-Id: I501e94150ddeb41b79588816576a5d7a06fa92aa
cinder-api in trunk will no longer start with
keystone.middleware, only with keystoneclient.middleware.
The issue is discussed here:
https://bugs.launchpad.net/nova/+bug/1143998
Change-Id: I29f81ca2503957b4df58e86b2392ef4b3c15d6fe
This allows openstack services to be git/pip installed
without installing any startup scripts.
This is useful for keystone-db or nova-db elements, for
example, where the service must be installed to perform
database migration, but no service start scripts need
be installed.
Additionally, add a tool to create openstack sql databases.
Use openstack pypi mirror:
Use the openstack pypi mirror for openstack service
installation. It's much faster than pypi.org.
Also, pip install $svc_repo/tools/pip-requires first,
if it exists, which is required to pick up oslo.config.
Change-Id: I72751d4da59f8597d20aea2f27a9dfabe2f63a8f
Currently, the local-config and stackuser elements
are intertwined. local-config installs an authorized_key
for stackuser, if the stackuser element is in use.
This change does two things:
- add authorized_key for root from local-config element,
regardeless of whether stackuser element is in use.
- install key for stack user from stackuser element,
only if local-config element is in use.
Change-Id: I0d07b61404119ea0650c5c0fb98d6786adcf3ca9
* Provide entry point for Heat Metadata to drive os-refresh-config
* Use os-config-applier for configuring heat cfn tools
Change-Id: Ie38b932173948f1e7c414c48caa1b037916bef84
Copies all templates found in the root of any element into the
default template location.
These templates will be filled in from heat metadata by
os-config-applier whenever it is run.
Update keystone to install templates in this way as an example.
Change-Id: I0be0a79a431e9ba5b80e84f130c48d5ce8b100ae
Co-Authored-By: Tim Miller <tim.miller.0@gmail.com>
Also add redhat-lsb to Fedora images for lsb_release which should prove
useful as a way to do distro-specific things.
Change-Id: Ie32566349319ca244fa02065bb9f81c36c1b49fb
Relies on https://cloud-images.ubuntu.com being served by a cert signed
by one of the CA's trusted by the build host.
Change-Id: I690b755acca54789110c2c8fa723c8b87b2485c9
This element installs all openstack sql databases into
a single image, useful for a bootstrap image.
Change-Id: I6efe807793ec620a60d5798d28a7e3103a5b90e8
Then creates symlinks into /opt/aws/bin using the included script.
This replaces the heat-jeos element, so anything that references heat-jeos will need to be replaced with heat-cfntools
Change-Id: Id286b005f69ea364357303b4bbc1ed29d4f0f8f9
Also modified dib-run-parts to apply a more workable solution for
filtering out unwanted files such as editor backups and VCS.
The script is installed in its own element, depended on by the OS
specific ubuntu element. This is because the ubuntu element (and
later other OS's) are responsible for populating the root filesystem.
If we try to install this in base, the root filesystem will look to
be populated already and we will skip automatically choosing ubuntu.
Change-Id: I017646748c1a8360299106289b57d976d45875a8
This includes the install-packages implementation for dpkg, apt http proxy
config, daemon blocking and unblocking.
Change-Id: I8f159021d2b223d7003cec067de3aa605ad06974
This is very helpful when performing automation.
Additionally, removed addition of stack user to admin group,
since that group does not exist on all systems, and
passwordless sudo eliminates the need it was addressing.
Change-Id: I727bd51acb920c6c2e66247069998b6ee4267895
Move common openstack service installation operations
into a new script `os-svc-install`, which simplifies
the elements for openstack services.
Change-Id: Ied8ac3278e7fe8af76e24748ec4e598a84afa03c
Disables cloud-init from querying EC2 metadata, or any cloud metadata
service. It keeps only the Nocloud and None data sources, which are both
expected to be "baked in" to an image. This speeds up boot by 2 minutes
on hosts where cloud-init cannot reach the EC2 metadata service.
Change-Id: I43522c4a442c0f2fc2044d4898768ceb4f839099
os-refresh-config simply runs things in a predictable order in directories
that other elements can populate. This element installs it and sets it
up to run at boot.
We also modify os-config-applier element to be called by this rather
than to have its own upstart job. This provides an example of how other
elements can insert themselves into the refresh process.
Change-Id: I52d72b7fd302ec317840af87fa77299387af99ad
See https://bugs.launchpad.net/heat/+bug/1122472 for information on the
specific problem in heat. This can be removed when that bug is resolved.
Change-Id: I76a7a42dea9472c54f03593c8c67856aa23f2c73
This is a necessary but not complete step towards supporting Fedora and Suse
distributions. Further work is needed (e.g. to quiesce daemons on
installation).
Change-Id: If3ea6093d41a21de755db52328226b84b5a3ede6
Remove the explicit installation of an English language pack and instead
just default LANG to C. This settls Perl's noisy warnings and also stops
any side effects of different language build hosts.
Note that this is set right at the start of the build process, so it
should be entirely possible to override if needs be.
Change-Id: Id3b31162d4198fa02dc5a4d11168e57dbcd14a5d
This disables apt's default behaviour of installing packages in the
Recommends field of other packages. While this can be useful, it
causes a lot of unnecessary software to be installed and may cause
explicit dependencies to be missed in some circumstances.
Change-Id: I10f4519290e6b3dafda3365c0c5dcc42ad17c090
This change causes the "base" element to install and generate English
locales, thereby mitigating a lot of build noise (at least for
English speaking users). Also included is a note in the element's
README.md about what users in other locales should do.
Change-Id: I053c7e0254a0a0d7dde081f7cc72204a12aa8ef4
Fix whitespace error in local-config/install.d/61-http-proxy that prevents
no-proxy being written.
Change-Id: Ie2511bb5b2504c42ff59ec277cd91bf92b56da59
Fix a typo in the base element itself.
Refer to it early in the README since it is likely to be useful to
almost everyone.
Change-Id: I4a753b61542f8a354ade7fcc4436076aec61e6c0
Previously, only http_proxy was copied into the local-config image.
We now preserve both http_proxy and https_proxy settings.
Change-Id: I4de2712f2772fb7d07b228cecdc0c8ec6f3df567
Co-authored-by: Chris Krelle <nobodycam@gmail.com>
devstack element points to github:tripleo/incubator, instead of demo.
Also, the element now pulls devstack into the image, rather than relying
on incubator/scripts/demo to pull it down after the image starts.
Change-Id: I878edcc8f5b1a509b14cadfb5fb4c33571a7f312
This adds support for other elements to feed into the hwdiscovery
results.
It also fixes a few bugs in the construction of the ramdisk.
It also adds support for specifying a server to POST the discovered
hardware data to, via the kernel command line
Change-Id: I163db2b1388f908880e8f458e16906fa6f9db7bc
It makes more sense for grub to be installed by the vm element, since
that is intended to produce directly bootable disk images.
Change-Id: Ic340ba9d3ec4dbeea3aed3576fc899c1424783a2
Flavour is overloaded in openstack due to it being used by nova. Element
seems to have the same feeling of combinability without using a term already
in active use in the openstack community.
Change-Id: Ia4c028d4062a8f69c66665821c94dd4bcdf06031
