The script to find best kernel from image (used by vm
and baremetal element) is duplicated and is not in sync
with each other. Moving the code to img-functions as a
function will reduce duplicated code and make it reusable
in future.
Since img-functions is not accessible in chroot env,
kernel selection is being moved from finalise.d to
cleanup.d in the vm element.
Change-Id: I8fbccc13a2c61a5191ef9ea5d2a8302a3e43b000
Updates the 51-install-tgt and 52-install-busybox files
in both the deploy-ironic and deploy-baremetal elements
so that they have unique names.
While an end user would never use these elements at the same time we
should always use unique names for things per DIB conventions.
Change-Id: I8926870e45849341c10b2d479ddee45f9b625e2b
While an end user would never use these elements at the same time we
should always use unique names for things per DIB conventions.
Change-Id: I53b10522183e8566c62922b91878e5cf767fe2c6
Both the rpm-distro and redhat-common elements had the
same exact 00-usr-local-bin-secure-path. This commit
deletes the one in redhat-common and leaves the one
in rpm-distro (which is also required by the same
elements).
Change-Id: Ie9ef29d268b3737a6741fce016ef335db6651b59
While an end user would never use both redhat-common and ubuntu
elements at the same time we should always use unique names
for things per DIB conventions.
Change-Id: I2cabc583a1fd41d4fe7fbf8cf93127d49e26aff4
Updates the yum element's bin/install-packages script
so that if supports using pkg-map instead of map-packages
for package mapping. To make use of the new change
simply add the -m <element name> to allow install-packages
to know which element namespace to use when installing packages.
Use of the new -m option is off by default.
As part of this change we also updated install-packages
to use getopt for in script argument parsing.
Change-Id: I19240e74b69859f7faf9cd80440633234a677d63
With this patch, /var/cache/apt/archives directory content is preserved.
The directory is actually a bind mount of the
~/.cache/image-create/apt/$DISTRO_NAME directory, much like what we do
for ccache.
You can use DIB_APT_LOCAL_CACHE=0 to disable this behavior.
This trivial change improve performance A LOT (>30%), even if a local
HTTP proxy because:
- there is no need to copy again and again the same files
- we avoid network latency
The patch has been tested with Debian and Ubuntu with every elements
from the tripleo-image-elements repository, the final size of the cache
directory is about 700MB per distribution subdirectory.
Change-Id: I4fab499493f734c7c546d4d23b1a98f0e7523a39
OVS service is called:
- openvswitch-switch on Debian/Ubuntu
- openvswitch on RHEL/Fedora
This changes is needed to get the neutron element to work
on Debian/Ubuntu with systemd:
I6e3df30dc3a6918f3a949a7dac47289ede5c3d1f
Change-Id: I2748a9db7f8ee456698187a9646bc7cda2256fe7
This change adjusts the "inline" python function used to profile
run-parts duration to make it run successfully on both py2 and py3
Change-Id: Ib5287a56b8e2b3e882dca489192f01d9f01cd8ba
Now that issues associated with this bump have been addressed
elsewhere, reinstate trusty as the default DIB_RELEASE for
the ubuntu element. Also adds a note about memory requirements
to the ramdisk element.
Change-Id: Ia89c83ba0ee4ad3bd89b833ba0db08135eac63ed
Default to only having cloud-init query Ec2 on first boot for Ubuntu,
until cloud-init has been SRU'd to fix the CloudSigma data source issue
that causes Trusty boots to hang.
Change-Id: Icb3734d5ae78f4a0a6c0fae1af4a2ce3c809308c
Partial-bug: #1316475
This patch removes a couple of lines of redundant code. In Python,
the get method on a dictionary defaults the return value to None,
so the method never raises a key error.
https://docs.python.org/2/library/stdtypes.html#typesmapping
Change-Id: I180877f9be23d373cd63771bb3682d6587ec6fac
There is a substitution of suffix in rhel element, like
rhel-guest-image-6.5-20140121.0.x86_64.qcow2 to
rhel-guest-image-6.5-20140121.0.x86_64.raw
The convert is correct but the substitution of the suffix is wrong.
${PARAMETER#PATTERN} will match the string from the beginning and generate rhel-guest-image-6.5-20140121.0.x86_64.qcow2.raw
We should use ${PARAMETER%PATTERN} instead, which is from the end.
Change-Id: Ic45a3f800058ea9e5d746ff543c068a54632407d
This moves cloud-init data source configuration to a general purpose
cloud-init-datasources element that can be used to explicitly configure
the list of cloud-init sources that will be queried on first boot.
cloud-init-nocloud now depends on this new element to configure the
datasource_list while continuing to prep the image for a nocloud first boot.
Change-Id: Ibcc3b86d6ca567a23f89b7a1a36bc713e444ef68
Closes-bug: #1316475
Updates the various operating system elements so that we
set the DISTRO_NAME variable for each distro.
This env var is used by bin/pkg-map to set the default
distribution name for package name mappings.
Change-Id: Ib4b05eb7191dd50d25799a0bac51fd15c01b74cb
Adds a new pkg-map element which can be used to install
packages based on an in element 'pkg-map' JSON config
file format.
As part of this change we expose DIB's IMAGE_ELEMENT variable
so that we can have access to the element names which are being
installed in a clean manner.
This change is intended to decouple elements from DIB
and allow new elements to support multiple distributions
(with various package naming schemes) without having to
constantly maintain DIB's various bin/map-packages files.
Change-Id: Ic0a061995e2ae708c95a535cba6237bff58f7d93
A few scripts were added without set -o pipefail since the original
change to add that everywhere. This will get the dib-lint check
passing again.
Change-Id: I96bef45cc10ff9bbcf2c4f1b796b8cd188e10485
Add the ability to supply a gerrit ref in DIB_REPOREF_* and
have that reference fetched from the remote and checked out for
building within the image.
Closes-Bug: #1314021
Change-Id: I5e5742c4cbff98e14121c50dde5e512f192b5415
Co-Authored-By: Robert Collins <rbtcollins@hp.com>
Using -n for bash arrays was a little bit bad, and also resulted in
errors when running under set -u. Wean the yum element off the array
and switch to a straight string.
Change-Id: Ifd61fce7dadd5820022e9b2ab42020ba0cb71ce6
When using custom-built kernels, their filenames may not necessarily
follow the standard naming conventions for the stock systems in question.
This patch permits the specification of an alternative pattern to use
when looking for the correct kernel to construct the PXE images from.
Example:
export DIB_BAREMETAL_KERNEL_PATTERN="kernel*jang"
export DIB_BAREMETAL_INITRD_PATTERN="initrd*jang"
Change-Id: I86e5254051addd5e89dd4e363e9bba117e65353d
Add support for supplying a custom keyring and debootstrap script
using two new environment variables:
* DIB_DEBIAN_KEYRING - Path to a gpg keyring.
* DIB_DEBIAN_DEBOOTSTRAP_SCRIPT - Path to a alternative debootstrap
script.
Change-Id: I42e8832b435b8a77b0681b5bc9d9332bbe860cd5
Although set -u allows empty WHITELIST=() list to be set, you can't test
-n "$WHITELIST". Looks like the intention of the commit that caused
this issue (Change-ID Iad2d490c86dceab148ea9ab08f457c49a5d5352e) is to
set a policy to prevent future use of bugs due to unbound variables.
Thanks StevenK for pairing with me and helping me hone my bash skills.
Change-Id: I2349c990ba53b0aaecd5119812582c7c848f1e4e
Closes-Bug: #1319987
If the kernel is restricted read (as a security measure) we may not be
able to read it without chmodding it.
Change-Id: Ib577705c27d8c351288334de03a7848cf49f2de9
If 'DIB_RHSM_USER' isn't set, the image build with rhel
element fails because of check 'set -u'. So, we should export
'DIB_RHSM_USER' in script to avoid error during build
Change-Id: Ic63a20e3d91a25bbe804440a5919b035489c60a3
Closes-bug: #1318717
When the cache tarball gets fail the output doesn't indicate
what url is failing. Adding '-x' displays the urls attempted.
Change-Id: I71b2dbfd712e72c591410ca9b68edc76d493ad6e
As with the previous similar changes, this is intended to catch
problems as they happen instead of ignoring them and continuing on
to potentially fail later. Setting this on all existing scripts
will allow us to enforce use via Jenkins.
Change-Id: Iad2d490c86dceab148ea9ab08f457c49a5d5352e
Updates the fedora element so that DIB_DISTRIBUTION_MIRROR can
be used to configure a custom Yum mirror on Fedora. This
implementation replaces the baseurl in each of the default
Fedora Yum .repo files so that a custom mirror URL/path
can be supplied. The URL should point to the directory
where the updates and release directories exist. Example:
http://download.fedoraproject.org/pub/fedora/linux
As part of this change we also remove any existing
metalink settings in the .repo files to cover the case
where an end user runs their own createrepo command
to create the mirror... which means your metadata won't match that
of the normal Fedora mirrors and your baseurl won't get used.
Change-Id: I1a49a2941b1cc489e91a72b87430393cd32e865d
DIB's map-services is currently used in TIE's os-svc-config
binaries to support service name lookups. Many of the systemd
specific functions require bin/map-services to exist.
This change adds bin/map-services files that are noops
to both the debian and ubuntu elements so that as those
distros move towards using systemd they can make use
of the functions we already have in place.
Change-Id: If8d61f3858b11de86c3292e840d033e5e3cecedb
If scripts are not set -e then errors can be ignored, causing more
confusing failures later.
Also adds an exclusion comment to the ramdisk init script since we
don't want that to exit on failure.
Change-Id: Idf43993bd10b1ef16c1d3b0d9df8d0ad94c46458
Adds a new element, package-installs, that provides an interface for
declarative package installs and uninstalls. Packages to install can be
added to an install.d/package-installs-<element-name> file. The set of
packages listed across such files are installed in a single transaction
at the beginning of install.d.
Prefacing the package name with a "-" indicates that the package should
be uninstalled at the end of the install.d phase. Again, the full set of
uninstalls are done in a single transaction.
An element providing a package-installs file should add package-installs
to its element-deps file.
Change-Id: I5b540388eff1079c8dee933b869463371481152b
This commit fixes the if/else block around Debian stable/unstable which
was badly merged during the last rebase.
Change-Id: Ia2eb806da3d7c8d28afffb026f35600c7125d8d7
This makes it easier to work with temporary files - less traps
scattered around higher layer code.
Change-Id: I2fdd93115a7b0d289c2e28f8c133d4059de75b87
This patch adds a debian-systemd element to get systemd support with
Debian.
A backport of systemd is used on Debian Wheezy.
Change-Id: I095c81b2c32844fccf940c97b79054923c25b8c2
Upstart cannot be installed by debootstrap on Wheezy because of a
conflict with sysvinit. sysvinit is flagged a being “essential” and
apt-get will refuse to remove it.
See: https://bugs.debian.org/668001
This patch uses another strategy. We install a standard sysvinit
Debian chroot with debootstrap and during a second step, we
replace sysvinit with the wanted init system.
Change-Id: I081c81b2c32874fccf940c97b79054923c25b8c2
Cloud images expect the primary interface to run dhclient in order
to come up and acquire an IP address. Although we have technology
to manage without this for the undercloud and so on, the seed
still relies on this machinery in order to bring up its first
interface.
Change-Id: I079a679779de760ee9ca6f93cbb22e69a65e03af
Using set -e in all of our scripts will prevent some subtle bugs
from slipping in, and will allow us to enforce use of set -e with
tooling.
This change also adds -u and set -o pipefail in the less complex
scripts where it is unlikely to cause problems. A follow-up change
will enable those options in the complex scripts so that if it
breaks something it can be reverted easily.
Change-Id: I0ad358ccb98da7277a0ee2e9ce8fda98438675eb
In the mailing list discussion, everyone seemed to agree that we
should just be explicit that these are bash scripts since that's
where we're testing them and there's no good way to ensure they
work on any other shells right now.
Change-Id: I86c41d2cddf6e91e7deeb4c2ab51635c80a1ce0c
The phrase is no longer needed as of August 23, 2000 with Nicaragua's
joining of the Berne Convention.
Additionally, in at least one instance,
elements/cache-url/bin/cache-url, its existence in the file between
Copyright lines is just weird and feels misleading, even though it is
not.
Remove all of the lines, because sanity.
Change-Id: I24fd76c2b4f66b8036010b5079db39ead729abee
Trusty ramdisks do not boot, failing to exec init. This reverts
the default DIB_RELEASE back to Saucy until Trusty ramdisks have
been properly tested.
Change-Id: I5ac81822b78805275268cecc021a74a62c22ef58
Nodepool needs to be able to install kernel headers, which means we
need a mapping for redhat systems here.
Change-Id: Ic83d642b5ebe85b042cf7b23b295be74bbe185fa
Provide package name mappings for openjdk-7-jre-headless:
RHEL/Fedora: java-1.7.0-openjdk-headless
SUSE: java-1_7_0-openjdk-headless
See https://review.openstack.org/#/c/86314
Change-Id: I3a351d0f9854f6f320ba145a71ffc66fd02c231c
Unlike the redhat-common and opensuse map-packages maps, the RHEL one
appears to be unsorted. Fix that.
Change-Id: I75c5d2694525731660c2a3b5b58ee696c61923ba
This patch adds support for sysv init to the dhcp-all-interfaces element.
Specifically, it adds a sysv init script and the relevant install/config steps.
Tested with Debian jessie/testing.
Change-Id: I4ae1d5365bafda188cb65ea1fb72d8ffe2408910
I3166951c6fbb2faec53ffc339edd2fd0c3512300 recently landed without two
fixes that I suggested, namely using rm -f rather than testing for
file existance, and a comment fix.
Change-Id: I68e187e20802f3482d4c524a6c435fec7526178e
This change adds extra tests to the
elements/manifests/cleanup.d/01-copy-manifests-dir script to prevent it
from exiting with an error due to non-existant files.
The premature exit condition only seems to happen when an image is being
created without including the "base" element.
Changes
* adding file detection tests for dib_arguments and dib_environment
files
Closes-Bug: #1308224
Change-Id: Iaa8bafb6765241408b9f02d4896913d15f2f401d
This is dependent upon change in tripleo-incubator
I1ff4bb98c99dfe87ccc4fb19767b93e27707d3a7 and concludes the
renaming of deploy to deploy-baremetal
The rename was performed so that each element had a more descriptive
name, and so that each element could accurately set an element-provides
of "deploy", given that the elements enable tripleo deploys via the
ramdisk image.
Change-Id: Ic2ca5f95b8895fa0d6d7314cab9de30a7204541c
Use "deploy" to denote that an element provides the deploy
capability for a ramdisk for tripleo installation purposes.
This will be a 3-part change to enable renaming deploy to
deploy-baremetal so that the generic "deploy" capability can
be used for the provides file, as a dependant change will have to
be made in tripleo-incubator.
The tripleo-incubator change is
I1ff4bb98c99dfe87ccc4fb19767b93e27707d3a7 and that swaps to using
deploy-baremetal in place of deploy for ramdisk creation when using
baremetal.
Change-Id: Ibb4026e8fc8ba6378061461c4796a91ab2fa991f
/etc/lsb-release doesn't always exist in cloud images and doesn't
serve any significant purpose in this element so removing the
dependency entirely.
Change-Id: I4d972074e6e10fea09a875cbc4ed4e60248d46d9
DIB_OFFLINE has wide-ranging effect. This patch adds a tunable that
only turns on the tarball-caching behaviour specifically for the
debian element.
Change-Id: Idd646d7909df6907afc0eeeec1129c3f1a1f92bc
Moves the code that removes default cloud interfaces
into the dhcp-all-interfaces element. Having it live in
stable-interface-names won't work right now because we
need this for virtualized testing.
Change-Id: I3166951c6fbb2faec53ffc339edd2fd0c3512300
“debian” element now source the ${TARGET_ROOT}/.extra_settings file. We
can use this file to adjust its behavior and specifiy what init system
should be used.
Change-Id: Id57ce879341184a57851d2578cf70978f409a48e
Fixes problems found by set -eu and pipefail, including:
-Many unset variables
-Commands that can fail under normal circumstances, which breaks
with set -e. This change swallows those expected errors to allow
our existing error code to handle them.
-The dkms element was not finding Fedora kernel versions correctly.
This may be an issue for other distros too, but since Fedora was
working fine without this functionality I only changed it to print
a warning message rather than failing the build when it happens.
-The ramdisk init script will not be set -eu because if it fails
the result is a kernel panic, which can be tricky to debug.
However, in testing with set -e a few failing commands were found
and have been fixed in this patch.
Change-Id: I44cf98dfc80cfcaec54b88cc83be80a3dbf2cec3
Mariadb package in RDO repository is mariadb-galera-server, but
this name is mapped to MariaDB-Galera-server because of
mariadb installed from mariadb.org.
To avoid mapping conflicts mariadb-rdo uses virtual package name
mariadb-rdo-galera-server which is mapped to mariadb-galera-server.
Change-Id: Idf77d91876baeb81d89eb688808b892ece2bf19a
This was supposed to have been moved to the redhat-common element,
and having it here too is both confusing and may break builds
depending on which version gets used.
Change-Id: Icae5cc633e9e49a9ee63bd9dfc94657c6795c6f1
Use "operating-system" to denote that an element provides that to the
image being created. This can then be used to specify an abstract
dependency in other elements to assist in build-time image verification.
Change-Id: Ic125803da65e0f8f8c20ef90b08815f4f7d09762
* Create a standard element to copy manifests out of the image and
into the build area.
* Ensure all current manifests are copied into a standard manifests
directory
Change-Id: I37eff5a13a14564e1adc33eb4f0144d571267617
The version of depmod in busybox does not work if two versions of
the same kernel version are available, as search paths are not
honoured correctly. However, we don't need to actually call
depmod as it's done during image building anyway.
Change-Id: Idd3cd5854e7c86e7ca1f2b0e50758d7db79ab6e7
Co-Author: matthew.gilliard@hp.com
Closes-bug: #1302856
In the bash script "99-override-default-apt-conf" 'set -u'
is being used to check for uninitialised variables.
This resulted in generic 'unbound variable' messages rather
than the intended error messages.
Tweaked the check for DIB_APT_CONF to work if the variable is unbound;
so that we get the intended error message rather than the `unbound
variable` message.
Change-Id: Ib8e287060f32312ab80dd7f44e2792ecca48edbb
Adds an environment variable DIB_ADD_APT_KEYS pointing to a folder of
apt repo GPG keys. Each key will be verified and installed with apt-key
to the apt keyring.
Change-Id: I4b15347a76aa3241e741931bb94f17598641ae42
The ${TARGET_ROOT} chroot is owned by the root user.
Without this change, the build will fail for a standard user
with a “Permission denied” error.
Change-Id: I6eb87d3a14292da5620bdfd623ea7ef846fe2af6
There is a known issue in legacy grub. When using splashimage with
serial console, Legacy grub wont start properly. This commit is to
ensure that when serial console is used, legacy grub should not
contain splashimage. It also adds serial support to legacy grub
configuration file.
Change-Id: I2007916998dfe7c66ef51bf8c57c1698dc0fcb75
When the repository references are sha1s the fetch command used will
not work. This will ensure that all of the objects are still fetched,
and all of the local cache references for branches and tags will be
updated rather than just the specified reference in use.
Change-Id: Iad2554b3087a5da37dac6e9f6a72432d49fce83d
Closes-Bug: #1301435
--target has been introduced in grub2. This patch ensure we use
the minimal version before adding the --target XXX flag.
Debian Wheezy and Precise share the same limitation regarding the
root device.
Change-Id: I5b3ad42b9fb536d13fb65d84339bc15bc23d99a3
The grub install scripts on Fedora have hardcoded checks for
/sys/firmware/efi and if the directory exists, it uses efi
specific instruction like initrdefi, linuxefi etc. vm element
should fix it once the configuration is generated.
Closes-bug: #1231946
Change-Id: I888304cb9167b8e074b04ca6c5b2f9143b70d37d
This was calling it for all interfaces previously, making the fixes
we'd put in place be ineffective.
Change-Id: I16bf81225a942e3b19a7e745d949e6ddea2fd50e
When high VM churn is encountered dhcp-all-interfaces got tangled up
and leaked instances until upstart run out of filedescriptors.
This happened because we were locking on every invocation, and the
lock logic was disabling the single-interface code - so we always
probed every interface every time - and this was serialized across
every instance of dhcp-all-interfaces.
Change-Id: I4ef1fb83d978cc99d4cb9cc2666bde302c6db7d1
To create an auditable log of packages installed in an image use
dpkg-query to create a json data file with the package names, version
and install status.
Change-Id: I59018221edcb944a876ad47b402538c63c426bc7
The cloud-init package on Debian Linux overwrites the system
hostname to "(None)" unless a hostname is specified in one
of the /etc/cloud/cloud.cfg.d/* files. This change applies
a default hostname of "debian" using cloud-init.
Change-Id: I9b411df44cfa05f899e708900f95719c0bcd6267
Corrects an issue in the config_exists function that could
cause duplicate interfaces to be configured on Ubuntu.
Previously we used 'return ifquery' directly which caused
a silent 'numberic argument required' error to go unnoticed.
This would also return 1 meaning a new interface would get
configured.
The new logic should handle things properly.
Change-Id: I625225e15113d7e184e3bcb5054df1616dec008a
Closes-bug: #1298430
Virtual baremetal instances have their NICs show up as ens4
which doesn't play nicely as a default in devtest at
the moment.
Ideally we'd want it to be em1 like real baremetal
(which is why we put this in the baremetal element
to begin with). Turns out the baremetal element is
required in devtest so we can extract the ramdisk and kernel
so lets leave it alone for now and move this elsewhere
until we get a better solution for fake 'baremetal'
testing.
Closes-bug: #1298152
Change-Id: Ia71e1d32b93db0c4c844a6dc1ebcd04ab0c13f05
