Many python extensions are built from source each time an image is
built. Repeating these is wasteful, so ccache is employed to eliminate
that waste. A cache dir outside the chroot is used to speed up
subsequent image builds.
Change-Id: Ib73563ddbe5f3be7454bfc54ab91cedb559a1304
Before each phase dib-run-parts will pick these up to setup the
environment for the commands it runs.
Change-Id: I56697d7bc370918d4063eee67a1c35d4988a5359
The URL to download the Fedora 19 images is not persistent anymore,
they are now download from http://download.fedoraproject.org where the
request will be redirected to a different location. Passing the -L option
to the curl command will make it redo the request on the new place and
download the image correctly.
Related to bug #1199237
Change-Id: Ib9b9d65734992e2bf1a6888c5f2eb9de366f8204
On Fedora 19, there is a delay between the time loop0p2
is created to the time it is ready to be mounted.
This patches waits for 5 seconds for /dev/mapper/loop0p2
to be accessible. If it is not ready then the script
exits with status 1.
Fixes: bug #1197340
Change-Id: Iff0cfd0566673799a77500e0e24954f5f3743a17
When downloading the base image for the first time, the response
test expression fails with '[: 200: unexpected operator'. This is
caused by using '==' in the test expression, which is not supported
in sh scripts.
Fixes bug 1195030
Change-Id: I66260814cb591371dc5c10f8436f90c2f18d78cf
Add "NONZEROCONF=yes" to /etc/sysconfig/network to prevent
169.254.0.0 route from being created. This route interferes
with fedora baremetal nodes' access to the nova metadata
server at 169.254.169.254.
Change-Id: Iad77abfc97c0eba944680772f233389481a270ff
On fedora 18 the package pyOpenSSL is conflicting with one of the
packages needed to build the bootstack image. The problem gets sorted
if you install pyOpenSSL at the beginning of the image building process
and this is what this patch is doing.
Change-Id: I681683b81ad638dfeb690fde1bc7d08a2116aaed
Use the new source-repository interface to git clone
source repository.
Also there was a few thing in this element that needed to be fixed.
o url to the git repo wasn't curren't.
o pip and git are also dependencies that needed to be installed.
Change-Id: Ieb7be6576fc909aa4bba0233b7b81cab03395d76
Git isn't a dependency of this element, but is a dependency of pbr
if you are doing pip install on a git cloned directoy, because this
looks to be a fairly common pattern it makes sense to have git on the
target image.
Change-Id: I49ee3fa4eac493b9dfa379a674ab9b3390cfc89a
Commit cc5ecb4e9c switched from using
run-parts in favor of dib-run-parts.
/usr/local/bin isn't part of PATH on Ubuntu causing
"/usr/sbin/dib-first-boot: line 8: dib-run-parts: command not found".
Change-Id: Ia7e401221add197edf880af82e0148f3ba7d2b9c
The fedora element downloads images too, so we should re-use the caching
code from the ubuntu element.
There doesn't seem to be other examples of code shared between root.d
scripts. In the fedora and dpkg elements we copy install-packages into
the chroot, but that model doesn't apply when we're running scripts
outside of the chroot. Seems sane to just run it directly from the bin/
dir in the temporary hooks directory.
Change-Id: Iaa6aca660042fea323cab4271633a4bdbbc271b8
If a cached copy of the file doesn't exist, cache_url() passes a
non-existent path to -z/--time-cond and you see this warning:
Warning: Illegal date format for -z, --timecond (and not a file name).
Warning: Disabling time condition. See curl_getdate(3) for valid date syntax.
It works just fine, but the warning is ugly.
Change-Id: Ic6f13a2c596b988308d7fca9cd1745e5d48ae5fb
At least on RedHat the run-parts command is very precarious and doesn't
even check for the return code of the scripts it runs causing the
dib-first-boot to delete itself even when something went wrong with the
configuration of the first boot of the image. Using the dib-run-parts
instead will make the whole first boot routine to be more robust and
less platform dependent.
Change-Id: I0073fd52fc1fc8848ca9277a84603124215cdb6e
Adds a new err_msg function which is used to:
-echo feedback to the deploy ramdisk console
-keep track of the first error message we hit so
that we can send it along to the baremetal-deploy-helper.
Also, updates our wget request back to baremetal-deploy-helper
to include the first the first error message (if any) as
the 'e' parameter.
The err_msg uses a new simple safe_url_encode function to ensure
we don't send invalid characters in our HTTP post requests.
Change-Id: I5a623a6f66cde8d81ff1e75800dc2953ca2703a8
Adds a new 'troubleshooting' function which encapsulates
the things we might want to do when something bad happens.
Typically this is echo'ing a simple message and launching a
bash shell.
The new troubleshooting function also makes use of a new
'troubleshoot' kernel param which if specific will allow you to
hang the deployment process and interactively debug the deploy_ramdisk
via a bash shell on a console.
Troubleshoot is disabled by default.
Fixes LP Bug #1191043.
Change-Id: I2cba8a9674075ba7e420027d40ef8cbe756cf07e
Fixes bug 1184943
Adding an element that allows other elements to register
repositories it would like to be retrieved for it.
Doing this outside of the chroot allows locally cached
repositories to be used. It also gives the ability to d-i-b
to specify what revision to use if an alternative to the
most recent is required.
Effectively allowing a CI system to test d-i-b and elements
without being effected by unavailable git repositories or
breakages in actively developed source code being used by
the elements.
Change-Id: I1527facebaad256a357af680e017b34b1788575d
Updates our deploy element's init script so that it calls
find_disk in a loop until it returns true (meaning we have a disk).
Previously on some bare metal machines find_disk would return ""
which causes the subsequent iscsi target commands to fail due to
a bad backing store.
This commit also updates the error message when start_iscsi_target
fails to be a bit more accurate as well.
Fixes LP Bug #1190984.
Change-Id: I3cd535d6672c197c1c3c539c83bba36be7a14e18
Having all security enabled is not always interesting, e.g. when
developing or testing new elements.
Change-Id: I828ecedc805ce4f89d60d185994eaa9c651d436a
When the target directory is empty, the for loop printing our profile
data was failing because of a glob problem.
Change-Id: I34e612021c70d6b8f75423de0d8be969c52e58e2
This patch will append /usr/local/bin to the sudoers secure_path variable,
by doing so commands prefixed with sudo will then find some binaries
delivered by other elements in that path (e.g nova-rootwrap).
Change-Id: I1ebad8b1932620559a3d90aa39298f9b12e636fd
Bug 1124593 reported that ssh host keys were not being created, and the
lack of seed metadata files was the cause. Add those files and remove
the workaround.
Change-Id: I6fb55be7f8c7efd1f75aeba9123f495e312ff695
The images produced by the diskimage-builder have their filesystem with
a label of "cloudimg-rootfs", so we need to change the default /etc/fstab
on fedora to reflect that.
Change-Id: Id1bb00cb81cb200a114f500e26272624be577da0
This element will install the network modules for dracut and regenerate
the initramfs. The network support would allow us to configure the network
of the image via kernel cmdline instead of injecting files into the image.
Change-Id: I829405127e77aec80b99ac8dd2032cb0c6d81063
After the refactor of the first-boot routine there's no use for this
script anymore, in fact all it's doing is to create a broken link.
Change-Id: Ic09e7208bc950380341e8a073647950ac99a05fd
Fedora's partprobe does not create devices the same way it does on
Ubuntu. Instead of using partprobe, this patch uses kpartx, which
works on both Fedora and Ubuntu.
Change-Id: Iac5cbac083e423425693c8210cd5d058ce4bce72
Updates the new 99-dkms element so that it installs the package
before using it.
Fixes LP Bug #1183785.
Change-Id: Ia191ed3cae707a8c8591f849dc0c6293e849d64a
Work around ordering issues that cause dkms modules to not always
be built for all installed kernels, add a post-install.d script to
the base element that will force the issue.
(Note that any module/kernel pairs with a pre-existing valid
module will not be rebuilt, so this should not impact performance
unnecessarily)
Change-Id: Iccea14c6318014cc7e1fec77e71c14a224f7b76e
Per bug 1182648 we were not setting the mode and as a result couldn't
log in, defeating the purpose of having a local-config element :).
Change-Id: I4114b9536392f1517eac8f4bbe6f4b60ce1186a1
Apply kernel command line parameters to force Linux to remain in text
mode. Typical distro kernels quickly switch into a graphical screenmode
and this is incompatible with common Lights Out Management hardware
which is able to intercept text mode consoles to display them remotely.
Change-Id: Id553972c4fd87e78c9e6fe344331a399913d965e
Setting up a routine to run at first time the system boot may vary from
distribution to distribution. This patch will remove the logic from the
diskimage-builder code and put it into the elements.
The base element will now deploy a dib-first-boot script and each OS
element will ensure that this script runs at rc.local time. By doing
that we will put the OS specific stuff in the OS elements and remove
the embedded code out of the img-function file.
Change-Id: I24c5d1b1185de5693f145347fe912245f1ba7dfe
Boot time was 30 seconds shorter in an all-in-one devstack
environment.
Thanks to Ghe Rivero for a pointer to the solution
https://bugzilla.redhat.com/show_bug.cgi?id=208275
Change-Id: I90d0c96d5659326ba67d6119b96d9a4113adf7fe
With systemd rc.local lives in /etc/rc.d and is only run
if the rc-local service is enabled.
This change creates a /etc/rc.d/rc.local symlink, and modifies img-functions
prepare_first_boot to tolerate an initially missing /etc/rc.local.
Fixes bug #1179674
Change-Id: I7e6102b62224950c1de5d48205dc4feff60f9389
baseline-tools expects lsb_release (although tolerates it not being there)
Order needs to be 01-override-yum-arch, 02-lsb, 03-baseline-tools
hence the renames.
Possibly lsb_release isn't cared enough about to do this.
Change-Id: I03abffb1ab2f560e746ba4ffd407605de31f4930
The later pulls in most of a desktop. The former still pulls in a lot
but significantly less, and still provides lsb_release.
Change-Id: Ie27310f01c037b4a10feb5e380e6f5c4ecf21364
We need to support mellanox devices in disk images as well as ramdisks.
This adds the requisite element structure to provide that support.
Change-Id: Iaf383f5b32b84fbb69569ef42ec2460ab1b231e5
The PPA would have to be updated for each release we want to build for,
including raring. The PPA also gives the tripleo team root on every
built image, something we don't necessarily want.
A shallow review of the packages in the PPA and known elements shows
only one dependency in the tripleo-image-elements heat-cfntools element.
Change-Id: Ic514f929eba26014dd3a0d76d5bb3b8e9e0caf3c
This also switches to using curl which some people may not have
installed. However, curl is far superior for this type of download.
Change-Id: I7ac5a84b30eb8daad320c082f976931c41a24669
Qemu-nbd does not perform well with older versions of qemu due to
the lack of writeback caching mode. It also only builds qcow2 images
and there is a desire for raw image support. Finally, qemu-nbd makes
it very difficult to build images concurrently due to the somewhat
opaque nature of how it selects a /dev/nbd# device. losetup, on
the other hand, makes this process very straight forward.
Change-Id: I309fad8af4fd1e8d1720c17b65e1897a76d5e897
Co-Author: Clint Byrum <clint@fewbar.com>
Update audit in order to fix up a conflict with a file from the glibc
package on a fresh Fedora 18 install.
Change-Id: Ib44c8415bce4ec47e21626a697cf426b96a9061a
Post install scripts are useful because they can perform tasks you want
to handle after the OS/application install but before the first boot
of the image. e.g: Clean the cache left by the package manager reducing
the size of the image.
Change-Id: I03e77f602192bbdce29c02999d1b57fac8051ddc
Fixes: Bug #1145786
This switches $CLOUD_IMAGES and $RELEASE to the DIB_ namespace so
they will survive future changes to the sanitisation of the build
environment.
Change-Id: I7dc2aa82fb9ef452705b080cc404f41046014f20
The root user tries to `sudo -u stack`, without a tty,
causing fedora+devstack images builds to fail in Jenkins.
Change-Id: Ia0a7fb315cf9bd17cf250e70dba06363a697c97c
cloud-init is expected to generate unique hostkeys on first
boot, but sometimes it doesn't.
This change ensures that hostkeys are always generated.
This change should be removed if the cloud-init problem is solved.
Change-Id: I7d7e05fbd12727992cf1aebec547ae0488e002ff
ARM doesn't have a generic Linux image due to the soc-specific nature of Linux
kernels today, so we drop the manual installation of that package, replacing it
with a dist-upgrade instead. This involved tweaks to the dpkg and fedora
install-package scripts.
Change-Id: I97924b80ca87781307e1087b9fe4b18215770e84
Errors in mysqldump would have been missed due to not having pipefail
set. Also there was a type-o in the invocation of mysqldump that was
going undetected, that has been repaired. Finally we should not pass
--master-data if we don't have slave credentials, as the master data,
even if provided, is useless without a valid slave user/pass.
Change-Id: I0f80af34c747f465250ef17fae4a44dac919c9cd
Strict sql_mode settings are very application specific, many will simply
not work with them. This setting was likely inherited from some other
production MySQL config file and is not appropriate for a generic
configuration. Specifically, this broke the MySQL package maintainer
scripts.
Change-Id: I068fcc7b3c770541d3d1ecfb372384750f182584
This will write two files in /etc/ that contain the environment and
command line arguments used in the creation of an image. This should
assist with later efforts to repeat the creation of an image.
Change-Id: Icdbe6693380bed6c406feee10d2cb1a88a992932
Quantum-ovs was continually re-creating interfaces
every second or so, because it was configured to use tunneling,
and the default version of OVS included with ubuntu does
not support tunneling.
Change-Id: I1246181c4b9c96ebe2795b337454dc8ddcb35e8f
Use Heat Metadata to specify host and credentials to pull whole database
snapshot and setup Replication. Also include root credentials for use after
the snapshot has been applied.
Change-Id: Ie3aa92463d28db54a523f520c2b4cdfb528acf9d
There's no need to run os-config-applier at any time except when
os-refresh-config is in its configure state.
Change-Id: Id9758294c60469a976b61c68fc26d9b83130599a
'glance' element now results in a properly-configured,
working glance service on first boot.
While glance-api and glance-reg may need to be broken
out again in the future, the intention now is to focus
on a working service for an end-to-end PoC.
Change-Id: I4fb1cd5ad6defef5a5c4fddfded46d51e8097b5a
Best practice is to use start-stop-daemon rather than sudo, as sudo will
apply user-centric pam limits and create a wtmp entry. Also there is no
need for a script stanza, as we are only execing one command.
Change-Id: I0c2f12536b56d90fd43ab40e74424350efcc0b61
Future work is intended to allow this to serve
as a nova-virt element as well, by allowing the config
and which services are run (such as bm-deploy-helper)
based upon metadata config settings.
Change-Id: I59718a0b329e6adc59097492638cf02c76162c0a
Python package dependency conflicts have been observed
to occur for certain combinations of services at certain
revision.
Running all services in virtualenvs removes the issue.
Change-Id: I100817569b43a5af3427b0ae20cebdc7d55d03a5
Ensures that all network interfaces are present in
/etc/network/interfaces. Any interface not already defined there
will be added and configured for DHCP.
Change-Id: I27e0902e62804e8d719acd7288109bd0e294fd16
Enable templates to specify a list of users that need passwords assigned
in Metadata. These passwords are then communicated back to the template
via an optional wait condition handle using cfn-signal.
Change-Id: Iaaf4d4a9d0d757b7d44ea39e77eed3c55ffffd88
Do nothing if we are booted on a server which has no instance-id set,
such as a non-cloud VM or server.
Change-Id: I6df81edbea1b2b4e4f1ef11a9f2ddaf100d98eda
We didn't need to do most of this anyway, we just needed to configure
mysql before we installed the packages in the image. Some of the other
things were half-baked ideas which will come in later changes to add
os-refresh-config and os-config-applier support to the element.
Change-Id: Ib1b66a8b3b70fac10f3f826b1efd414c67f9bfc6
It is valid to request os-refresh-config without putting any scripts in
the element root. This fixes failing builds of heat-cfntools.
Change-Id: I4f70b33e453399ff5baa48919557995a6eb27520
The code which raises an error if two elements try
to install different versions of the same git repo
currently works correctly for SHAs, or if a branch
is specified first, followed by a SHA.
However it fails if two branches are specified, because
the shallow clone does not fetch all branch heads.
This change correctly handles any combination of
SHAs/branch-heads/tags.
Change-Id: I501e94150ddeb41b79588816576a5d7a06fa92aa
cinder-api in trunk will no longer start with
keystone.middleware, only with keystoneclient.middleware.
The issue is discussed here:
https://bugs.launchpad.net/nova/+bug/1143998
Change-Id: I29f81ca2503957b4df58e86b2392ef4b3c15d6fe
This allows openstack services to be git/pip installed
without installing any startup scripts.
This is useful for keystone-db or nova-db elements, for
example, where the service must be installed to perform
database migration, but no service start scripts need
be installed.
Additionally, add a tool to create openstack sql databases.
Use openstack pypi mirror:
Use the openstack pypi mirror for openstack service
installation. It's much faster than pypi.org.
Also, pip install $svc_repo/tools/pip-requires first,
if it exists, which is required to pick up oslo.config.
Change-Id: I72751d4da59f8597d20aea2f27a9dfabe2f63a8f
Currently, the local-config and stackuser elements
are intertwined. local-config installs an authorized_key
for stackuser, if the stackuser element is in use.
This change does two things:
- add authorized_key for root from local-config element,
regardeless of whether stackuser element is in use.
- install key for stack user from stackuser element,
only if local-config element is in use.
Change-Id: I0d07b61404119ea0650c5c0fb98d6786adcf3ca9
* Provide entry point for Heat Metadata to drive os-refresh-config
* Use os-config-applier for configuring heat cfn tools
Change-Id: Ie38b932173948f1e7c414c48caa1b037916bef84
Copies all templates found in the root of any element into the
default template location.
These templates will be filled in from heat metadata by
os-config-applier whenever it is run.
Update keystone to install templates in this way as an example.
Change-Id: I0be0a79a431e9ba5b80e84f130c48d5ce8b100ae
Co-Authored-By: Tim Miller <tim.miller.0@gmail.com>
Also add redhat-lsb to Fedora images for lsb_release which should prove
useful as a way to do distro-specific things.
Change-Id: Ie32566349319ca244fa02065bb9f81c36c1b49fb
Relies on https://cloud-images.ubuntu.com being served by a cert signed
by one of the CA's trusted by the build host.
Change-Id: I690b755acca54789110c2c8fa723c8b87b2485c9
This element installs all openstack sql databases into
a single image, useful for a bootstrap image.
Change-Id: I6efe807793ec620a60d5798d28a7e3103a5b90e8
Then creates symlinks into /opt/aws/bin using the included script.
This replaces the heat-jeos element, so anything that references heat-jeos will need to be replaced with heat-cfntools
Change-Id: Id286b005f69ea364357303b4bbc1ed29d4f0f8f9
Also modified dib-run-parts to apply a more workable solution for
filtering out unwanted files such as editor backups and VCS.
The script is installed in its own element, depended on by the OS
specific ubuntu element. This is because the ubuntu element (and
later other OS's) are responsible for populating the root filesystem.
If we try to install this in base, the root filesystem will look to
be populated already and we will skip automatically choosing ubuntu.
Change-Id: I017646748c1a8360299106289b57d976d45875a8
This includes the install-packages implementation for dpkg, apt http proxy
config, daemon blocking and unblocking.
Change-Id: I8f159021d2b223d7003cec067de3aa605ad06974
This is very helpful when performing automation.
Additionally, removed addition of stack user to admin group,
since that group does not exist on all systems, and
passwordless sudo eliminates the need it was addressing.
Change-Id: I727bd51acb920c6c2e66247069998b6ee4267895
Move common openstack service installation operations
into a new script `os-svc-install`, which simplifies
the elements for openstack services.
Change-Id: Ied8ac3278e7fe8af76e24748ec4e598a84afa03c
Disables cloud-init from querying EC2 metadata, or any cloud metadata
service. It keeps only the Nocloud and None data sources, which are both
expected to be "baked in" to an image. This speeds up boot by 2 minutes
on hosts where cloud-init cannot reach the EC2 metadata service.
Change-Id: I43522c4a442c0f2fc2044d4898768ceb4f839099
os-refresh-config simply runs things in a predictable order in directories
that other elements can populate. This element installs it and sets it
up to run at boot.
We also modify os-config-applier element to be called by this rather
than to have its own upstart job. This provides an example of how other
elements can insert themselves into the refresh process.
Change-Id: I52d72b7fd302ec317840af87fa77299387af99ad
See https://bugs.launchpad.net/heat/+bug/1122472 for information on the
specific problem in heat. This can be removed when that bug is resolved.
Change-Id: I76a7a42dea9472c54f03593c8c67856aa23f2c73
This is a necessary but not complete step towards supporting Fedora and Suse
distributions. Further work is needed (e.g. to quiesce daemons on
installation).
Change-Id: If3ea6093d41a21de755db52328226b84b5a3ede6
Remove the explicit installation of an English language pack and instead
just default LANG to C. This settls Perl's noisy warnings and also stops
any side effects of different language build hosts.
Note that this is set right at the start of the build process, so it
should be entirely possible to override if needs be.
Change-Id: Id3b31162d4198fa02dc5a4d11168e57dbcd14a5d
This disables apt's default behaviour of installing packages in the
Recommends field of other packages. While this can be useful, it
causes a lot of unnecessary software to be installed and may cause
explicit dependencies to be missed in some circumstances.
Change-Id: I10f4519290e6b3dafda3365c0c5dcc42ad17c090
This change causes the "base" element to install and generate English
locales, thereby mitigating a lot of build noise (at least for
English speaking users). Also included is a note in the element's
README.md about what users in other locales should do.
Change-Id: I053c7e0254a0a0d7dde081f7cc72204a12aa8ef4
Fix whitespace error in local-config/install.d/61-http-proxy that prevents
no-proxy being written.
Change-Id: Ie2511bb5b2504c42ff59ec277cd91bf92b56da59
Fix a typo in the base element itself.
Refer to it early in the README since it is likely to be useful to
almost everyone.
Change-Id: I4a753b61542f8a354ade7fcc4436076aec61e6c0
