The yum element provides for specific customizations
for yum based distros. The included customization is
a yum cache mounted outside of the chroot so that yum
downloaded packages are reused on subsequent image builds.
Change-Id: I6833c9fdbc83cb09debec6a789082e105c917800
The dynamic kernel module system is not available on RHEL, CentOS,
Scientific Linux, or SUSE. Make it part of the distro post-install
rather then base post-install.
Change-Id: Ic2c345bf9f0738dadae611194e263d3a5d424a3e
- pip install to a github URL without a git+ prefix fails.
- Need to close EOF heredoc when creating mirror.yaml
Change-Id: Ic555174881e9c06fede1cf731141cbe6eb0e0b02
Using a custom pypi mirror can be very convenient, making image builds
substantially faster - because we create multiple virtual
environments we benefit more than single-virtualenv users would.
Change-Id: I997daf1f9477c447e1fb30818aea9e80a49b31a6
Manually listing the mount points that need cleanup is not
maintainable and makes it hard to write elements that use bind
mounting to inject resources into the build environment.
Change-Id: I7c9ade444f4ebe42552f8e321f257a7ec0a674ef
The images produced by the diskimage-builder have their filesystem with
a label of "cloudimg-rootfs", The regex changing fstab on fedora didn't
match the F19 case.
This regex matches both
F18 :
LABEL=_/ / ext4 defaults 1 1
F19 :
UUID=f2da0fc8-2e12-4a6e-8e95-a31a474aa1a8 / ext4 defaults,noatime 0 0
Change-Id: Iafcf5e069199f1df76874309b2d5064f343fb5e5
We try to expose metadata to users about what dib included in the
image and source-repositories was not doing that, so this fixes it. It
also permits easy introspection from other elements (such as an
upcoming PyPI cache layer).
Change-Id: I1e48e65f69ebe1f4556cc3bf35ccf195e99bcc76
Ubuntu 13.04 has been released now for 3 months. The updated libvirt,
openvswitch and kernel are all beneficial to various OpenStack components,
and many other software is updated beyond the versions in Ubuntu 12.10.
Change-Id: I358aed8bf906c3ff5103f19b1f9e6ac689b5d5ee
source-repositories uses find to discover things it should import. If
someone happens to use emacs, and happens to have had an autosave file
in a directory with a wildly broken version of a source-repositories-
file, then dib will attempt to execute the contents of the autosave file
and the user will be sad. Adding a filter to the find command for files
ending in ~ will prevent pain and suffering.
Change-Id: I27c8a4b53d3c9b913ca926db16b4271941f6fb5d
devstack-gate needs to be able to make an image that has pre-downloaded
but not installed a sequence of packages. To support that, add a -d flag
to install-packages to allow downloading but not installing. -d passes
through on a dpkg install, and with this patch should be transformed
to --downloadonly which is the yum version.
Change-Id: Ia4e61fa4304df86afbee7dfc8067ea1bdf528f3e
Symlink /usr/lib -> ../lib for Fedora deployment ramdisk. This
symlink is needed for systemd-udevd. Otherwise, the network device is
not loaded and initialized when the deployment image is booted. This
logic was previously contained in
elements/deploy/root.d/50-redhat-prepare-fs, but now that ramdisk
images are built in a chroot, the change needs to be moved so that it
is applied to the ramdisk build.
Change-Id: Icea43230126956ccf5fb8a6a96ca706b75d5c32f
Create an install hook where core utilities can be installed
on Fedora. In this case we are installing which.
Which is being used by
ramdisk/post-install.d/01-ensure-binaries
vm/finalise.d/51-grub
The absence to which caused the ramdisk build to fail on F19.
On Ubuntu which is installed with the package debianutils, this pretty
much has to exist on Ubuntu images, so a corresponding package install for
ubuntu isn't needed.
Fixes Bug 1202612
Change-Id: I847c1619cdab021c9348caed3fa32db64e663e8e
- Ensures /sbin and friends are in $PATH when invoked (without this,
various sudo invocations fail in exciting ways).
- Use dib-run-parts in lib/common-functions instead of run-parts
(neither SLES nor openSUSE ship run-parts).
- Ensure dib-run-parts doesn't descend into subdirectories (same
behaviour as run-parts).
- Move dib-run-parts from root.d to bin (cleaner, consistent with
other elements with separate bin scripts).
- Tested by building Ubuntu image on openSUSE 12.3.
- Note: this doesn't add support for creating SUSE images, it just
lets you run disk-image-create on SUSE-based distros.
Change-Id: I906c6bc3cf51cdf2c4415adeae1ca250faac25e1
The search and replace that converted dashes in REPONAME only converted
the first dash. They should all be converted.
Change-Id: I76a48cf5bdfa8664f0a65d0238fa4da7d80305db
source-repositories does a git reset on the .git directory of cached
repositories. But doesn't specify the directory to reset. A working
directory needs to be specified so that the $PWD isn't used.
Without this change $PWD is polluted with the contents of repositories
being cached.
Change-Id: Ic37b702ac579bf766bb2204a988fa9468d308abf
Caching PyPI objects will require inspecting the content downloaded by
the source repositories, so needs to run after it.
Change-Id: I551c075e0bb6853b2ef79268b7f6455a4f06743d
Cloning large repositories over the internet can take considerable
time. Caching them locally makes repeated image builds significantly
faster, so lets do that.
When users override the element source they will often be using a
local repository, so in those cases I don't cache - but we could
easily change our minds on that in the future.
Change-Id: I9822362cf722b904d9806dbbb4bb07cfe2b33437
Similar to the Ubuntu element the Fedora element had not been updated
to honour --offline. Also similarly we only check for the existence of
the final cached file rather than the image we directly download,
because they are generated separately.
The -u change is just hygiene to detect future buggy changes.
Change-Id: Ib92a9d419bb485c8861da041066827e8d6aac0b7
When --offline is set elements should not revalidate cached data. The
ubuntu element had not been updated to match this. SHA checking is
also skipped as we only move a new cached file into place when the
hash matches, and we might download a new hash before updating the
image cache, which would cause persistent --offline failures.
Change-Id: If1a0366b51951a73b7a3ffe23a29a3d910b08938
In the 52-force-text-mode-console hook, gracefully exit when the
/boot/grub2 directory does not exist on the Fedora 19 cloud image
filesystem. By default the Fedora 19 cloud image is using extlinux to boot
the image instead of GRUB. The decision was taken because GRUB is quite
big (would pull in ~30MB of dependencies) and there's very little benefit
in using it because cloud images don't need to cover all the corner cases.
Change-Id: I52723d4d14c21b1787b4c7e0dd21a09cdbdae0d1
If you have an EFI host but the image doesn't have the EFI modules
installed the grub-install command will fail in case the --target
argument is not set. The problem is that the grub-install script will
check whether the /sys/firmware/efi file exists or not to determine if
it's an EFI installation, but this value comes from the host, so this
patch will look for the /sys/firmware/efi file and will also check if
the EFI modules are installed in the image, if not set the --target to
a non-efi platform.
Change-Id: I4481b43e4a8fe4144be9c7eb9d9c618bbb2df21e
In the 51-grub hook, after the GRUB installation, the script will look
for a GRUB configuration file and in case it's not present one will
be generated using the grub-mkconfig command. The reason why it have
to be done is because the new Fedora 19 cloud images is using extlinux
by default.
Change-Id: I80b15b3122698d98ac4d47dc06faf5909a90ab00
Ramdisks are now built inside a chroot which is built by the normal
image build process. Doing so improves our independence of the
precise state of the build host.
This fixes bug 1194055.
Change-Id: Ibc254fbb9e7b404b5f38c1b35bcde8a4136e8e28
Previously debconf was used and thus this only worked on dpkg based
systems. Now we can support any system that runs a reasonably recent
cloud-init.
Change-Id: I1397aefb3af400adcfd98f858c5a32fad2b584b5
Latest Fedora cloud images are available here
http://cloud.fedoraproject.org/
With this change, when a Fedora version comes up,
d-i-b will point to it automatically.
Fixes 1199237
(Also, thanks to Lucas Gomes for testing.)
Change-Id: Ifcb8a48f3baf3ef841f00e6a20c6fd9f79fea129
Many python extensions are built from source each time an image is
built. Repeating these is wasteful, so ccache is employed to eliminate
that waste. A cache dir outside the chroot is used to speed up
subsequent image builds.
Change-Id: Ib73563ddbe5f3be7454bfc54ab91cedb559a1304
Before each phase dib-run-parts will pick these up to setup the
environment for the commands it runs.
Change-Id: I56697d7bc370918d4063eee67a1c35d4988a5359
The URL to download the Fedora 19 images is not persistent anymore,
they are now download from http://download.fedoraproject.org where the
request will be redirected to a different location. Passing the -L option
to the curl command will make it redo the request on the new place and
download the image correctly.
Related to bug #1199237
Change-Id: Ib9b9d65734992e2bf1a6888c5f2eb9de366f8204
On Fedora 19, there is a delay between the time loop0p2
is created to the time it is ready to be mounted.
This patches waits for 5 seconds for /dev/mapper/loop0p2
to be accessible. If it is not ready then the script
exits with status 1.
Fixes: bug #1197340
Change-Id: Iff0cfd0566673799a77500e0e24954f5f3743a17
When downloading the base image for the first time, the response
test expression fails with '[: 200: unexpected operator'. This is
caused by using '==' in the test expression, which is not supported
in sh scripts.
Fixes bug 1195030
Change-Id: I66260814cb591371dc5c10f8436f90c2f18d78cf
Add "NONZEROCONF=yes" to /etc/sysconfig/network to prevent
169.254.0.0 route from being created. This route interferes
with fedora baremetal nodes' access to the nova metadata
server at 169.254.169.254.
Change-Id: Iad77abfc97c0eba944680772f233389481a270ff
On fedora 18 the package pyOpenSSL is conflicting with one of the
packages needed to build the bootstack image. The problem gets sorted
if you install pyOpenSSL at the beginning of the image building process
and this is what this patch is doing.
Change-Id: I681683b81ad638dfeb690fde1bc7d08a2116aaed
Use the new source-repository interface to git clone
source repository.
Also there was a few thing in this element that needed to be fixed.
o url to the git repo wasn't curren't.
o pip and git are also dependencies that needed to be installed.
Change-Id: Ieb7be6576fc909aa4bba0233b7b81cab03395d76
Git isn't a dependency of this element, but is a dependency of pbr
if you are doing pip install on a git cloned directoy, because this
looks to be a fairly common pattern it makes sense to have git on the
target image.
Change-Id: I49ee3fa4eac493b9dfa379a674ab9b3390cfc89a
Commit cc5ecb4e9c switched from using
run-parts in favor of dib-run-parts.
/usr/local/bin isn't part of PATH on Ubuntu causing
"/usr/sbin/dib-first-boot: line 8: dib-run-parts: command not found".
Change-Id: Ia7e401221add197edf880af82e0148f3ba7d2b9c
The fedora element downloads images too, so we should re-use the caching
code from the ubuntu element.
There doesn't seem to be other examples of code shared between root.d
scripts. In the fedora and dpkg elements we copy install-packages into
the chroot, but that model doesn't apply when we're running scripts
outside of the chroot. Seems sane to just run it directly from the bin/
dir in the temporary hooks directory.
Change-Id: Iaa6aca660042fea323cab4271633a4bdbbc271b8
If a cached copy of the file doesn't exist, cache_url() passes a
non-existent path to -z/--time-cond and you see this warning:
Warning: Illegal date format for -z, --timecond (and not a file name).
Warning: Disabling time condition. See curl_getdate(3) for valid date syntax.
It works just fine, but the warning is ugly.
Change-Id: Ic6f13a2c596b988308d7fca9cd1745e5d48ae5fb
At least on RedHat the run-parts command is very precarious and doesn't
even check for the return code of the scripts it runs causing the
dib-first-boot to delete itself even when something went wrong with the
configuration of the first boot of the image. Using the dib-run-parts
instead will make the whole first boot routine to be more robust and
less platform dependent.
Change-Id: I0073fd52fc1fc8848ca9277a84603124215cdb6e
Adds a new err_msg function which is used to:
-echo feedback to the deploy ramdisk console
-keep track of the first error message we hit so
that we can send it along to the baremetal-deploy-helper.
Also, updates our wget request back to baremetal-deploy-helper
to include the first the first error message (if any) as
the 'e' parameter.
The err_msg uses a new simple safe_url_encode function to ensure
we don't send invalid characters in our HTTP post requests.
Change-Id: I5a623a6f66cde8d81ff1e75800dc2953ca2703a8
Adds a new 'troubleshooting' function which encapsulates
the things we might want to do when something bad happens.
Typically this is echo'ing a simple message and launching a
bash shell.
The new troubleshooting function also makes use of a new
'troubleshoot' kernel param which if specific will allow you to
hang the deployment process and interactively debug the deploy_ramdisk
via a bash shell on a console.
Troubleshoot is disabled by default.
Fixes LP Bug #1191043.
Change-Id: I2cba8a9674075ba7e420027d40ef8cbe756cf07e
Fixes bug 1184943
Adding an element that allows other elements to register
repositories it would like to be retrieved for it.
Doing this outside of the chroot allows locally cached
repositories to be used. It also gives the ability to d-i-b
to specify what revision to use if an alternative to the
most recent is required.
Effectively allowing a CI system to test d-i-b and elements
without being effected by unavailable git repositories or
breakages in actively developed source code being used by
the elements.
Change-Id: I1527facebaad256a357af680e017b34b1788575d
Updates our deploy element's init script so that it calls
find_disk in a loop until it returns true (meaning we have a disk).
Previously on some bare metal machines find_disk would return ""
which causes the subsequent iscsi target commands to fail due to
a bad backing store.
This commit also updates the error message when start_iscsi_target
fails to be a bit more accurate as well.
Fixes LP Bug #1190984.
Change-Id: I3cd535d6672c197c1c3c539c83bba36be7a14e18
Having all security enabled is not always interesting, e.g. when
developing or testing new elements.
Change-Id: I828ecedc805ce4f89d60d185994eaa9c651d436a
When the target directory is empty, the for loop printing our profile
data was failing because of a glob problem.
Change-Id: I34e612021c70d6b8f75423de0d8be969c52e58e2
This patch will append /usr/local/bin to the sudoers secure_path variable,
by doing so commands prefixed with sudo will then find some binaries
delivered by other elements in that path (e.g nova-rootwrap).
Change-Id: I1ebad8b1932620559a3d90aa39298f9b12e636fd
Bug 1124593 reported that ssh host keys were not being created, and the
lack of seed metadata files was the cause. Add those files and remove
the workaround.
Change-Id: I6fb55be7f8c7efd1f75aeba9123f495e312ff695
The images produced by the diskimage-builder have their filesystem with
a label of "cloudimg-rootfs", so we need to change the default /etc/fstab
on fedora to reflect that.
Change-Id: Id1bb00cb81cb200a114f500e26272624be577da0
This element will install the network modules for dracut and regenerate
the initramfs. The network support would allow us to configure the network
of the image via kernel cmdline instead of injecting files into the image.
Change-Id: I829405127e77aec80b99ac8dd2032cb0c6d81063
After the refactor of the first-boot routine there's no use for this
script anymore, in fact all it's doing is to create a broken link.
Change-Id: Ic09e7208bc950380341e8a073647950ac99a05fd
Fedora's partprobe does not create devices the same way it does on
Ubuntu. Instead of using partprobe, this patch uses kpartx, which
works on both Fedora and Ubuntu.
Change-Id: Iac5cbac083e423425693c8210cd5d058ce4bce72
Updates the new 99-dkms element so that it installs the package
before using it.
Fixes LP Bug #1183785.
Change-Id: Ia191ed3cae707a8c8591f849dc0c6293e849d64a
Work around ordering issues that cause dkms modules to not always
be built for all installed kernels, add a post-install.d script to
the base element that will force the issue.
(Note that any module/kernel pairs with a pre-existing valid
module will not be rebuilt, so this should not impact performance
unnecessarily)
Change-Id: Iccea14c6318014cc7e1fec77e71c14a224f7b76e
Per bug 1182648 we were not setting the mode and as a result couldn't
log in, defeating the purpose of having a local-config element :).
Change-Id: I4114b9536392f1517eac8f4bbe6f4b60ce1186a1
Apply kernel command line parameters to force Linux to remain in text
mode. Typical distro kernels quickly switch into a graphical screenmode
and this is incompatible with common Lights Out Management hardware
which is able to intercept text mode consoles to display them remotely.
Change-Id: Id553972c4fd87e78c9e6fe344331a399913d965e
Setting up a routine to run at first time the system boot may vary from
distribution to distribution. This patch will remove the logic from the
diskimage-builder code and put it into the elements.
The base element will now deploy a dib-first-boot script and each OS
element will ensure that this script runs at rc.local time. By doing
that we will put the OS specific stuff in the OS elements and remove
the embedded code out of the img-function file.
Change-Id: I24c5d1b1185de5693f145347fe912245f1ba7dfe
Boot time was 30 seconds shorter in an all-in-one devstack
environment.
Thanks to Ghe Rivero for a pointer to the solution
https://bugzilla.redhat.com/show_bug.cgi?id=208275
Change-Id: I90d0c96d5659326ba67d6119b96d9a4113adf7fe
With systemd rc.local lives in /etc/rc.d and is only run
if the rc-local service is enabled.
This change creates a /etc/rc.d/rc.local symlink, and modifies img-functions
prepare_first_boot to tolerate an initially missing /etc/rc.local.
Fixes bug #1179674
Change-Id: I7e6102b62224950c1de5d48205dc4feff60f9389
baseline-tools expects lsb_release (although tolerates it not being there)
Order needs to be 01-override-yum-arch, 02-lsb, 03-baseline-tools
hence the renames.
Possibly lsb_release isn't cared enough about to do this.
Change-Id: I03abffb1ab2f560e746ba4ffd407605de31f4930
The later pulls in most of a desktop. The former still pulls in a lot
but significantly less, and still provides lsb_release.
Change-Id: Ie27310f01c037b4a10feb5e380e6f5c4ecf21364
We need to support mellanox devices in disk images as well as ramdisks.
This adds the requisite element structure to provide that support.
Change-Id: Iaf383f5b32b84fbb69569ef42ec2460ab1b231e5
The PPA would have to be updated for each release we want to build for,
including raring. The PPA also gives the tripleo team root on every
built image, something we don't necessarily want.
A shallow review of the packages in the PPA and known elements shows
only one dependency in the tripleo-image-elements heat-cfntools element.
Change-Id: Ic514f929eba26014dd3a0d76d5bb3b8e9e0caf3c
This also switches to using curl which some people may not have
installed. However, curl is far superior for this type of download.
Change-Id: I7ac5a84b30eb8daad320c082f976931c41a24669
Qemu-nbd does not perform well with older versions of qemu due to
the lack of writeback caching mode. It also only builds qcow2 images
and there is a desire for raw image support. Finally, qemu-nbd makes
it very difficult to build images concurrently due to the somewhat
opaque nature of how it selects a /dev/nbd# device. losetup, on
the other hand, makes this process very straight forward.
Change-Id: I309fad8af4fd1e8d1720c17b65e1897a76d5e897
Co-Author: Clint Byrum <clint@fewbar.com>
Update audit in order to fix up a conflict with a file from the glibc
package on a fresh Fedora 18 install.
Change-Id: Ib44c8415bce4ec47e21626a697cf426b96a9061a
Post install scripts are useful because they can perform tasks you want
to handle after the OS/application install but before the first boot
of the image. e.g: Clean the cache left by the package manager reducing
the size of the image.
Change-Id: I03e77f602192bbdce29c02999d1b57fac8051ddc
Fixes: Bug #1145786
This switches $CLOUD_IMAGES and $RELEASE to the DIB_ namespace so
they will survive future changes to the sanitisation of the build
environment.
Change-Id: I7dc2aa82fb9ef452705b080cc404f41046014f20
The root user tries to `sudo -u stack`, without a tty,
causing fedora+devstack images builds to fail in Jenkins.
Change-Id: Ia0a7fb315cf9bd17cf250e70dba06363a697c97c
cloud-init is expected to generate unique hostkeys on first
boot, but sometimes it doesn't.
This change ensures that hostkeys are always generated.
This change should be removed if the cloud-init problem is solved.
Change-Id: I7d7e05fbd12727992cf1aebec547ae0488e002ff
ARM doesn't have a generic Linux image due to the soc-specific nature of Linux
kernels today, so we drop the manual installation of that package, replacing it
with a dist-upgrade instead. This involved tweaks to the dpkg and fedora
install-package scripts.
Change-Id: I97924b80ca87781307e1087b9fe4b18215770e84
Errors in mysqldump would have been missed due to not having pipefail
set. Also there was a type-o in the invocation of mysqldump that was
going undetected, that has been repaired. Finally we should not pass
--master-data if we don't have slave credentials, as the master data,
even if provided, is useless without a valid slave user/pass.
Change-Id: I0f80af34c747f465250ef17fae4a44dac919c9cd
Strict sql_mode settings are very application specific, many will simply
not work with them. This setting was likely inherited from some other
production MySQL config file and is not appropriate for a generic
configuration. Specifically, this broke the MySQL package maintainer
scripts.
Change-Id: I068fcc7b3c770541d3d1ecfb372384750f182584
This will write two files in /etc/ that contain the environment and
command line arguments used in the creation of an image. This should
assist with later efforts to repeat the creation of an image.
Change-Id: Icdbe6693380bed6c406feee10d2cb1a88a992932
Quantum-ovs was continually re-creating interfaces
every second or so, because it was configured to use tunneling,
and the default version of OVS included with ubuntu does
not support tunneling.
Change-Id: I1246181c4b9c96ebe2795b337454dc8ddcb35e8f
Use Heat Metadata to specify host and credentials to pull whole database
snapshot and setup Replication. Also include root credentials for use after
the snapshot has been applied.
Change-Id: Ie3aa92463d28db54a523f520c2b4cdfb528acf9d
There's no need to run os-config-applier at any time except when
os-refresh-config is in its configure state.
Change-Id: Id9758294c60469a976b61c68fc26d9b83130599a
'glance' element now results in a properly-configured,
working glance service on first boot.
While glance-api and glance-reg may need to be broken
out again in the future, the intention now is to focus
on a working service for an end-to-end PoC.
Change-Id: I4fb1cd5ad6defef5a5c4fddfded46d51e8097b5a
