qemu-img convert can take a few minutes to complete, or far longer
on heavily loaded systems. Notify the user that the image is
undergoing conversion to save them just seeing END PROFILING.
Change-Id: I1ad61ce5ed011b721d48e2d12fb42ef4aa5cd1f6
Finding the Debian boot kernel and initrd is collapsed into a
single statement so that the script doesn't bail out when
'set -o pipefail' is set and the *generic kernel and initrd are
absent.
Change-Id: Ifd616818f7387e6f102636c7cf186be6097f1d5c
Closes-Bug: 1335016
For some of the scripts, the unversioned name is not
good enough. so determine the link target of "vmlinuz" and
"initrd" to get the full versioned path.
Change-Id: I52f9b5435fafaf7925e2332058a9e41b26478f1c
Add a --options flag that allows disk-image-create to send custom
options to qemu-img calls. It is useful for forcing compat into
qemu-img created images.
Usage:
disk-image-create --qemu-img-options compat=0.10 base ubuntu
Change-Id: I73ee2c86abc115220d51ffebbbe9ce3c4d92f188
Closes-Bug: #1329746
The script to find best kernel from image (used by vm
and baremetal element) is duplicated and is not in sync
with each other. Moving the code to img-functions as a
function will reduce duplicated code and make it reusable
in future.
Since img-functions is not accessible in chroot env,
kernel selection is being moved from finalise.d to
cleanup.d in the vm element.
Change-Id: I8fbccc13a2c61a5191ef9ea5d2a8302a3e43b000
In some instances, the content of the image may want to convey specific
resolv.conf settings. For instance, if unbound is installed, it's also
desirable to set resolv.conf to point to it.
Short circuit diskimage-builder's friendliness around resolv.conf files
by not dying in a fire if it finds the file to be immutable.
Change-Id: I88632fb79289681dcf95d32f39a0ad658ba39a60
save_image is used to copy kernel and ramdisks out of the image, which
we will sometimes want to keep the source, and sometimes not. However
for the main image itself, the temp copy is never kept, so use mv
rather than cp and avoid the excess IO.
Change-Id: I5a9f0d69ffee3e6b872a8927537ac17f02f5aa4d
disk-image-get-kernel is pretty noisy and you see e.g.:
$> load-image overcloud-compute.qcow2
XXX -d '/tmp/image.lWGCgPoj' -o 'tmp' -i '/home/stack/overcloud-compute.qcow2' --
Extracting kernel + ramdisk from /home/stack/overcloud-compute.qcow2 and writing them to /tmp/image.lWGCgPoj
nbd 17554 0
nbd 17554 0
basename: missing operand
Try 'basename --help' for more information.
/dev/nbd0 disconnected
tmp-vmlinuz,tmp-initrd
Clean all this up so we just get:
$> load-image overcloud-compute.qcow2
Extracting kernel + ramdisk from /home/stack/overcloud-compute.qcow2 to tmp-vmlinuz and tmp-initrd in /tmp/image.g6b0lG88
Change-Id: I8971ec0bbcd87157b07fc17254c56bb9f9f2a597
Commit c7d80dd (Cleanup mount points automatically) removed the unmount
of $TMP_MOUNT_PATH/tmp/ccache in run_d_in_target() and moved the
"rm /tmp/ccache" to elements/base/finalise.d/02-remove-ccache. There
are two problems with this:
1) Not unmounting at the end of run_d_in_target() results in tmp/ccache
being bind mounted muliple times on top of itself (three times, if you
just run `disk-image-create base`). It is eventually unmounted, but
somehow the auto unmount code is confused, and tries to unmount it
one more time than it was mounted, which results in an error like
"umount: /tmp/image.THQkZxQa/mnt/tmp/ccache: not mounted".
This doesn't actually break anything, but it's a little messy.
2) "rm /tmp/ccache" in elements/base/finalise.d/02-remove-ccache never
succeeds in removing /tmp/ccache, because that hook is invoked by
run_d_in_target(), *while* /tmp/ccache is mounted.
This present commit solves the above by moving the ccache setup glue out
of img-functions and into the base element's root.d. This has the
following implications:
1) lib/img-functions is a little cleaner.
2) /tmp/ccache is available in the chroot during the root, extra-data,
pre-install, install and post-install stages. It is not available
during block-device, finalise and cleanup stages as it will have been
automatically unmounted by then.
3) /tmp/ccache won't be setup if you're building an image that doesn't
include the base element.
Change-Id: Ief4c0a6f4ec622db6c6f652776215684178d8943
A problem with unmounting the dev filesystem in Ubuntu images caused
the umount of the /dev bind mount to fail, which left it there to be
removed during the mv -t step, causing the build host's /dev to be
wiped out.
The lazy umount will detach it from the filesystem hierarchy and then
clean up the mount reference later.
Change-Id: I8f8cea857c445fb0b4fd02bc063722fb1553c947
When compressing an image, this is done in the same dir where the raw
image resides, doubling the amount of space needed (scarce when
using tmpfs), and then it's moved to the .cache folder in disk.
Combining these two functions, we reduce the amount of space needed
in the tmpfs partition (when in use), and the compressed image is
created directly on the .cache folder disk, so there is no need to
move the compressed image after the process into disk.
Change-Id: I451d24bdd6fa0983414244135dff5e96c0549833
here is a bug associated with this patch because there is a
workaround included. Calling sync to enable the unmounting...
https://bugs.launchpad.net/diskimage-builder/+bug/1214388
Change-Id: I9d5a1f092fd8f3ae1c14de03dd516203baab4da3
Manually listing the mount points that need cleanup is not
maintainable and makes it hard to write elements that use bind
mounting to inject resources into the build environment.
Change-Id: I7c9ade444f4ebe42552f8e321f257a7ec0a674ef
A user running di-b several times while developing an element may not
want to drop to a shell in all cases but may only want to do so if one
of their in target hooks failed.
This patch gives them the ability to do so, If break=after-error is set
then a user will be provided a in target shell taking over from where the
last failed command left off.
Change-Id: Ia2f7ac4c21b64b971f87f4ae9cb867981b13eb5e
Ramdisks are now built inside a chroot which is built by the normal
image build process. Doing so improves our independence of the
precise state of the build host.
This fixes bug 1194055.
Change-Id: Ibc254fbb9e7b404b5f38c1b35bcde8a4136e8e28
Many python extensions are built from source each time an image is
built. Repeating these is wasteful, so ccache is employed to eliminate
that waste. A cache dir outside the chroot is used to speed up
subsequent image builds.
Change-Id: Ib73563ddbe5f3be7454bfc54ab91cedb559a1304
The finalise_base function was creating a broken link at /etc/resolv.conf
when building a non ubuntu image.
Change-Id: I974d43fd6db365dbff9c85195e3b72950f3409da
This is simply a prerequisite and we should not try to
apt-get install it inline.
Remove this code and exit with an error if qemu-img is
not found.
Change-Id: I52d47a95d24d4bfe801ba664200604bd385dc67f
Setting up a routine to run at first time the system boot may vary from
distribution to distribution. This patch will remove the logic from the
diskimage-builder code and put it into the elements.
The base element will now deploy a dib-first-boot script and each OS
element will ensure that this script runs at rc.local time. By doing
that we will put the OS specific stuff in the OS elements and remove
the embedded code out of the img-function file.
Change-Id: I24c5d1b1185de5693f145347fe912245f1ba7dfe
With systemd rc.local lives in /etc/rc.d and is only run
if the rc-local service is enabled.
This change creates a /etc/rc.d/rc.local symlink, and modifies img-functions
prepare_first_boot to tolerate an initially missing /etc/rc.local.
Fixes bug #1179674
Change-Id: I7e6102b62224950c1de5d48205dc4feff60f9389
Sudo compiled with the --with-secure-path option enabled (default on
Ubuntu, Fedora and potentially other distros) will cause sudo to ignore
any changes made to the PATH variable in the user's environment (even when
the -E option is used). Also, some distros might not have /usr/local/bin
set in their sudoers secure_path causing the diskimage-builder to fail
when building an image using such distros as host. This patch will force
the inclusion of the /usr/local/bin in the PATH enviroment variable
inside the chroot.
Change-Id: Idb7156d8ff124bfe685cc721337d9bb21f49dc9e
Fixes: bug #1175980
Ramdisk-image-create sources both img-functions and ramdisk-functions
causing duplicate functions. To correct this situation this patch
removes the import of img-functions from ramdisk-image-create.
It moves cleanup_dirs and run_d functions from img-functions to
common-functions to allow this.
fixes bug 1175427
Authored-by: Chris Krelle <nobodycam@gmail.com>
Change-Id: I971d019b0d92b06d9de661663b49c433e823ea42
For machines with low RAM (or no access to /proc/meminfo) the builder
will still run, but will just build inside the filesystem that is
hosting /tmp. This will result in a slower build (especially if there
are a lot of .deb packages installed).
Fixes bug #1175453
Change-Id: I79f2672058c11e377548820df0ab4fad8f47ffdc
Sudo command (even using -E) resets $PATH for security reasons, Fedora
and other distos might have to explicitly add /usr/local/bin to $PATH
in order to run the commands in the chroot environment.
Change-Id: I95068ecf1e7437152e11c6d6789f59bcfd6ae9cc
Qemu-nbd does not perform well with older versions of qemu due to
the lack of writeback caching mode. It also only builds qcow2 images
and there is a desire for raw image support. Finally, qemu-nbd makes
it very difficult to build images concurrently due to the somewhat
opaque nature of how it selects a /dev/nbd# device. losetup, on
the other hand, makes this process very straight forward.
Change-Id: I309fad8af4fd1e8d1720c17b65e1897a76d5e897
Co-Author: Clint Byrum <clint@fewbar.com>
Using the IMAGE_TYPE env variable, you can select which ouput disk format you want for your image.
Only qcow2 format supports compression.
Change-Id: Ia01d9079c354dbd3cfcf870d86f97ee57bd9d479
Log first-boot.d script output to a file, to ease troubleshooting,
since the first-boot scripts are removed from the system after
running, even in case of failure.
Change-Id: I07d5c782dd918a804e054bf08efce89ed9442e0c
Also modified dib-run-parts to apply a more workable solution for
filtering out unwanted files such as editor backups and VCS.
The script is installed in its own element, depended on by the OS
specific ubuntu element. This is because the ubuntu element (and
later other OS's) are responsible for populating the root filesystem.
If we try to install this in base, the root filesystem will look to
be populated already and we will skip automatically choosing ubuntu.
Change-Id: I017646748c1a8360299106289b57d976d45875a8
This includes the install-packages implementation for dpkg, apt http proxy
config, daemon blocking and unblocking.
Change-Id: I8f159021d2b223d7003cec067de3aa605ad06974
Move common openstack service installation operations
into a new script `os-svc-install`, which simplifies
the elements for openstack services.
Change-Id: Ied8ac3278e7fe8af76e24748ec4e598a84afa03c
This is a necessary but not complete step towards supporting Fedora and Suse
distributions. Further work is needed (e.g. to quiesce daemons on
installation).
Change-Id: If3ea6093d41a21de755db52328226b84b5a3ede6
Rather than force creation of an element for a single package install,
allow people to do this from the command line.
Change-Id: I63e2e7e50c4a7dbb8a8e198581dfadce91773621
Flavour is overloaded in openstack due to it being used by nova. Element
seems to have the same feeling of combinability without using a term already
in active use in the openstack community.
Change-Id: Ia4c028d4062a8f69c66665821c94dd4bcdf06031
Where possible use the system configured resolv.conf file
to to support resolution of internal servers such as proxies.
Use previous hardcoded google dns nameserver as fallback
Change-Id: Ibc1e8a3b96e733a2f7fbcdd4b2c5d3e8926424c4
