Commit Graph

167 Commits

Author SHA1 Message Date
Jenkins
34c273d8df Merge "Make diskimage-builder work in Docker." 2014-10-24 20:13:45 +00:00
Jenkins
2ea4f6ec8a Merge "Avoid overwritting of hooks" 2014-10-20 11:04:07 +00:00
Jenkins
bb07de7fe9 Merge "Enable dracut deploy ramdisks" 2014-10-20 11:03:54 +00:00
Ben Nemec
eed30adc48 Enable dracut deploy ramdisks
The element builds dracut from source on Ubuntu because the
Ubuntu dracut package is broken and very old, so it can't be
installed properly and causes a number of other issues that
are fixed by using a newer version of Dracut.

This initial version should work in virtualized environments.
Further validation of its suitability for real baremetal
deployments will need to be done in the future, but this should
be sufficient to enable that work.

Regarding Dracut specifically, in order to limit the changes
needed in the existing scripts this element continues to use a
cut down version of the /init script that we were building for the
existing ramdisk.  However, instead of running it as pid 0 it is
run as a Dracut pre-mount hook.  This allows Dracut to set up all
of the hardware and system bits, while falling early enough in the
Dracut sequence to complete the deployment before Dracut would try
to boot off the hard disk.

bp tripleo-juno-dracut-ramdisks
Change-Id: I144c8993fe040169f440bd4f7a428fdbe3d745cf
2014-10-17 21:44:09 -05:00
Ghe Rivero
88271757a5 Avoid overwritting of hooks
Until now there was a possibility for two elements to install hooks
with the same name, so one of them was overwritten. Change logic to
copy the hooks and fail in case one with the same name exists.

Change-Id: Ic2c46835b27c9319f7a889ffd0ccf3f5ccc1f0cd
Closes-Bug: 1251952
2014-10-17 08:16:46 +00:00
Jenkins
2b60bea961 Merge "Allow for multiple image outputs from raw source" 2014-10-16 21:56:28 +00:00
Jenkins
212c6814aa Merge "Force empty $TMPDIR inside the chroot" 2014-10-16 20:04:50 +00:00
Jenkins
5b71da8899 Merge "Remove first-boot.d support" 2014-10-16 18:36:59 +00:00
Ghe Rivero
873de01491 Remove first-boot.d support
After being deprecated two releases ago, finally remove any reference
for the support of first-boot.d

Change-Id: I08d67404ef48cad61db3b18fb86e970abfa5d2b6
2014-10-16 09:48:14 +00:00
Ghe Rivero
d3c89d85b6 Force empty $TMPDIR inside the chroot
There are some situations (use of libpam-tmpdir) where the $TMPDIR is
defined per user, so any sudo call will define an specific $TMPDIR for
the root user which doesn't exist inside the chroot.

Change-Id: I866651fed520007506c7be83837d3791ed9f2235
Closes-Bug: #1330290
2014-10-16 09:28:53 +00:00
Jenkins
16b0e5e856 Merge "Preserve exit value when leaving cleanup trap" 2014-10-14 21:05:14 +00:00
Clark Boylan
ae928057bd Allow for multiple image outputs from raw source
When uploading images to multiple clouds it is possible that the same
image will be needed in multiple formats to accomodate hypervisors
across clouds. Update disk-image-create's -t flag to take a list of
desired output image formats so that a single disk-image-create can
output all of the desired image formats.

Change-Id: If121b2342ae888855ba435aa3189f039e985b812
2014-10-13 09:16:42 -07:00
Steve Kowalik
c6f3c1aae1 Echo that qemu-img convert is running
qemu-img convert can take a few minutes to complete, or far longer
on heavily loaded systems. Notify the user that the image is
undergoing conversion to save them just seeing END PROFILING.

Change-Id: I1ad61ce5ed011b721d48e2d12fb42ef4aa5cd1f6
2014-10-13 15:01:49 +11:00
Gregory Haynes
f06493bcd6 Preserve exit value when leaving cleanup trap
If we entered the cleanup trap due to exit with an error code we should
exit dib with an error code.

Change-Id: Iee1a05668b3239113fb91a2da0d9a66d7de4db6b
2014-10-10 17:04:50 -07:00
Jenkins
8a8bbb5673 Merge "Handle non-existing *generic kernel and initrd" 2014-10-09 16:44:52 +00:00
Clint Byrum
c781115784 Do not try to detach non-existant loopback devices
A user reported symptoms where the losetup line used to detach the
loopback device was failing in tar mode. We don't need to detach a
device that does not exist.

Change-Id: I807996e16199288927b49b4f300ae9b461cb8fe7
Closes-Bug: #1378033
2014-10-08 10:37:48 -07:00
Christian Berendt
6192bb78fb Use $((EXPRESSION)) instead of $[EXPRESSION]
$((EXPRESSION)) is POSIX-conform and should be preferred.

Change-Id: Ibd0524194aeda40961e202b6761dbaed2ea27dfa
2014-09-30 14:52:46 +02:00
Jenkins
6518876a5f Merge "Use dib-run-parts from dib-utils" 2014-09-17 14:31:49 +00:00
Juerg Haefliger
3af14a5bad Handle non-existing *generic kernel and initrd
Finding the Debian boot kernel and initrd is collapsed into a
single statement so that the script doesn't bail out when
'set -o pipefail' is set and the *generic kernel and initrd are
absent.

Change-Id: Ifd616818f7387e6f102636c7cf186be6097f1d5c
Closes-Bug: 1335016
2014-09-16 11:57:01 +02:00
Victor Lowther
38b14df72c Make diskimage-builder work in Docker.
When running inside a Docker container, we cannot rely on devices in
/dev/mapper to be automagically created by udev, because we probably
don't have a udev at all.  To work around this, run dmsetup mknodes
after every kpartx run.

Change-Id: If7e30579224ce54c5ed26d08974d8293c144719a
2014-09-15 06:43:53 -05:00
Dirk Mueller
d4198bde24 Replace backticks with $()
It seems this is generally preferred in shell
code to what I heard from reviewers.

Change-Id: If61813bd3c4bc61d0282232c99f8011e776eba8b
2014-09-05 19:00:06 +02:00
Dirk Mueller
58f5a543d4 Fix openSUSE kernel/initrd detection
For some of the scripts, the unversioned name is not
good enough. so determine the link target of "vmlinuz" and
"initrd" to get the full versioned path.

Change-Id: I52f9b5435fafaf7925e2332058a9e41b26478f1c
2014-09-05 18:59:03 +02:00
Ben Nemec
644598941c Use dib-run-parts from dib-utils
Now that dib-run-parts has been moved to the dib-utils project, we
need to update diskimage-builder to use it instead of the version
directly in diskimage-builder.

This change removes the old copy of the dib-run-parts script in
the element, adds dib-utils as a dependency of diskimage-builder,
and updates the uses of dib-run-parts to correctly handle the fact
that it is now external to the project.

Requires I0be1f876d0e4a7d38e0d5c6010a552a8ebb158a4

Change-Id: Ia0a0df7784a14c49b5c47ac0b03e6c2602c84b3b
2014-08-26 16:44:18 -05:00
Robert Collins
4319730357 Add a ramdisk-install.d hook path.
We need to be able to do install.d like things for ramdisks
themselves, but install.d runs outside the ramdisk context - and its
likely to break peoples brains if we mangle the two together - so this
adds a new hook point, ramdisk-install, specifically for installing
things into the ramdisk.

Change-Id: I37d1660309cda6e28bd0b316b08f61db4e080613
2014-07-16 19:23:50 +12:00
Jenkins
348a281795 Merge "Allow overwriting old images" 2014-07-15 14:52:06 +00:00
Jenkins
c9bcce2030 Merge "Optimizing directory creation." 2014-07-11 16:36:38 +00:00
AzherKhan
a22305de05 Optimizing directory creation.
Optimized the creation of directories by using the Brace expansion mechanism.

And corrected the dirname resolution to follow correct convention.

Change-Id: I048c04385d2706c4e38f8548555a59b0437e1fcd
2014-07-08 09:39:58 -07:00
Steve Kowalik
84b2ac45de Allow overwriting old images
The two duplicated functions, save_image and finish_image will move
an existing image out of the way if it exists, but it isn't
configurable. Check an environment variable is 0 before doing so.

Switch save_image to just calling finish_image, rather than
duplicating its code exactly.

Change-Id: I26a5a8fa4b6e853c9440bffab195b0bc3728be40
2014-07-08 14:08:10 +10:00
AzherKhan
d06a1caafb Correction: if then statement code style.
Corrected the if; then statement to follow bash convention.

Change-Id: Ia3f80520bb9446c4feb5ce36dccffa2b1c1d72c4
2014-07-07 03:43:15 -07:00
Jenkins
d92cbb7b1e Merge "Disk-image-create should allow sending compat flags to qemu-img" 2014-06-28 00:14:07 +00:00
Yolanda Robla
fabbb8da58 Disk-image-create should allow sending compat flags to qemu-img
Add a --options flag that allows disk-image-create to send custom
options to qemu-img calls. It is useful for forcing compat into
qemu-img created images.

Usage:
    disk-image-create --qemu-img-options compat=0.10 base ubuntu

Change-Id: I73ee2c86abc115220d51ffebbbe9ce3c4d92f188
Closes-Bug: #1329746
2014-06-25 12:38:06 +02:00
Jenkins
20879b2ec9 Merge "avoid failure if /lib/firmware doesn't exist" 2014-06-24 16:46:20 +00:00
Jenkins
c0501291c9 Merge "Refactor code to select boot kernel" 2014-06-14 04:01:45 +00:00
Gonéri Le Bouder
131fb8c216 avoid failure if /lib/firmware doesn't exist
On Debian, /lib/firmware is provided by optional packages like
firmware-iwlwifi or firmware-linux-free. That's why this directory
may not exist.
This change ensure the directory exist before trying to copy it
content.

Change-Id: Icf9eedf4148141c92e9171f16ae744b88a8d8519
2014-06-14 00:05:27 +02:00
Jenkins
bedd50f80d Merge "fail at startup with no operating-system element" 2014-06-13 14:36:59 +00:00
Om Kumar
bccffc8bfd Refactor code to select boot kernel
The script to find best kernel from image (used by vm
and baremetal element) is duplicated and is not in sync
with each other. Moving the code to img-functions as a
function will reduce duplicated code and make it reusable
in future.

Since img-functions is not accessible in chroot env,
kernel selection is being moved from finalise.d to
cleanup.d in the vm element.

Change-Id: I8fbccc13a2c61a5191ef9ea5d2a8302a3e43b000
2014-06-13 11:46:39 +05:30
Jenkins
fa9a561819 Merge "Rename old image file instead of rewrite it" 2014-06-12 22:19:15 +00:00
Gonéri Le Bouder
1011adf370 fail at startup with no operating-system element
Raise an error if there is no element with “operating-system”
in the element-provides file.

Change-Id: I2242537abc4c610252984c32c286bc6eb25de672
2014-06-12 11:19:20 +02:00
iberezovskiy
b740c50c76 Rename old image file instead of rewrite it
If we have the old image file with same name as new image file
we shouldn't rewrite it, just need to rename old.
Uses date number in old image name.

Fixes TODO in lib/common-functions

Change-Id: Ica58a0f4435217912bddf950322299d2ed2889f3
2014-06-06 12:59:28 +04:00
James Slagle
813b4bc242 Check return code of element-info
Ensure the return code of element-info is actually checked by moving the
export IMAGE_ELEMENT to the subsequent line.

A recent change (Ic0a061995e2ae708c95a535cba6237bff58f7d93) exported the
IMAGE_ELEMENT variable. This caused the return code of element-info to not
actually be checked, since the export command is now checked instead and
almost always will return 0.

This caused a misleading error message when using diskimage-builder with an
unknown element since the script didn't exit when it should have:
$ bin/disk-image-create -a amd64 foobar vm
Building elements: base  foobar vm
ERROR: Element 'foobar' not found in 'bin/../elements'
Expanded element dependencies to:
Building in /var/tmp/image.DuIDY1lW
Please include at least one distribution root element.

Change-Id: I3d9c02bf0b1a4f320280185dd3fab0697d2c92f2
2014-05-30 15:41:05 -04:00
Dan Prince
c55539b142 Add pkg-map element.
Adds a new pkg-map element which can be used to install
packages based on an in element 'pkg-map' JSON config
file format.

As part of this change we expose DIB's IMAGE_ELEMENT variable
so that we can have access to the element names which are being
installed in a clean manner.

This change is intended to decouple elements from DIB
and allow new elements to support multiple distributions
(with various package naming schemes) without having to
constantly maintain DIB's various bin/map-packages files.

Change-Id: Ic0a061995e2ae708c95a535cba6237bff58f7d93
2014-05-28 12:21:44 -04:00
Jenkins
dcb5494e04 Merge "Build raw image in separate tmpfs" 2014-05-08 23:15:27 +00:00
Dan Prince
956acd5f59 Eliminate 'tr' in favor of inline bash.
Updates the lib/common-functions generate_hooks() function
to use bash instead of exec'ing tr.

Change-Id: Ie32d3ce5c7c43affd0031a568d57763e36209603
2014-05-02 13:10:33 -04:00
Derek Higgins
3fada2566a Build raw image in separate tmpfs
If the tmpfs being used to hold the image filesystem is close to being
full then housing the raw disk image in the same place can be
problematic as it tends to grow faster then the filesystem shrinks when
data is being moved into it.

Putting them both into separate tmpfs's will allow there to be an
overlap where they jointly use more then the size limit for a singe tmpfs.

Change-Id: Ia17ca357d3b865d70a2d3e13e0479d008ca5f924
Closes-Bug: #1289582
2014-05-01 11:46:29 +01:00
Monty Taylor
0c08692e75 Respect inmutable resolv.conf in the image
In some instances, the content of the image may want to convey specific
resolv.conf settings. For instance, if unbound is installed, it's also
desirable to set resolv.conf to point to it.

Short circuit diskimage-builder's friendliness around resolv.conf files
by not dying in a fire if it finds the file to be immutable.

Change-Id: I88632fb79289681dcf95d32f39a0ad658ba39a60
2014-04-25 16:35:44 -07:00
Ben Nemec
381ff6ab1d Fix set -eu and pipefail failures
Fixes problems found by set -eu and pipefail, including:
-Many unset variables
-Commands that can fail under normal circumstances, which breaks
with set -e.  This change swallows those expected errors to allow
our existing error code to handle them.
-The dkms element was not finding Fedora kernel versions correctly.
This may be an issue for other distros too, but since Fedora was
working fine without this functionality I only changed it to print
a warning message rather than failing the build when it happens.
-The ramdisk init script will not be set -eu because if it fails
the result is a kernel panic, which can be tricky to debug.
However, in testing with set -e a few failing commands were found
and have been fixed in this patch.

Change-Id: I44cf98dfc80cfcaec54b88cc83be80a3dbf2cec3
2014-04-15 20:53:15 -05:00
Derek Higgins
85dadd0d90 Revert "Only create a tmpfs big enough for DIB_MIN_TMPFS"
This didn't pass CI and is now causing failures as it moves us back to a
4G tmpfs.

Change-Id: Ie92fd909ad21216b1fda55c21457893df38e882f
2014-03-26 10:07:41 +00:00
Jenkins
bfba0ccd03 Merge "Remove an excess cp of disk images." 2014-03-26 08:38:52 +00:00
Robert Collins
dc93feee69 Remove an excess cp of disk images.
save_image is used to copy kernel and ramdisks out of the image, which
we will sometimes want to keep the source, and sometimes not. However
for the main image itself, the temp copy is never kept, so use mv
  rather than cp and avoid the excess IO.

Change-Id: I5a9f0d69ffee3e6b872a8927537ac17f02f5aa4d
2014-03-25 17:20:37 +00:00
Clint Byrum
e63625a7e1 Only create a tmpfs big enough for DIB_MIN_TMPFS
This will help to ensure images that would cause bug #1289582 would
still fail even on systems that have more RAM. Also this will ensure we
only reserve enough tmpfs space for this build.

Change-Id: Icece64e691d126a5fc89262897630fdbef1ffdf1
2014-03-07 12:52:15 -08:00