Commit graph

193 commits

Author SHA1 Message Date
Gregory Haynes
b9b6640fa7 Initial element tests
Adding a test function which allows us to use elements to perform
element-specific tests. In order for this to work sanely, also adding
some configuration to our break system so we can assert on negative
tests.

Also adding a test for apt-sources to verify this code actually works.

Change-Id: I378a74255010eca192f5766b653f8a42404be5ea
2015-05-17 02:07:40 +00:00
Pino Toscano
cb37ef5384 ramdisk: switch from lsb_release to $DISTRO_NAME
Instead of executing `lsb_release` to know the current distro being
built, use the $DISTRO_NAME environment variable, already provided by
distribution elements.

Change-Id: I22b67afb481983cc40c198fd408ad5c7f4d68bec
2015-05-12 17:03:46 +02:00
Jenkins
9ba196394c Merge "Remove unused map_nbd function" 2015-05-06 02:50:17 +00:00
Jenkins
b2c2c47dc1 Merge "Actually set a sane PATH for inside chroot" 2015-05-05 01:53:11 +00:00
Jenkins
f8302171f5 Merge "Add element ubuntu-signed to provide signed kernel" 2015-04-28 11:02:44 +00:00
Gregory Haynes
f2a893b74c Actually set a sane PATH for inside chroot
In I084aff7e449f5de811a6169ec90e352ada7da439 we attemped to address a
bug for systems which dont have a path that works well for inside a
chroot. Turns out there were multiple issues:
 * The PATH we were setting was after we attempted to call sh.
 * PATH was being set in a sibling process to the command being run.
 * PATH was not being exported so it was not effecting child processes.

Using env to set a sane path before we attempt to run our commands
addresses these issues.

Change-Id: I4285f8048465ee5c2490116447d32033007bd185
2015-04-28 00:02:15 +00:00
Jenkins
74b777dae8 Merge "Fix dhclient in Fedora ramdisks" 2015-04-23 03:58:33 +00:00
Jenkins
5440d4b166 Merge "Dont try to unmount if were not using tmpfs" 2015-04-23 03:55:10 +00:00
Jenkins
f0736e7d97 Merge "Append full path to img-functions:run_in_target" 2015-04-22 19:06:12 +00:00
Jenkins
44d6d5cd88 Merge "Support VHD output format" 2015-04-22 05:31:04 +00:00
Jenkins
212da2f149 Merge "Try 5 times for rmdir command call after umount" 2015-04-21 21:22:19 +00:00
Ben Kero
4a2939c207 Append full path to img-functions:run_in_target
The build host might not necessarily have a complete set of dirs
required in its PATH. It will likely be better to statically code a
complete(ish) set of PATH entries inside the function.

Some distros (ex: archlinux) are lacking some dirs commonly found in a
PATH. If this is used as a host OS, it will transfer this incomplete
PATH into the chroot, where other guests (ubuntu, centos) will fail to
find basic binaries due to the lack of a /bin entry.

Change-Id: I084aff7e449f5de811a6169ec90e352ada7da439
2015-04-16 11:19:44 -07:00
Gregory Haynes
ea4a823810 Remove unused map_nbd function
This is a leftover from when we used nbd. We no longer do and this
function is not used.

Change-Id: If037bdd0100ef27401ddd5931ec3db7017302675
2015-04-15 20:20:12 +00:00
Gregory Haynes
7937fb5392 Support VHD output format
This useful for building images for xen.

Change-Id: Ib716bd7fc1ed979968ea92f46f1644038e40df66
2015-04-14 13:39:18 -04:00
Shivanand Tendulker
06e3d7c767 Add element ubuntu-signed to provide signed kernel
ubuntu-signed element would install 'linux-signed-image-generic' that
provides signed kernel that can be used for deploy in UEFI secure boot mode.

Package 'linux-signed-image-generic' ships signed kernel with extension
'.efi.signed' (Ex. '/boot/vmlinuz-3.13.0-49-generic.efi.signed').

The kernel modules directory for signed kernel and unsigned kernel is same.
It is without 'efi.signed' extension to its name. This is different from normal
practice of directory naming in '/lib/modules' (Ex. For signed kernel
'vmlinuz-3.13.0-49-generic.efi.signed', modules directory is
'/lib/modules/3.13.0-49-generic').
This needed some changes in '/lib/ramdisk-functions' and 'ramdisk' element to
copy kernel modules.

The signed kernel package contains both signed and unsigned kernel. The
unsiged kernel is without extension '.efi.signed' (Ex.
'/boot/vmlinuz-3.13.0-49-generic'). This required change into
'/lib/img-functions' and 'baremetal' element to pick up signed kernel version
when this element is used.

Closes-Bug: 1443076
Change-Id: I60061cbea847b47fa752b9463cfd387e8e7f0635
2015-04-12 11:36:17 -07:00
Haomeng, Wang
cc9870ec11 Try 5 times for rmdir command call after umount
Sometime, we will get "Device or resource busy" during the mount point
deleting, umount return 0, but the resouce is busy for a while, so need
to add sleep interval to wait resource free, then we can delete it.
Change-Id: Idaa219d12e847824960eec8907739add5d619d1a
Closes-Bug: 1332521
2015-04-07 02:01:29 +00:00
Gregory Haynes
d9bcde8a4d Export image properties
Some elements (such as the manifests element) want to use these
variables. We currently do not consistently export them (IMAGE_NAME is
only exported if you actually specify a name).

Change-Id: I43d17ddcdd7d0ff3cbb4c530caeebb8da915f4ef
2015-04-01 20:33:17 +00:00
Derek Higgins
5671a38e8e Fix dhclient in Fedora ramdisks
Copy all of the necessary parts for a Fedora based dhclient to work. This
includes a number of network scripts. Also grab the ip command supplied by
the iproute package, the busybox "ip addr" command was missing the valid_lft
and preferred_lft options.

This will allow the dhcp to work in the ramdisk instead of getting passed the
PXE net config.

Related-Bug: #1417026
Change-Id: I8feee9a740855dab7b47162c5727bf91db77fcc6
2015-03-31 12:27:40 +01:00
Gregory Haynes
2d79e9d395 Short circuit qemu-img convert for raw images
We currently use qemu-img convert with a raw source and dest when
building raw images. We can just mv the file for increased speed.

Change-Id: I3da095cb9ecad7224a121a434a9fb204132bf6df
2015-03-25 22:50:42 +00:00
Gregory Haynes
f06e3fdb01 Dont try to unmount if were not using tmpfs
I regularly see users report that their build fails because this unmount
line reports an error. Even though we dont bail here because of the ||
true, as a user it is hard to distinguish this from an error.

Change-Id: Ic43f4fb24c53c58329fdf501bba6ba14024ec2aa
2015-03-24 18:30:55 +00:00
Jenkins
8e313b112f Merge "ramdisk-image-create: add support for vmlinux file" 2015-02-04 06:51:51 +00:00
Jenkins
df7ae94df2 Merge "Corrected element-info usage" 2015-02-03 15:50:06 +00:00
jodewey
8fc027227a Corrected element-info usage
Deprecated the `--expand-dependencies` flag from `element-info` usage.
The flag was required and not optional.  We can rely on argparse to exit non-0
when the required positional argument is not provided.

Change-Id: Iaf8eb962eb600760974bc33c30b809a07a23278e
Closes-Bug: 1265649
2015-01-29 13:43:48 -08:00
Michael Kerrin
6337e6fe17 Fix issue with leaking /tmp/image.* directories
When building the ramdisk we don't cleanup the temporary
directories after ourselves. This leaves /tmp/image.* directories
mounted and /tmp/image.* directories on the system.

Also the ramdisk-functions duplicate, from what I can see,
the cleanup function from common-functions. So when a job
is killed off it ends up leaving /tmp/image.* directories
on the system.

Change-Id: I2d73aabd0eb176027b4e7368580db08902e2b6ab
2015-01-28 12:07:07 +00:00
yogananth subramanian
57705694e7 ramdisk-image-create: add support for vmlinux file
ramdisk-image-create assumes the presence of vmlinuz file in OS image.
But ubuntu PPC LE images have only vmlinux file. This patch adds
support for using vmlinux file.

Change-Id: I5dd0b8ceb46b73be57d4c15b39b96f99b524fa3f
Closes-Bug: 1413362
2015-01-22 02:10:39 +05:30
AzherKhan
bb38d0e325 Optimize speed of deletion in find command
I would like to recommend to use + instead of \; in the find
command. As this will ensure the removal of all selected
directories in a single invocation.

Hence improve the speed of deletion.

Change-Id: I409fe11aae217afb6f790491591005c679264ed4
2014-10-27 21:04:09 -07:00
Jenkins
34c273d8df Merge "Make diskimage-builder work in Docker." 2014-10-24 20:13:45 +00:00
Jenkins
2ea4f6ec8a Merge "Avoid overwritting of hooks" 2014-10-20 11:04:07 +00:00
Jenkins
bb07de7fe9 Merge "Enable dracut deploy ramdisks" 2014-10-20 11:03:54 +00:00
Ben Nemec
eed30adc48 Enable dracut deploy ramdisks
The element builds dracut from source on Ubuntu because the
Ubuntu dracut package is broken and very old, so it can't be
installed properly and causes a number of other issues that
are fixed by using a newer version of Dracut.

This initial version should work in virtualized environments.
Further validation of its suitability for real baremetal
deployments will need to be done in the future, but this should
be sufficient to enable that work.

Regarding Dracut specifically, in order to limit the changes
needed in the existing scripts this element continues to use a
cut down version of the /init script that we were building for the
existing ramdisk.  However, instead of running it as pid 0 it is
run as a Dracut pre-mount hook.  This allows Dracut to set up all
of the hardware and system bits, while falling early enough in the
Dracut sequence to complete the deployment before Dracut would try
to boot off the hard disk.

bp tripleo-juno-dracut-ramdisks
Change-Id: I144c8993fe040169f440bd4f7a428fdbe3d745cf
2014-10-17 21:44:09 -05:00
Ghe Rivero
88271757a5 Avoid overwritting of hooks
Until now there was a possibility for two elements to install hooks
with the same name, so one of them was overwritten. Change logic to
copy the hooks and fail in case one with the same name exists.

Change-Id: Ic2c46835b27c9319f7a889ffd0ccf3f5ccc1f0cd
Closes-Bug: 1251952
2014-10-17 08:16:46 +00:00
Jenkins
2b60bea961 Merge "Allow for multiple image outputs from raw source" 2014-10-16 21:56:28 +00:00
Jenkins
212c6814aa Merge "Force empty $TMPDIR inside the chroot" 2014-10-16 20:04:50 +00:00
Jenkins
5b71da8899 Merge "Remove first-boot.d support" 2014-10-16 18:36:59 +00:00
Ghe Rivero
873de01491 Remove first-boot.d support
After being deprecated two releases ago, finally remove any reference
for the support of first-boot.d

Change-Id: I08d67404ef48cad61db3b18fb86e970abfa5d2b6
2014-10-16 09:48:14 +00:00
Ghe Rivero
d3c89d85b6 Force empty $TMPDIR inside the chroot
There are some situations (use of libpam-tmpdir) where the $TMPDIR is
defined per user, so any sudo call will define an specific $TMPDIR for
the root user which doesn't exist inside the chroot.

Change-Id: I866651fed520007506c7be83837d3791ed9f2235
Closes-Bug: #1330290
2014-10-16 09:28:53 +00:00
Jenkins
16b0e5e856 Merge "Preserve exit value when leaving cleanup trap" 2014-10-14 21:05:14 +00:00
Clark Boylan
ae928057bd Allow for multiple image outputs from raw source
When uploading images to multiple clouds it is possible that the same
image will be needed in multiple formats to accomodate hypervisors
across clouds. Update disk-image-create's -t flag to take a list of
desired output image formats so that a single disk-image-create can
output all of the desired image formats.

Change-Id: If121b2342ae888855ba435aa3189f039e985b812
2014-10-13 09:16:42 -07:00
Steve Kowalik
c6f3c1aae1 Echo that qemu-img convert is running
qemu-img convert can take a few minutes to complete, or far longer
on heavily loaded systems. Notify the user that the image is
undergoing conversion to save them just seeing END PROFILING.

Change-Id: I1ad61ce5ed011b721d48e2d12fb42ef4aa5cd1f6
2014-10-13 15:01:49 +11:00
Gregory Haynes
f06493bcd6 Preserve exit value when leaving cleanup trap
If we entered the cleanup trap due to exit with an error code we should
exit dib with an error code.

Change-Id: Iee1a05668b3239113fb91a2da0d9a66d7de4db6b
2014-10-10 17:04:50 -07:00
Jenkins
8a8bbb5673 Merge "Handle non-existing *generic kernel and initrd" 2014-10-09 16:44:52 +00:00
Clint Byrum
c781115784 Do not try to detach non-existant loopback devices
A user reported symptoms where the losetup line used to detach the
loopback device was failing in tar mode. We don't need to detach a
device that does not exist.

Change-Id: I807996e16199288927b49b4f300ae9b461cb8fe7
Closes-Bug: #1378033
2014-10-08 10:37:48 -07:00
Christian Berendt
6192bb78fb Use $((EXPRESSION)) instead of $[EXPRESSION]
$((EXPRESSION)) is POSIX-conform and should be preferred.

Change-Id: Ibd0524194aeda40961e202b6761dbaed2ea27dfa
2014-09-30 14:52:46 +02:00
Jenkins
6518876a5f Merge "Use dib-run-parts from dib-utils" 2014-09-17 14:31:49 +00:00
Juerg Haefliger
3af14a5bad Handle non-existing *generic kernel and initrd
Finding the Debian boot kernel and initrd is collapsed into a
single statement so that the script doesn't bail out when
'set -o pipefail' is set and the *generic kernel and initrd are
absent.

Change-Id: Ifd616818f7387e6f102636c7cf186be6097f1d5c
Closes-Bug: 1335016
2014-09-16 11:57:01 +02:00
Victor Lowther
38b14df72c Make diskimage-builder work in Docker.
When running inside a Docker container, we cannot rely on devices in
/dev/mapper to be automagically created by udev, because we probably
don't have a udev at all.  To work around this, run dmsetup mknodes
after every kpartx run.

Change-Id: If7e30579224ce54c5ed26d08974d8293c144719a
2014-09-15 06:43:53 -05:00
Dirk Mueller
d4198bde24 Replace backticks with $()
It seems this is generally preferred in shell
code to what I heard from reviewers.

Change-Id: If61813bd3c4bc61d0282232c99f8011e776eba8b
2014-09-05 19:00:06 +02:00
Dirk Mueller
58f5a543d4 Fix openSUSE kernel/initrd detection
For some of the scripts, the unversioned name is not
good enough. so determine the link target of "vmlinuz" and
"initrd" to get the full versioned path.

Change-Id: I52f9b5435fafaf7925e2332058a9e41b26478f1c
2014-09-05 18:59:03 +02:00
Ben Nemec
644598941c Use dib-run-parts from dib-utils
Now that dib-run-parts has been moved to the dib-utils project, we
need to update diskimage-builder to use it instead of the version
directly in diskimage-builder.

This change removes the old copy of the dib-run-parts script in
the element, adds dib-utils as a dependency of diskimage-builder,
and updates the uses of dib-run-parts to correctly handle the fact
that it is now external to the project.

Requires I0be1f876d0e4a7d38e0d5c6010a552a8ebb158a4

Change-Id: Ia0a0df7784a14c49b5c47ac0b03e6c2602c84b3b
2014-08-26 16:44:18 -05:00
Robert Collins
4319730357 Add a ramdisk-install.d hook path.
We need to be able to do install.d like things for ramdisks
themselves, but install.d runs outside the ramdisk context - and its
likely to break peoples brains if we mangle the two together - so this
adds a new hook point, ramdisk-install, specifically for installing
things into the ramdisk.

Change-Id: I37d1660309cda6e28bd0b316b08f61db4e080613
2014-07-16 19:23:50 +12:00