We have some test cases which attempt to build docker images, therefore
we need docker.
Fix a few bugs that showed up when we run docker tests - we need to
docker rm with sudo and docker images don't always have a /tmp so check
before unmounting it.
Change-Id: I147d0ef3f2ea83f35bac568214573a6bde0b1967
This cuts the image size down alot, esspecially if there were lots of
small file deletes.
The fstrim utility is in the util-linux package and should be on
most all systems. fstrim also works with XFS, ext4, btrfs, etc
prodiving the kernel is new enough.
A reduction of 25% or more in size is common.
Change-Id: I269b4416be450369616f9b8e030f84c30e329804
This reverts commit 5184d02a7c.
The decision was made to go with fstrim because it is faster and more
universal that zerofree. The related-id has the patchset that implements
fstrim.
Related-Id: I269b4416be450369616f9b8e030f84c30e329804
Change-Id: If40cf2fc0ecd8686768cbfeac9ecee90907674e7
If the image has an ext filesystem and the zerofree utility is present
on the build system then run zerofree. This should make images as
compressable as possible which is a nice feature when building
compressed qcow2 images.
Change-Id: Ia6062c291f7a3f58b85a4f408ecb3d0574c65d53
Cleanup this function to work with a symlinked directory. Document
it's behaviour more exactly, and add a simple unit-test for it (not
run by default, due to doing things like mounting and unmounting
system dirs on a live system, which doesn't seem safe for CI. But it
is useful for developers ensuring sanity).
Change-Id: I335316019ef948758392b03e91f9869102a472b9
uses upstream's stage4 images, includes all the needed bells and
whistles for openstack on kvm.
Change-Id: Ibca43173c30c2a74a73a2e2d9dd6d6d832c62694
Closes-Bug: 1530911
Due to a bug in how we were running some of our phases we were not
detecting some nonzero exit's. When this is fixed, dib fails early during
cleanup (leaving some resources attatched) due to not propertly
accounting for pipefail.
Change-Id: Icc0b35acbe035cac12a9291e2d07b6c690c3a6ad
With a slow file system, umount can return 0 and the immediately
following remove can fail with a "Device or resource busy" error.
This happened in DevStack in disk-image-create where unmount_image
is followed by an immediate cleanup_build_dir.
Solution is to apply same logic from bug 1332521 to allow the
remove to retry on failure (up to 5s) in case the umount has not
completed.
Change-Id: I3337e2b4ad0111e77f79dc179439cdfea8ebdeda
Closes-Bug: #1527721
The ramdisk-functions overwrites user modified
/etc/modprobe.d/blacklist.conf with a single entry
"blacklist evbug". Due to this, ramdisk fails to recognise
the hardware/disks etc for which user has modified
/etc/modprobe.d/blacklist.conf.
The commit enables copying all the *.conf files under
/etc/modprobe.d to the ramdisk. It creates
a config file /etc/modprobe.d/blacklist-dib-ramdisk.conf
instead of overwriting /etc/modprobe.d/blacklist.conf.
Closes bug: #1492804
Change-Id: Ib88272e4b8f4641c58e7e68bb0c2e4b82efc4fc1
Currently when these files are opened your editor doesn't know what to
do with them. Add #!/bin/bash to library functions so that editors,
diff-tools, etc can do syntax highlighting.
There are other ways to skin this cat, such as renaming to ".sh",
adding -* style editor flags, etc. We had this discussion in DevStack
too, and came to the conclusion the simplest thing that works for
everyone is to just put the #! at the top.
Change-Id: I4cf64321e14844696139f5d40e4d719436390b35
Temp dirs are created with mktemp and thus belong to the user. There
is no need to chown them unless we used `mount -t tmpfs`.
Move chown under the tmpfs_check conditional.
Change-Id: I37efe18ced3a06d461364dc5cb20600f1527e995
This reverts commit ea4a823810.
This function was actually still in use in lib/common-functions
and removing it causes the disk-image-get-kernel /bin
to fail entirely.
Change-Id: Icddb3ca369922a6ea915af8b1b62c434cb1bdf28
Closes-bug: 1464031
Split the cleanup_dirs function in two, i.e. cleanup of the build dir
and the image dir, and use the former to cleanup the temporary build
subdirs after their unmount, before the conversion to other disk
formats; they are not needed anyway at that point, and allows to save
disk space during the conversion phase.
Change-Id: Ie30d7e6033613d6979148423326ae7e17a7342e7
This allow custom elements to be added with symlink. Without -follow
a symlinked element is valid but scripts in *.d directory aren't used.
Change-Id: If50b7d9c3b1f6fe278c28488146709efe5cf065f
Closes-Bug: 1461124
By that point in the build it isn't generally useful, and it causes
confusion when builds fail because people think that's the error.
Change-Id: I26dee4ac0947b71a4a065ef6c5a18103e7df6667
Given this is often the final output, it can look like an error occured.
Changing the wording makes this clearer.
Change-Id: I70f157054e3120cffee6fa5241b1ffe0b7bfa650
Clean all the content in the /tmp directory of the guest, leaving the
directory itself (usually has special attributes).
Modern distributions usually either setup a tmpfs on /tmp, or clean it
at every boot, so the leftovers will be just few bytes in the generated
image.
Regarding other distributions, a clean /tmp at their first boot will
surely not be a bad idea anyway.
Change-Id: I2b0f8864bc4909542d924f5bd9296dca5d0189f2
Adding a test function which allows us to use elements to perform
element-specific tests. In order for this to work sanely, also adding
some configuration to our break system so we can assert on negative
tests.
Also adding a test for apt-sources to verify this code actually works.
Change-Id: I378a74255010eca192f5766b653f8a42404be5ea
Instead of executing `lsb_release` to know the current distro being
built, use the $DISTRO_NAME environment variable, already provided by
distribution elements.
Change-Id: I22b67afb481983cc40c198fd408ad5c7f4d68bec
In I084aff7e449f5de811a6169ec90e352ada7da439 we attemped to address a
bug for systems which dont have a path that works well for inside a
chroot. Turns out there were multiple issues:
* The PATH we were setting was after we attempted to call sh.
* PATH was being set in a sibling process to the command being run.
* PATH was not being exported so it was not effecting child processes.
Using env to set a sane path before we attempt to run our commands
addresses these issues.
Change-Id: I4285f8048465ee5c2490116447d32033007bd185
The build host might not necessarily have a complete set of dirs
required in its PATH. It will likely be better to statically code a
complete(ish) set of PATH entries inside the function.
Some distros (ex: archlinux) are lacking some dirs commonly found in a
PATH. If this is used as a host OS, it will transfer this incomplete
PATH into the chroot, where other guests (ubuntu, centos) will fail to
find basic binaries due to the lack of a /bin entry.
Change-Id: I084aff7e449f5de811a6169ec90e352ada7da439
ubuntu-signed element would install 'linux-signed-image-generic' that
provides signed kernel that can be used for deploy in UEFI secure boot mode.
Package 'linux-signed-image-generic' ships signed kernel with extension
'.efi.signed' (Ex. '/boot/vmlinuz-3.13.0-49-generic.efi.signed').
The kernel modules directory for signed kernel and unsigned kernel is same.
It is without 'efi.signed' extension to its name. This is different from normal
practice of directory naming in '/lib/modules' (Ex. For signed kernel
'vmlinuz-3.13.0-49-generic.efi.signed', modules directory is
'/lib/modules/3.13.0-49-generic').
This needed some changes in '/lib/ramdisk-functions' and 'ramdisk' element to
copy kernel modules.
The signed kernel package contains both signed and unsigned kernel. The
unsiged kernel is without extension '.efi.signed' (Ex.
'/boot/vmlinuz-3.13.0-49-generic'). This required change into
'/lib/img-functions' and 'baremetal' element to pick up signed kernel version
when this element is used.
Closes-Bug: 1443076
Change-Id: I60061cbea847b47fa752b9463cfd387e8e7f0635
Sometime, we will get "Device or resource busy" during the mount point
deleting, umount return 0, but the resouce is busy for a while, so need
to add sleep interval to wait resource free, then we can delete it.
Change-Id: Idaa219d12e847824960eec8907739add5d619d1a
Closes-Bug: 1332521
Some elements (such as the manifests element) want to use these
variables. We currently do not consistently export them (IMAGE_NAME is
only exported if you actually specify a name).
Change-Id: I43d17ddcdd7d0ff3cbb4c530caeebb8da915f4ef
Copy all of the necessary parts for a Fedora based dhclient to work. This
includes a number of network scripts. Also grab the ip command supplied by
the iproute package, the busybox "ip addr" command was missing the valid_lft
and preferred_lft options.
This will allow the dhcp to work in the ramdisk instead of getting passed the
PXE net config.
Related-Bug: #1417026
Change-Id: I8feee9a740855dab7b47162c5727bf91db77fcc6
We currently use qemu-img convert with a raw source and dest when
building raw images. We can just mv the file for increased speed.
Change-Id: I3da095cb9ecad7224a121a434a9fb204132bf6df
I regularly see users report that their build fails because this unmount
line reports an error. Even though we dont bail here because of the ||
true, as a user it is hard to distinguish this from an error.
Change-Id: Ic43f4fb24c53c58329fdf501bba6ba14024ec2aa
Deprecated the `--expand-dependencies` flag from `element-info` usage.
The flag was required and not optional. We can rely on argparse to exit non-0
when the required positional argument is not provided.
Change-Id: Iaf8eb962eb600760974bc33c30b809a07a23278e
Closes-Bug: 1265649
When building the ramdisk we don't cleanup the temporary
directories after ourselves. This leaves /tmp/image.* directories
mounted and /tmp/image.* directories on the system.
Also the ramdisk-functions duplicate, from what I can see,
the cleanup function from common-functions. So when a job
is killed off it ends up leaving /tmp/image.* directories
on the system.
Change-Id: I2d73aabd0eb176027b4e7368580db08902e2b6ab
ramdisk-image-create assumes the presence of vmlinuz file in OS image.
But ubuntu PPC LE images have only vmlinux file. This patch adds
support for using vmlinux file.
Change-Id: I5dd0b8ceb46b73be57d4c15b39b96f99b524fa3f
Closes-Bug: 1413362
I would like to recommend to use + instead of \; in the find
command. As this will ensure the removal of all selected
directories in a single invocation.
Hence improve the speed of deletion.
Change-Id: I409fe11aae217afb6f790491591005c679264ed4
