Per [1], our current root label of cloudimg-root does not work with
XFS because XFS only allows 12 character labels. This change
addresses that by allowing a custom rootfs label to be specified
in the call to disk-image-create. There should be no backwards
compatibility concerns as the default label is unchanged. Any
external elements dealing with the label would need to be updated
to support this new feature, but should continue to work as before
as long as a custom label is not specified.
[1]: https://bugzilla.redhat.com/show_bug.cgi?id=1139584
Change-Id: I596104d1a63b5dc6549e8460a1ae3da00165ef04
Just remove globally /lost+found after the root and install phases, so
distribution elements don't need to do that by themselves.
Change-Id: Ic783e613bd8794aefd3f40c9a7c308d14cd04b8d
Add a --options flag that allows disk-image-create to send custom
options to qemu-img calls. It is useful for forcing compat into
qemu-img created images.
Usage:
disk-image-create --qemu-img-options compat=0.10 base ubuntu
Change-Id: I73ee2c86abc115220d51ffebbbe9ce3c4d92f188
Closes-Bug: #1329746
Add support for d-i-b to create tarballs, the primary consumer for
tarball images would be linux containers.
Change-Id: I27d67401f3e4415226a4a51e1dde46f739c0220a
The phrase is no longer needed as of August 23, 2000 with Nicaragua's
joining of the Berne Convention.
Additionally, in at least one instance,
elements/cache-url/bin/cache-url, its existence in the file between
Copyright lines is just weird and feels misleading, even though it is
not.
Remove all of the lines, because sanity.
Change-Id: I24fd76c2b4f66b8036010b5079db39ead729abee
save_image is used to copy kernel and ramdisks out of the image, which
we will sometimes want to keep the source, and sometimes not. However
for the main image itself, the temp copy is never kept, so use mv
rather than cp and avoid the excess IO.
Change-Id: I5a9f0d69ffee3e6b872a8927537ac17f02f5aa4d
A recent change[1] relies on IMAGE_NAME being available when
98-source-repositories was run. However IMAGE_NAME was previously only
exported if the -o option had been used.
[1] I7dbe9e163ad38a418cf2869a81e720de2c27dfb1
Change-Id: Ife3d94fd45a2e0e0948c17414c369e0e6e040442
In fb246a02eb we introduced an
ext4 option to allow root filesystems to be resized up to 1PB.
This appears to cause an ext4 resize2fs bug in some images.
When the issue occurs an image will hit a kernel bug when
cloud-init runs the resize2fs command during first boot:
kernel BUG at fs/ext4/resize.c:409!
In this commit we add a new option for max-online-resize
which can be used if a really large root partition is
desirable.
The root cause of all this is really a design problem in
DIB/TripleO at the moment in that we shouldn't have to worry about the
max size of the root file system when creating our images. Ideally we'd
just mkfs on the root file system itself. Much more efficient,
avoids this problem altogether...
I think the best thing to do today to avoid this is make setting
max-online-resize an option in DIB. This will allow us to stick
to the (well tested) ext4 defaults for most cases, and if someone has
need for a large root filesystem they can easily bump the setting. This
may be temporary until we either fix the design... or the ext4 fix is
released.
Change-Id: I371f62555d2753cec48790c8fd811c4342af925c
Closes-bug: #1280709
Fixes issues with option parsing. The --min-tmpfs and --image-cache
options where previously broken.
Closes-bug: 1282077
Change-Id: I40c62b16d854335902d1b6b5ceab7f0e1992623a
With smaller base images, such as Debian, the padding value used to
size the root filesystem does not leave enough room for grub which is
installed after the filesystem is created.
Change-Id: Ic2ab9e2efc9bf1b02f802ee47a36e3fff9c3512e
Occasionally cloud-init fails to resize the disk on first boot, this is
occuring because the filesystem has only 10% free. Nudging the disk size
up a bit more should give us a little more headroom so this is less
likely to occur.
I havn't been able to find out what value ensures we wont hit this
problem but I can say for sure this fixes all casses of the problem I
have seen.
Change-Id: Ib23f3a654151338ad91839e49b323b65b4054245
Converts our existing default root element code to be just a check
which exits with a failure message if no root/distribution element
is found.
Change-Id: I954a6abfd7871d5807b1a171a03fa98932410cff
Or environment variables like OS_USERNAME OS_PASSWORD will be stored
in /etc/dib_environment also. It's potentially unsafe.
Change-Id: I3a65d5bc0e4469a071db9ac23ebc82196a3f3feb
Adds an option for --image-size, which sets the DIB_IMAGE_SIZE
environment variable. Having the cli option makes disk-image-create
more consistent in that parameters are able to be specified via a flag
on the command line or an environment variable. Also, you may not be
able to remember all the environment variables that you can tweak, yet
--help will tell you about the cli parameters. Preserves the old
behavior as well where if DIB_IMAGE_SIZE is set in your environment, it
will be honored.
Change-Id: I195c9144a80ce7b8bd5809b57f2bed71a2cbdf26
the problem is that the journal isn't large enough to allow online
resizing. Solution is straight forward. So the file system can be
resized successfully to disk size specified in flavor.
Fixes bug #1233008
Change-Id: Ie84fb8aea8d334706574d1a8006ec9eaee5bb5be
DIB_IMAGE_CACHE will be a user override for the location where images
are cached. Default location is ~/.cache/image-create
Change-Id: I3e9b9f970864d555c9ec9436344b53f6d3d66dfa
In some scenarios, the required space in the tmpfs partition can be
larger (or smaller) than the default one, producing errors due to
the lack of enough space (or performance penalties for not using
tmpfs).
Using --min-tmpfs <size>, we can hint the working set size we'll need
and let dib choose to avoid or use tmpfs.
Change-Id: I7d5fe498302a100c8555ae542268e14b21f3a0c5
When compressing an image, this is done in the same dir where the raw
image resides, doubling the amount of space needed (scarce when
using tmpfs), and then it's moved to the .cache folder in disk.
Combining these two functions, we reduce the amount of space needed
in the tmpfs partition (when in use), and the compressed image is
created directly on the .cache folder disk, so there is no need to
move the compressed image after the process into disk.
Change-Id: I451d24bdd6fa0983414244135dff5e96c0549833
A user running di-b several times while developing an element may not
want to drop to a shell in all cases but may only want to do so if one
of their in target hooks failed.
This patch gives them the ability to do so, If break=after-error is set
then a user will be provided a in target shell taking over from where the
last failed command left off.
Change-Id: Ia2f7ac4c21b64b971f87f4ae9cb867981b13eb5e
(Based on review https://review.openstack.org/#/c/36009)
Scripts test for existence of ../share/diskimage-builder and
fall-back to ../ if not found. This allows scripts to run unmodified
from a packaged installation or a local archive/repository.
Change-Id: I0cf4c1fdb8e42ec284c56860cb15818632b93b9e
I missed the getopt parameter and forgot defaults are imported after
option processing. Untested code is broken code!
Change-Id: I133a691909d38e834c204950276a57f4884fc4ed
Complex image builds can download hundreds of MB of data from the
internet with many separate lookups. It would be nice to allow users
to ask for a fast build where those lookups are entirely avoided,
using locally cached resources (where possible). This new interface
allows users to signal to elements that they wish to operate without
updating cached resources, which will in turn allow us to avoid
checking for stale data at all.
As part of this I've also documented where we cache data, so that
things like the ccache cache dir and image cache files are not a
surprise to users.
Change-Id: I27f5de6ceaa4e9c6390721b7c434fe0908df84f5
Ramdisks are now built inside a chroot which is built by the normal
image build process. Doing so improves our independence of the
precise state of the build host.
This fixes bug 1194055.
Change-Id: Ibc254fbb9e7b404b5f38c1b35bcde8a4136e8e28
Move the ensure_nbd function call from the disk-image-create to the
disk-image-get-kernel. The image creation process does not use qemu-nbd
anymore.
Change-Id: I8ff2ad54f322a9e33d52408fa4e253a7b45900c7
The recent change to use loopdev instead of nbd stopped
honoring the DIB_IMAGE_SIZE setting.
This change adds it back, by resizing the image to
DIB_IMAGE_SIZE GB, if specified. If unspecified, it resizes
the image to (actual usage + 20%), as it did before this change.
Change-Id: I69afd9584e644ddacc948619100f153d3d8713a4
When partition tables are added to loopback devices, this can set off a
chain of udev hooks that may still be holding the loopback open. Failing
to detach loopback devices was the reason we were seeing leaked tmpfs
volumes.
Fixes bug #1178091
Change-Id: I836d6e2bbce824951dd4786e3ef28273ea18ee73
For machines with low RAM (or no access to /proc/meminfo) the builder
will still run, but will just build inside the filesystem that is
hosting /tmp. This will result in a slower build (especially if there
are a lot of .deb packages installed).
Fixes bug #1175453
Change-Id: I79f2672058c11e377548820df0ab4fad8f47ffdc
Sudo command (even using -E) resets $PATH for security reasons, Fedora
and other distos might have to explicitly add /usr/local/bin to $PATH
in order to run the commands in the chroot environment.
Change-Id: I95068ecf1e7437152e11c6d6789f59bcfd6ae9cc
Qemu-nbd does not perform well with older versions of qemu due to
the lack of writeback caching mode. It also only builds qcow2 images
and there is a desire for raw image support. Finally, qemu-nbd makes
it very difficult to build images concurrently due to the somewhat
opaque nature of how it selects a /dev/nbd# device. losetup, on
the other hand, makes this process very straight forward.
Change-Id: I309fad8af4fd1e8d1720c17b65e1897a76d5e897
Co-Author: Clint Byrum <clint@fewbar.com>
Post install scripts are useful because they can perform tasks you want
to handle after the OS/application install but before the first boot
of the image. e.g: Clean the cache left by the package manager reducing
the size of the image.
Change-Id: I03e77f602192bbdce29c02999d1b57fac8051ddc
Fixes: Bug #1145786
Document the use of DIB_ as a prefix on environmental variables that can
be overridden at build time.
Introduce the first such variable by migrating IMAGE_SIZE to
DIB_IMAGE_SIZE.
Change-Id: Ie36b734991b913a23f37f2add47d470d7c1576e4
ARM doesn't have a generic Linux image due to the soc-specific nature of Linux
kernels today, so we drop the manual installation of that package, replacing it
with a dist-upgrade instead. This involved tweaks to the dpkg and fedora
install-package scripts.
Change-Id: I97924b80ca87781307e1087b9fe4b18215770e84
This will write two files in /etc/ that contain the environment and
command line arguments used in the creation of an image. This should
assist with later efforts to repeat the creation of an image.
Change-Id: Icdbe6693380bed6c406feee10d2cb1a88a992932
Using ELEMENTS_DIR env variable, you can specify multiple dirs containint your elements.
It must be a ":" separated elements list.
When an element is seen in one of those lists, it will be chosen and will continue with the next element.
Change-Id: I18eca27d943139cd6ca1ebd232b419e502d7b048
This includes the install-packages implementation for dpkg, apt http proxy
config, daemon blocking and unblocking.
Change-Id: I8f159021d2b223d7003cec067de3aa605ad06974
Move common openstack service installation operations
into a new script `os-svc-install`, which simplifies
the elements for openstack services.
Change-Id: Ied8ac3278e7fe8af76e24748ec4e598a84afa03c
This change includes the 'base' element by default and adds a
command line switch to prevent this behaviour. This should reduce
confusion since this element is almost always necessary.
Change-Id: Ib4a2ef53a76b119b764751fa21058f4e2cb37741
This is a necessary but not complete step towards supporting Fedora and Suse
distributions. Further work is needed (e.g. to quiesce daemons on
installation).
Change-Id: If3ea6093d41a21de755db52328226b84b5a3ede6
This adds an option to disk-image-create that allows it to re-exec with
an empty environment if required.
Change-Id: I9e51aa07b903e18cdd0ed5f953800307f9899b5d
Remove the explicit installation of an English language pack and instead
just default LANG to C. This settls Perl's noisy warnings and also stops
any side effects of different language build hosts.
Note that this is set right at the start of the build process, so it
should be entirely possible to override if needs be.
Change-Id: Id3b31162d4198fa02dc5a4d11168e57dbcd14a5d
This adds a new optional file to the root of elements. The
file lists dependencies which will be added to the list
requested by the user during disk image creation.
Change-Id: Id71c3b333563604bbbaf90f9cf40e24fa9738fc8
Rather than force creation of an element for a single package install,
allow people to do this from the command line.
Change-Id: I63e2e7e50c4a7dbb8a8e198581dfadce91773621
We originally checked the return code of getopt, but subsequently
a debugging echo has been inserted and now we are checking the
return code of "echo", which is very rarely going to be useful.
Change-Id: I054fecaf6576d5e9d42b07159bba7d50335bf6cc
Flavour is overloaded in openstack due to it being used by nova. Element
seems to have the same feeling of combinability without using a term already
in active use in the openstack community.
Change-Id: Ia4c028d4062a8f69c66665821c94dd4bcdf06031
