Commit Graph

94 Commits

Author SHA1 Message Date
yogananth subramanian
f84b910f15 Enable decimal value for $DIB_IMAGE_SIZE.
Patch adds support for using decimal values for $DIB_IMAGE_SIZE.
This allows for creating images that are <1G in size.

Change-Id: I945644a8e77fecfb0b83efa282dc00bb29514e0b
Closes-Bug: #1366909
2015-10-13 15:51:27 +00:00
James Slagle
82b328bb13 Fix $DIB_DEFAULT_INSTALLTYPE export statement
$DIB_DEFAULT_INSTALLTYPE is propogated into the chroot by writing out an
export statement to a file under environment.d. However, that export
statement does not account for other elements that may choose to define
$DIB_DEFAULT_INSTALLTYPE earlier on in their own environment.d files.

Change-Id: I5865b1859c080c328251e8dce072c0634ef75c16
2015-05-01 16:37:56 -04:00
Gregory Haynes
8266e14adc Add functional smoke test for disk-image-create
We do not have any testing inside DIB for testing disk-image-create
logic. Lets do some smoke testing for all our supported image formats.

Also adding a run_functests.sh so we can extend this later without editing
the jenkins job.

Change-Id: Ie491e27f00bde54f73af6b47c9696ec04d973b14
2015-04-24 16:37:29 +00:00
Jenkins
44d6d5cd88 Merge "Support VHD output format" 2015-04-22 05:31:04 +00:00
Gregory Haynes
361a7a60b9 Fix image size to fit filesystem journal
Our logic to determine needed image size doesnt fully account for the
in-filesystem journal. This only shows up when creating images that are
very small relative to the FS journal size.

Change-Id: Ic3c2bcd31ec4fee6bcd9f67767842eb3fbe20d3a
2015-04-20 23:26:12 +00:00
Jenkins
e9ac9c9823 Merge "Open MKFS_OPTS for extension in disk-image-create" 2015-04-20 19:08:44 +00:00
Gregory Haynes
7937fb5392 Support VHD output format
This useful for building images for xen.

Change-Id: Ib716bd7fc1ed979968ea92f46f1644038e40df66
2015-04-14 13:39:18 -04:00
Gregory Haynes
7347577784 Correctly handle raw type ordering
We need to run raw outputting last, and were incorrectly detecting that
image type.

Change-Id: Idfbe46c34e783046b940d3eb28c846a0cf9ac0a6
2015-04-01 20:28:44 +00:00
Ethan Gafford
199196e399 Open MKFS_OPTS for extension in disk-image-create
At present, MKFS_OPTS is closed for modification. The ability
to extend the set of MKFS_OPTS adds a great deal of power for
knowledgeable end-users. (And in some specific circumstances,
it is vital to success, as in the case of building RHEL/CentOS
6 images from RHEL/CentOS 7 hosts, in which case -O ^64bit is
required in order for the image to boot.)

Change-Id: I714e86a5a413779e63f598fbbb5a79d23cf6d8c3
2015-03-26 14:36:08 -04:00
Gregory Haynes
2d79e9d395 Short circuit qemu-img convert for raw images
We currently use qemu-img convert with a raw source and dest when
building raw images. We can just mv the file for increased speed.

Change-Id: I3da095cb9ecad7224a121a434a9fb204132bf6df
2015-03-25 22:50:42 +00:00
Ian Wienand
36b59c001c Standarise tracing for scripts
There is a wide variety of tracing options through the various shell
scripts.  Some use "set -eux", others explicity set xtrace and others
do nothing.  There is a "-x" option to bin/disk-image-create but it
doesn't flow down to the many scripts it calls.

This adds a global integer variable set by disk-image-create
DIB_DEBUG_TRACE.  All scripts have a stanza added to detect this and
turn on tracing.  Any other tracing methods are rolled into this.  So
the standard header is

---
if [ "${DIB_DEBUG_TRACE:-0}" -gt 0 ]; then
    set -x
fi
set -eu
set -o pipefail
---

Multiple -x options can be specified to dib-create-image, which
increases the value of DIB_DEBUG_TRACE.  If script authors feel their
script should only trace at higher levels, they should modify the
"-gt" value.  If they feel it should trace by default, they can modify
the default value also.

Changes to pachset 16 : scripts which currently trace themselves by
default have retained this behaviour with DIB_DEBUG_TRACE defaulting
to "1".  This was done by running [1] on patch set 15.  See the thread
beginning at [2]

dib-lint is also updated to look for the variable being matched.

[1] https://gist.github.com/ianw/71bbda9e6acc74ccd0fd
[2] http://lists.openstack.org/pipermail/openstack-dev/2014-November/051575.html

Change-Id: I6c5a962260741dcf6f89da9a33b96372a719b7b0
2015-02-12 10:41:32 +11:00
Michael Kerrin
6337e6fe17 Fix issue with leaking /tmp/image.* directories
When building the ramdisk we don't cleanup the temporary
directories after ourselves. This leaves /tmp/image.* directories
mounted and /tmp/image.* directories on the system.

Also the ramdisk-functions duplicate, from what I can see,
the cleanup function from common-functions. So when a job
is killed off it ends up leaving /tmp/image.* directories
on the system.

Change-Id: I2d73aabd0eb176027b4e7368580db08902e2b6ab
2015-01-28 12:07:07 +00:00
Gregory Haynes
c94880aa2f Fail helpfully if uuidgen is missing
If uuidgen is not installed we currently fail when performing tune2fs.

Change-Id: I4062eb1d62322c455dac5b83b856f454da211320
2014-12-15 15:42:35 -08:00
Ben Nemec
ff2a89215e Improve --ramdisk-element docs
Unlike most of the options to disk-image-create, this one didn't
document its default value, and it also didn't have any context for
why one might use it.

Change-Id: I57ea529cf387ab8ec8497d35be19ac64198df47a
Related-Bug: 1397073
2014-12-01 10:21:13 -06:00
Frederic Lepied
ccd7811dc0 disk-image-create: fix test when no argument is given.
Avoid a warning when multiple arguments are passed to disk-image-create
like:

bin/disk-image-create: 146: [: aa: unexpected operator

Change-Id: I619d7bf2e4b56885d629019b80b249bc80b8f43e
2014-11-25 22:28:01 +01:00
Gregory Haynes
9bb7ace03c Fail helpfully if no elements are specified
Instead of charging ahead and failing because we lack dib-run-parts we
can fail in this case with a useful error message.

Change-Id: Ie10176b5a7019a0385796984ebbaf7742876925a
2014-11-21 10:31:11 -08:00
Yuriy Zveryanskyy
676b1b738d Add element for building ramdisk with ironic-python-agent
This element creates kernel and ramdisk files based on Fedora,
example:
disk-image-create -a i386 -o test fedora ironic-agent disable-selinux

Change-Id: Ifa133d1680b81cb87d32a405aa7d7b40fe91f835
2014-11-13 19:30:44 +02:00
Dan Prince
022cb2c1c2 Support installing packages by default
Adds new disk-image-create --install-type option which
can be used to enable alternate install types. The
default install type is 'source'.

The motivation is to eliminate elements like
enable-package-installs which require coupling and also
don't work with elements in multiple element repositories.

This patch does not prevent you from using the previous
DIB_INSTALLTYPE_ variables to customize the install type,
rather it just changes the default so you don't have to
set it so often when using things like packages.

Change-Id: Icee98440fc2251728f2dca30e7c4789a0fd89b93
2014-10-28 15:05:53 -04:00
Jenkins
47502b8042 Merge "export DIB_ROOT_LABEL to make it global" 2014-10-28 15:51:19 +00:00
Ghe Rivero
48ce0fde51 export DIB_ROOT_LABEL to make it global
It is needed by the debian element in the root.d phase. (enviroment.d variables are not available after the base image is built)

Change-Id: Idb8f9795a619687569bcb0de774c87333d11b292
Closes-Bug: #1384103
2014-10-23 07:38:21 +00:00
Ramakrishnan G
13aed64e97 iso element to build bootable ISO images
This commit adds a new element named 'iso' to build a bootable
ISO image for the kernel/ramdisk emitted by the 'baremetal' or
'ramdisk' element.

Change-Id: I89d175a29e2d0bc64b47fe527f0d0f6875f6849a
2014-10-21 14:37:57 +05:30
Jenkins
bb07de7fe9 Merge "Enable dracut deploy ramdisks" 2014-10-20 11:03:54 +00:00
Ben Nemec
eed30adc48 Enable dracut deploy ramdisks
The element builds dracut from source on Ubuntu because the
Ubuntu dracut package is broken and very old, so it can't be
installed properly and causes a number of other issues that
are fixed by using a newer version of Dracut.

This initial version should work in virtualized environments.
Further validation of its suitability for real baremetal
deployments will need to be done in the future, but this should
be sufficient to enable that work.

Regarding Dracut specifically, in order to limit the changes
needed in the existing scripts this element continues to use a
cut down version of the /init script that we were building for the
existing ramdisk.  However, instead of running it as pid 0 it is
run as a Dracut pre-mount hook.  This allows Dracut to set up all
of the hardware and system bits, while falling early enough in the
Dracut sequence to complete the deployment before Dracut would try
to boot off the hard disk.

bp tripleo-juno-dracut-ramdisks
Change-Id: I144c8993fe040169f440bd4f7a428fdbe3d745cf
2014-10-17 21:44:09 -05:00
Jenkins
2b60bea961 Merge "Allow for multiple image outputs from raw source" 2014-10-16 21:56:28 +00:00
Ghe Rivero
873de01491 Remove first-boot.d support
After being deprecated two releases ago, finally remove any reference
for the support of first-boot.d

Change-Id: I08d67404ef48cad61db3b18fb86e970abfa5d2b6
2014-10-16 09:48:14 +00:00
Clark Boylan
ae928057bd Allow for multiple image outputs from raw source
When uploading images to multiple clouds it is possible that the same
image will be needed in multiple formats to accomodate hypervisors
across clouds. Update disk-image-create's -t flag to take a list of
desired output image formats so that a single disk-image-create can
output all of the desired image formats.

Change-Id: If121b2342ae888855ba435aa3189f039e985b812
2014-10-13 09:16:42 -07:00
Ben Nemec
dde3d24213 Allow custom rootfs labels
Per [1], our current root label of cloudimg-root does not work with
XFS because XFS only allows 12 character labels.  This change
addresses that by allowing a custom rootfs label to be specified
in the call to disk-image-create.  There should be no backwards
compatibility concerns as the default label is unchanged.  Any
external elements dealing with the label would need to be updated
to support this new feature, but should continue to work as before
as long as a custom label is not specified.

[1]: https://bugzilla.redhat.com/show_bug.cgi?id=1139584

Change-Id: I596104d1a63b5dc6549e8460a1ae3da00165ef04
2014-09-26 17:44:08 -05:00
Jenkins
7d0e7fe5ad Merge "Centralize handling of /lost+found" 2014-08-28 16:33:12 +00:00
Pino Toscano
267e96f52c Centralize handling of /lost+found
Just remove globally /lost+found after the root and install phases, so
distribution elements don't need to do that by themselves.

Change-Id: Ic783e613bd8794aefd3f40c9a7c308d14cd04b8d
2014-08-04 11:48:21 +02:00
iberezovskiy
a8d961e6ec Create cache directory
Creating of cache directory was missed in disk-image-create script

Change-Id: I4e5901bb455712d6403362a6aac48eaa93f62e29
2014-07-29 18:45:22 +04:00
JUNJIE NAN
d65b37fb03 Use readlink to get script path
So we can support usage:
ln -s ~/source/.../disk-image-create ~/bin/

Change-Id: Id54b613a247679b98cef3e9a6f37cd93155d029e
2014-07-10 15:41:41 +00:00
AzherKhan
d06a1caafb Correction: if then statement code style.
Corrected the if; then statement to follow bash convention.

Change-Id: Ia3f80520bb9446c4feb5ce36dccffa2b1c1d72c4
2014-07-07 03:43:15 -07:00
Yolanda Robla
fabbb8da58 Disk-image-create should allow sending compat flags to qemu-img
Add a --options flag that allows disk-image-create to send custom
options to qemu-img calls. It is useful for forcing compat into
qemu-img created images.

Usage:
    disk-image-create --qemu-img-options compat=0.10 base ubuntu

Change-Id: I73ee2c86abc115220d51ffebbbe9ce3c4d92f188
Closes-Bug: #1329746
2014-06-25 12:38:06 +02:00
Derek Higgins
e7d105a751 Add tar as an output type
Add support for d-i-b to create tarballs, the primary consumer for
tarball images would be linux containers.

Change-Id: I27d67401f3e4415226a4a51e1dde46f739c0220a
2014-06-11 10:48:50 +01:00
Monty Taylor
1c07d4ba1a Remove All Rights Reserved
The phrase is no longer needed as of August 23, 2000 with Nicaragua's
joining of the Berne Convention.

Additionally, in at least one instance,
elements/cache-url/bin/cache-url, its existence in the file between
Copyright lines is just weird and feels misleading, even though it is
not.

Remove all of the lines, because sanity.

Change-Id: I24fd76c2b4f66b8036010b5079db39ead729abee
2014-04-25 11:25:10 -07:00
Jenkins
bfba0ccd03 Merge "Remove an excess cp of disk images." 2014-03-26 08:38:52 +00:00
Robert Collins
dc93feee69 Remove an excess cp of disk images.
save_image is used to copy kernel and ramdisks out of the image, which
we will sometimes want to keep the source, and sometimes not. However
for the main image itself, the temp copy is never kept, so use mv
  rather than cp and avoid the excess IO.

Change-Id: I5a9f0d69ffee3e6b872a8927537ac17f02f5aa4d
2014-03-25 17:20:37 +00:00
James Slagle
3953fd5d2c Always export IMAGE_NAME
A recent change[1] relies on IMAGE_NAME being available when
98-source-repositories was run. However IMAGE_NAME was previously only
exported if the -o option had been used.

[1] I7dbe9e163ad38a418cf2869a81e720de2c27dfb1

Change-Id: Ife3d94fd45a2e0e0948c17414c369e0e6e040442
2014-03-13 16:35:36 -04:00
Dan Prince
ea555476cc Make max-online-resize an option.
In fb246a02eb we introduced an
ext4 option to allow root filesystems to be resized up to 1PB.
This appears to cause an ext4 resize2fs bug in some images.

When the issue occurs an image will hit a kernel bug when
cloud-init runs the resize2fs command during first boot:

  kernel BUG at fs/ext4/resize.c:409!

In this commit we add a new option for max-online-resize
which can be used if a really large root partition is
desirable.

The root cause of all this is really a design problem in
DIB/TripleO at the moment in that we shouldn't have to worry about the
max size of the root file system when creating our images.  Ideally we'd
just mkfs on the root file system itself. Much more efficient,
avoids this problem altogether...

I think the best thing to do today to avoid this is make setting
max-online-resize an option in DIB. This will allow us to stick
to the (well tested) ext4 defaults for most cases, and if someone has
need for a large root filesystem they can easily bump the setting. This
may be temporary until we either fix the design... or the ext4 fix is
released.

Change-Id: I371f62555d2753cec48790c8fd811c4342af925c
Closes-bug: #1280709
2014-02-19 08:37:56 -05:00
Dan Prince
21c1a6205b Correct DIB long option parsing
Fixes issues with option parsing. The --min-tmpfs and --image-cache
options where previously broken.

Closes-bug: 1282077
Change-Id: I40c62b16d854335902d1b6b5ceab7f0e1992623a
2014-02-19 08:33:08 -05:00
Clint Byrum
d2cb979ddf Increase padding to allow for smaller images
With smaller base images, such as Debian, the padding value used to
size the root filesystem does not leave enough room for grub which is
installed after the filesystem is created.

Change-Id: Ic2ab9e2efc9bf1b02f802ee47a36e3fff9c3512e
2014-01-11 18:06:33 -08:00
Derek Higgins
d5b30d126e Increase the size heuristic for images
Occasionally cloud-init fails to resize the disk on first boot, this is
occuring because the filesystem has only 10% free. Nudging the disk size
up a bit more should give us a little more headroom so this is less
likely to occur.

I havn't been able to find out what value ensures we wont hit this
problem but I can say for sure this fixes all casses of the problem I
have seen.

Change-Id: Ib23f3a654151338ad91839e49b323b65b4054245
2013-12-11 14:38:24 +00:00
JUN JIE NAN
65a3063da3 Add examples for ramdisk-image-create
The examples to create cfn-tools image do not make sense for
ramdisk-image-create.

Change-Id: Iff50898119a1f37f41e52e6215c6bbeca003dfff
2013-12-05 18:10:28 +08:00
Dan Prince
255387a288 Drop default distribution root element support
Converts our existing default root element code to be just a check
which exits with a failure message if no root/distribution element
is found.

Change-Id: I954a6abfd7871d5807b1a171a03fa98932410cff
2013-12-02 11:01:29 -05:00
JUN JIE NAN
7c2d1b2295 Store DIB_* only env variables
Or environment variables like OS_USERNAME OS_PASSWORD will be stored
in /etc/dib_environment also. It's potentially unsafe.

Change-Id: I3a65d5bc0e4469a071db9ac23ebc82196a3f3feb
2013-12-02 09:26:35 +08:00
James Slagle
844daff526 Add option --image-size.
Adds an option for --image-size, which sets the DIB_IMAGE_SIZE
environment variable.  Having the cli option makes disk-image-create
more consistent in that parameters are able to be specified via a flag
on the command line or an environment variable.  Also, you may not be
able to remember all the environment variables that you can tweak, yet
--help will tell you about the cli parameters.  Preserves the old
behavior as well where if DIB_IMAGE_SIZE is set in your environment, it
will be honored.

Change-Id: I195c9144a80ce7b8bd5809b57f2bed71a2cbdf26
2013-11-25 15:28:18 -05:00
JUN JIE NAN
6d6634e804 Usage message enhancement
Add --help, add a simple usage and two examples

Change-Id: Iddf07ec1935f337000bd8ffb3d7c1fcc65753e8d
2013-11-14 09:18:45 +08:00
JUN JIE NAN
39cbbd6980 Increase journal size to 64 M for ext4 file system
the problem is that the journal isn't large enough to allow online
resizing. Solution is straight forward. So the file system can be
resized successfully to disk size specified in flavor.

Fixes bug #1233008

Change-Id: Ie84fb8aea8d334706574d1a8006ec9eaee5bb5be
2013-10-05 16:53:53 +08:00
Matthew Farrellee
1d2a85d5e4 Add DIB_IMAGE_CACHE
DIB_IMAGE_CACHE will be a user override for the location where images
are cached. Default location is ~/.cache/image-create

Change-Id: I3e9b9f970864d555c9ec9436344b53f6d3d66dfa
2013-09-19 14:34:05 -04:00
Derek Higgins
e8c73cde49 Set raw image size to be multiple of 64k
This works around a bug in qemu-img causing it to sometimes error if
compressing a raw image that isn't a multiple of 64k (the qcow2
cluster size).
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1180021
http://lists.gnu.org/archive/html/qemu-devel/2013-04/msg02520.html

Change-Id: I88f8bb5c6bf3606ef5856b085a284af6b8858dbb
2013-09-08 21:36:29 +01:00