Commit Graph

40 Commits

Author SHA1 Message Date
Ghe Rivero
dbfca7a816 Combine compress and save image into one function
When compressing an image, this is done in the same dir where the raw
image resides, doubling the amount of space needed (scarce when
using tmpfs), and then it's moved to the .cache folder in disk.

Combining these two functions, we reduce the amount of space needed
in the tmpfs partition (when in use), and the compressed image is
created directly on the .cache folder disk, so there is no need to
move the compressed image after the process into disk.

Change-Id: I451d24bdd6fa0983414244135dff5e96c0549833
2013-08-28 10:51:40 +00:00
Chris Alfonso
1dea2f3524 Call sync before unmounting keeps the mount from being busy.
here is a bug associated with this patch because there is a
workaround included. Calling sync to enable the unmounting...
https://bugs.launchpad.net/diskimage-builder/+bug/1214388

Change-Id: I9d5a1f092fd8f3ae1c14de03dd516203baab4da3
2013-08-21 09:10:35 -04:00
Robert Collins
d6d941a8f1 Make $HOME in the chroot be reasonable.
With sudo -E $HOME is inherited, which is nearly always wrong.

Change-Id: Ic77b9c2892c19a2fc08a07d35f76da79a346ebac
2013-08-10 01:16:02 +12:00
Robert Collins
c7d80ddeed Cleanup mount points automatically.
Manually listing the mount points that need cleanup is not
maintainable and makes it hard to write elements that use bind
mounting to inject resources into the build environment.

Change-Id: I7c9ade444f4ebe42552f8e321f257a7ec0a674ef
2013-08-09 22:53:32 +12:00
Derek Higgins
4651913e55 Add a new break on error
A user running di-b several times while developing an element may not
want to drop to a shell in all cases but may only want to do so if one
of their in target hooks failed.

This patch gives them the ability to do so, If break=after-error is set
then a user will be provided a in target shell taking over from where the
last failed command left off.

Change-Id: Ia2f7ac4c21b64b971f87f4ae9cb867981b13eb5e
2013-08-01 17:08:54 +01:00
Chris Jones
8d1ce9c0c3 Build ramdisks in an image chroot.
Ramdisks are now built inside a chroot which is built by the normal
image build process. Doing so improves our independence of the
precise state of the build host.

This fixes bug 1194055.

Change-Id: Ibc254fbb9e7b404b5f38c1b35bcde8a4136e8e28
2013-07-12 11:09:35 +01:00
Clint Byrum
f6cc6bb185 Use ccache to cache all compiles between builds.
Many python extensions are built from source each time an image is
built. Repeating these is wasteful, so ccache is employed to eliminate
that waste. A cache dir outside the chroot is used to speed up
subsequent image builds.

Change-Id: Ib73563ddbe5f3be7454bfc54ab91cedb559a1304
2013-07-09 18:18:19 -07:00
Robert Collins
a969be49a2 Move functions to common-functions for reuse.
Change-Id: I12a5687d94df678d7f7e8c0b075ee4aba1082156
2013-06-25 17:36:37 +01:00
Lucas Alvares Gomes
1435f6e48d Make the finalise_base function less Ubuntu-ish.
The finalise_base function was creating a broken link at /etc/resolv.conf
when building a non ubuntu image.

Change-Id: I974d43fd6db365dbff9c85195e3b72950f3409da
2013-06-18 12:51:39 +01:00
Dan Prince
1fa27e745d Stop apt-get installing qemu-img.
This is simply a prerequisite and we should not try to
apt-get install it inline.

Remove this code and exit with an error if qemu-img is
not found.

Change-Id: I52d47a95d24d4bfe801ba664200604bd385dc67f
2013-06-11 10:03:10 -04:00
Lucas Alvares Gomes
02ee161e81 Refactor the first boot routine.
Setting up a routine to run at first time the system boot may vary from
distribution to distribution. This patch will remove the logic from the
diskimage-builder code and put it into the elements.

The base element will now deploy a dib-first-boot script and each OS
element will ensure that this script runs at rc.local time. By doing
that we will put the OS specific stuff in the OS elements and remove
the embedded code out of the img-function file.

Change-Id: I24c5d1b1185de5693f145347fe912245f1ba7dfe
2013-05-16 15:50:10 +01:00
Steve Baker
c6da246b73 Fix for running /etc/rc.local on Fedora
With systemd rc.local lives in /etc/rc.d and is only run
if the rc-local service is enabled.

This change creates a /etc/rc.d/rc.local symlink, and modifies img-functions
prepare_first_boot to tolerate an initially missing /etc/rc.local.

Fixes bug #1179674

Change-Id: I7e6102b62224950c1de5d48205dc4feff60f9389
2013-05-14 11:48:58 +12:00
Lucas Alvares Gomes
e537e046d1 Force the inclusion of /usr/local/bin in PATH.
Sudo compiled with the --with-secure-path option enabled (default on
Ubuntu, Fedora and potentially other distros) will cause sudo to ignore
any changes made to the PATH variable in the user's environment (even when
the -E option is used). Also, some distros might not have /usr/local/bin
set in their sudoers secure_path causing the diskimage-builder to fail
when building an image using such distros as host. This patch will force
the inclusion of the /usr/local/bin in the PATH enviroment variable
inside the chroot.

Change-Id: Idb7156d8ff124bfe685cc721337d9bb21f49dc9e
Fixes: bug #1175980
2013-05-08 23:32:47 +01:00
Chris Krelle
76d8d98a60 Remove img-functions from ramdisk-image-create.
Ramdisk-image-create sources both img-functions and ramdisk-functions
causing duplicate functions. To correct this situation this patch
removes the import of img-functions from ramdisk-image-create.
It moves cleanup_dirs and run_d functions from img-functions to
common-functions to allow this.

fixes bug 1175427
Authored-by: Chris Krelle <nobodycam@gmail.com>

Change-Id: I971d019b0d92b06d9de661663b49c433e823ea42
2013-05-08 07:39:30 -07:00
Clint Byrum
d9a2211d14 Only use tmpfs if build machine has 4GB+ RAM.
For machines with low RAM (or no access to /proc/meminfo) the builder
will still run, but will just build inside the filesystem that is
hosting /tmp. This will result in a slower build (especially if there
are a lot of .deb packages installed).

Fixes bug #1175453

Change-Id: I79f2672058c11e377548820df0ab4fad8f47ffdc
2013-05-03 06:55:53 -07:00
Robert Collins
d119c28f49 Revert "Improve Fedora build host support."
This reverts commit 591059ffd3.

Change-Id: I8ad5bde2eabe12c6575d51e7443d9b46521c0928
2013-05-03 08:16:43 +12:00
Jenkins
55993863fc Merge "Improve Fedora build host support." 2013-05-02 19:30:43 +00:00
Lucas Alvares Gomes
591059ffd3 Improve Fedora build host support.
Sudo command (even using -E) resets $PATH for security reasons, Fedora
and other distos might have to explicitly add /usr/local/bin to $PATH
in order to run the commands in the chroot environment.

Change-Id: I95068ecf1e7437152e11c6d6789f59bcfd6ae9cc
2013-05-01 09:40:15 +01:00
Robert Collins
cb62bae9b8 Build images using loopdev instead of qemu-nbd.
Qemu-nbd does not perform well with older versions of qemu due to
the lack of writeback caching mode. It also only builds qcow2 images
and there is a desire for raw image support. Finally, qemu-nbd makes
it very difficult to build images concurrently due to the somewhat
opaque nature of how it selects a /dev/nbd# device. losetup, on
the other hand, makes this process very straight forward.

Change-Id: I309fad8af4fd1e8d1720c17b65e1897a76d5e897
Co-Author: Clint Byrum <clint@fewbar.com>
2013-04-30 08:56:12 -07:00
Ghe Rivero
a05629b9f4 Support multiple outputs disk formats
Using the IMAGE_TYPE env variable, you can select which ouput disk format you want for your image.
Only qcow2 format supports compression.

Change-Id: Ia01d9079c354dbd3cfcf870d86f97ee57bd9d479
2013-02-21 09:26:44 +01:00
Tim Miller
5544da4605 first-boot.d scripts log to a file:
Log first-boot.d script output to a file, to ease troubleshooting,
since the first-boot scripts are removed from the system after
running, even in case of failure.

Change-Id: I07d5c782dd918a804e054bf08efce89ed9442e0c
2013-02-15 10:04:50 -08:00
Clint Byrum
91c30f6ab2 Use dib-run-parts for running scripts in target
Also modified dib-run-parts to apply a more workable solution for
filtering out unwanted files such as editor backups and VCS.

The script is installed in its own element, depended on by the OS
specific ubuntu element. This is  because the ubuntu element (and
later other OS's) are responsible for populating the root filesystem.
If we try to install this in base, the root filesystem will look to
be populated already and we will skip automatically choosing ubuntu.

Change-Id: I017646748c1a8360299106289b57d976d45875a8
2013-02-14 10:48:10 -08:00
Robert Collins
7f77729ec5 Move the dpkg specific stuff to a dpkg element rather than being hardcoded.
This includes the install-packages implementation for dpkg, apt http proxy
config, daemon blocking and unblocking.

Change-Id: I8f159021d2b223d7003cec067de3aa605ad06974
2013-02-14 11:16:12 +13:00
Robert Collins
3860af4be5 Change run-parts usage to be compatible with Fedora.
Change-Id: I7a4e1252b0ced940cd4d8fa2fbd096c016933954
2013-02-14 11:16:12 +13:00
Tim Miller
56e50ee5cf Consolidate common OS installation into a script:
Move common openstack service installation operations
into a new script `os-svc-install`, which simplifies
the elements for openstack services.

Change-Id: Ied8ac3278e7fe8af76e24748ec4e598a84afa03c
2013-02-13 11:06:00 -08:00
Robert Collins
8220c00095 Bad code landed, causing ubuntu to be always landed.
Change-Id: Ia77073563fde70140169dbbc5a7448aad54295e2
2013-02-13 13:58:35 +13:00
Robert Collins
1ee3a01447 Move initial root contents into a hook.
This is a necessary but not complete step towards supporting Fedora and Suse
distributions. Further work is needed (e.g. to quiesce daemons on
installation).

Change-Id: If3ea6093d41a21de755db52328226b84b5a3ede6
2013-02-11 17:01:36 +13:00
Ghe Rivero
11b4a5501d Create install-packages as a binary.
Only supports Debian family distribution so far.

Change-Id: I9fa0b833c34a614fe498c6c6ae599733a69c1309
2013-01-31 00:08:58 +01:00
Chris Jones
0f73578721 Allow manual installation of packages.
Rather than force creation of an element for a single package install,
allow people to do this from the command line.

Change-Id: I63e2e7e50c4a7dbb8a8e198581dfadce91773621
2012-12-21 17:46:44 +00:00
Robert Collins
d03825b504 Add the ability to break into a shell during builds.
Change-Id: I87af952d892f8622e4c916085fc896c735a35438
2012-12-14 20:17:29 +13:00
Robert Collins
012116cad6 Move everything 'common' into base, making it avoidable for tests.
Change-Id: I87edd00f6e0edcd609856192c2c98604ef3274f0
2012-12-14 13:41:57 +13:00
Robert Collins
e8a8e99a08 Make it possible to not recompress the qcow2 image at the end.
Change-Id: Icca922853ee583dcccb6d3970d080fbec11edac7
2012-12-14 13:25:18 +13:00
Ghe Rivero
258185b7a2 Added true to avoid exit on error
disk-image-create just died without any warning

Change-Id: Ia5da4188d64bb22540b57d01184b91faa59149e7
2012-12-13 18:32:34 +01:00
Monty Taylor
1eff4a436e Rename flavour to element.
Flavour is overloaded in openstack due to it being used by nova. Element
seems to have the same feeling of combinability without using a term already
in active use in the openstack community.

Change-Id: Ia4c028d4062a8f69c66665821c94dd4bcdf06031
2012-12-05 14:04:58 -08:00
Darragh Bailey
e5cda1f731 Use system resolv.conf file when available
Where possible use the system configured resolv.conf file
to to support resolution of internal servers such as proxies.
Use previous hardcoded google dns nameserver as fallback

Change-Id: Ibc1e8a3b96e733a2f7fbcdd4b2c5d3e8926424c4
2012-12-03 15:18:09 +00:00
Ghe Rivero
38dc7680c9 Jenkins image fixed
- Removed http_proxy option
- Added sudoers entry to cp first-bot.d
- Fixed typo on 11-jenkins-plugins
- Restored trap exit on run-parts
2012-11-27 18:37:57 +01:00
Ghe Rivero
a3f9f12a4a Untrap EXIT before run-parts 2012-11-26 19:05:12 +01:00
Robert Collins
4b7cdb4fdc Fix copyrights for HP work. 2012-11-15 16:20:32 +13:00
Chris Jones
eb584f14ed Port old baremetal-mkinitrd.sh to the new image creation standards 2012-11-10 10:50:15 +13:00
Robert Collins
5cc195e391 Bring across disk image code. 2012-11-10 00:04:13 +13:00