Commit graph

660 commits

Author SHA1 Message Date
Ghe Rivero
dbfca7a816 Combine compress and save image into one function
When compressing an image, this is done in the same dir where the raw
image resides, doubling the amount of space needed (scarce when
using tmpfs), and then it's moved to the .cache folder in disk.

Combining these two functions, we reduce the amount of space needed
in the tmpfs partition (when in use), and the compressed image is
created directly on the .cache folder disk, so there is no need to
move the compressed image after the process into disk.

Change-Id: I451d24bdd6fa0983414244135dff5e96c0549833
2013-08-28 10:51:40 +00:00
Lucas Alvares Gomes
5dd6c4a834 Extlinux fallback
If GRUB2 is not available in the system a fallback to install Extlinux
will happen. This patch also allows you to enforce the use of Extlinux
exporting the DIB_EXTLINUX variable.

Change-Id: I88fa792a0bd19cacca167134f2054b11c807a451
2013-08-27 11:14:44 +01:00
Lucas Alvares Gomes
b01f412c1e Rename 51-grub to 51-bootloader
This patch is part of a another big patch that will implement a fallback
to Extlinux in case GRUB2 is not available in the system. The reason why
it's being splitted in two patches is because git is not being able to
recognize the file changes when the rename happens in the same commit.

Change-Id: Ic2027dfa057fd6d62b532203b7ff50c3c739bccb
2013-08-27 10:43:26 +01:00
James Slagle
0702ce9843 Add a pip-cache element.
The element bind mounts a pip cache inside the image build chroot so
that pip downloads can be reused across image builds.  While similar
in purpose to the PyPi element that sets up a mirror, this element
just allows for a reusable download cache and doesn't require anything
to be setup beforehand.

The pip-cache element is not concurrency safe, and that is indicated
in the README for the element.  An upstream bug was file as well:
https://github.com/pypa/pip/issues/1141

Change-Id: Ibd1d4ea17c24923ed939357ada95b781e3179cfd
2013-08-26 15:17:41 -04:00
Derek Higgins
11df3a45f4 Fix URL to os-apply-config
Just incase anybody is actually using this element(which may not be the
case).

Change-Id: I663f8e4db86a38d8c9063d02da3dfaa6715e465a
2013-08-26 11:19:32 +01:00
Jenkins
6687bdd267 Merge "Fixed a network setup issue for F19" 2013-08-25 22:35:10 +00:00
Richard Su
1fb5d232be qpid package mappings
Change-Id: Iad2c21f3c2cd771f58d5444f61186b3aaf84e89e
2013-08-22 23:52:05 -07:00
Chris Alfonso
1dea2f3524 Call sync before unmounting keeps the mount from being busy.
here is a bug associated with this patch because there is a
workaround included. Calling sync to enable the unmounting...
https://bugs.launchpad.net/diskimage-builder/+bug/1214388

Change-Id: I9d5a1f092fd8f3ae1c14de03dd516203baab4da3
2013-08-21 09:10:35 -04:00
Jenkins
473d08cb84 Merge "Remove a device mapping, then let the loop device get removed" 2013-08-20 23:09:43 +00:00
Chris Alfonso
35364749de Remove a device mapping, then let the loop device get removed
Change-Id: Ie1eb621ff56688208421d0115547f127a3c7fd56
2013-08-20 16:58:25 -04:00
Chris Alfonso
301c3c4475 Extracting common functionality for rpm based distros
Rather than dublicating code to implement rhel or any
other derivitive, this patch introduces an rpm-distro
element that should be used as a dependency.

Change-Id: I8a92bb041764d03f430b438f0013704f79a8674c
2013-08-20 16:44:19 -04:00
Chris Alfonso
e7a8fefa0b Use kpartx if partition device doesn't exist
If the loop partition isn't created as a device by the kernel, go ahead and use
kpartx to create the device.

Change-Id: I53290b7724d0cb45a1fc9225ec096025db8978dd
2013-08-20 13:27:49 -04:00
sjing
6c26cec656 Set file permission to be executable.
When ramdisk-image-create is called with the hwdiscovery element,
it failed to install hwinfo package.

The root reason is 10-hwinfo is not executable, the fix is to
set the file permission of 10-hwinfo to be executable.

Change-Id: I3a57147103f607b3975eb6ad23a44475f26c2f1e
2013-08-20 08:44:43 +00:00
Jan Provaznik
c93dfeec7f Fixed a network setup issue for F19
IP address is passed as a kernel boot param when booting undercloud
machine. For Fedora 19, this IP is not set, instead IP is fetched
from DHCP.

The problem is caused by a bug in dracut-network version 029, which
causes that dracut network setup script is not executed properly. This
patch applies a fix for the bug when building undercloud image.

Fixes: bug #1213833
Change-Id: Ie94a86ed716044343791628973143c1a7d0778dc
2013-08-19 10:37:46 +02:00
Monty Taylor
d18891d952 Update stackforge references to openstack
Change-Id: I17e896eddf6fe84f4784cac8231552ee3d21aa39
2013-08-17 22:58:26 -04:00
Steve Baker
d6deb9f565 Package dib-run-parts
dib-run-parts is a symlink into the elements directory.

During packaging the symlink is preserved however
during installation dib-run-parts becomes a file containing
the contents of the symlink target.

Related-Bug: #1212482
Change-Id: I78c28a52689810a2ccfc9925a9ac8df017c44326
2013-08-15 12:39:39 +12:00
James Slagle
c9a6aef9f4 Add yum element.
The yum element provides for specific customizations
for yum based distros.  The included customization is
a yum cache mounted outside of the chroot so that yum
downloaded packages are reused on subsequent image builds.

Change-Id: I6833c9fdbc83cb09debec6a789082e105c917800
2013-08-14 18:26:51 -04:00
Steven Dake
2c73a001b2 dkms is unavailable on RHEL and derivatives
The dynamic kernel module system is not available on RHEL, CentOS,
Scientific Linux, or SUSE.  Make it part of the distro post-install
rather then base post-install.

Change-Id: Ic2c345bf9f0738dadae611194e263d3a5d424a3e
2013-08-12 19:07:28 -07:00
Clint Byrum
89fe51b83d Fix pypi element README.md errors
- pip install to a github URL without a git+ prefix fails.
- Need to close EOF heredoc when creating mirror.yaml

Change-Id: Ic555174881e9c06fede1cf731141cbe6eb0e0b02
2013-08-12 14:53:18 -07:00
Robert Collins
97bc5d7853 Allow using a pypi mirror to install via pip.
Using a custom pypi mirror can be very convenient, making image builds
substantially faster - because we create multiple virtual
environments we benefit more than single-virtualenv users would.

Change-Id: I997daf1f9477c447e1fb30818aea9e80a49b31a6
2013-08-10 22:07:30 +12:00
Robert Collins
d6d941a8f1 Make $HOME in the chroot be reasonable.
With sudo -E $HOME is inherited, which is nearly always wrong.

Change-Id: Ic77b9c2892c19a2fc08a07d35f76da79a346ebac
2013-08-10 01:16:02 +12:00
Robert Collins
c7d80ddeed Cleanup mount points automatically.
Manually listing the mount points that need cleanup is not
maintainable and makes it hard to write elements that use bind
mounting to inject resources into the build environment.

Change-Id: I7c9ade444f4ebe42552f8e321f257a7ec0a674ef
2013-08-09 22:53:32 +12:00
Derek Higgins
a5806cd482 Change the rootfs label in F18 and F19
The images produced by the diskimage-builder have their filesystem with
a label of "cloudimg-rootfs", The regex changing fstab on fedora didn't
match the F19 case.

This regex matches both
F18 :
LABEL=_/   /         ext4    defaults        1 1
F19 :
UUID=f2da0fc8-2e12-4a6e-8e95-a31a474aa1a8 / ext4 defaults,noatime 0 0

Change-Id: Iafcf5e069199f1df76874309b2d5064f343fb5e5
2013-08-07 09:57:21 +01:00
Jenkins
39d4d9b941 Merge "Remove explicit sudoers requirement." 2013-08-06 20:57:33 +00:00
Jenkins
585a155d17 Merge "Document the dangers of co-existing elements." 2013-08-06 20:55:50 +00:00
Chris Jones
357c2de681 Remove explicit sudoers requirement.
Most of our build process runs as root and the overhead of explicitly
tracking sudo commands in the shipped sudoers.d file, plus the
operational overhead of keeping the sudoers.d file updated on build
hosts over time, is now considered to be unnecessary.
Instead, we now document that the build process will expect sudo
commands to work and the operator must make this happen (either by
running as root, via sudo -E, or allowing passwordless sudo for the
build user).

Change-Id: If2628e7d0c7efde0fb99c84a217523ba29b1b38d
2013-08-06 16:08:35 +12:00
Chris Jones
3708e40051 Document the dangers of co-existing elements.
Add a potent example (and simply remedy) of how element authors can
unwittingly conflict with each other in ways that it is almost
impossible for us to guard against with low level policy/mechanism in
di-b. Fixes bug 1181309.

Change-Id: I51e845065702911371e448521327143495c3b405
2013-08-06 16:07:09 +12:00
Robert Collins
bcad3737e8 Blacklist H803
H803 isn't supported by community consensus. Blacklist it to avoid
breaking folk's muscle memory.

Change-Id: I8eea9eb2562e12136018e762c8a5e23527fcf48f
2013-08-06 13:59:13 +12:00
Robert Collins
c7c4ec19a3 Capture the repositories from source-repositories.
We try to expose metadata to users about what dib included in the
image and source-repositories was not doing that, so this fixes it. It
also permits easy introspection from other elements (such as an
upcoming PyPI cache layer).

Change-Id: I1e48e65f69ebe1f4556cc3bf35ccf195e99bcc76
2013-08-05 09:52:28 +12:00
Robert Collins
950fa15d64 Improve caching documentation.
It was a little weak, and folk need enough detail to reason about
performance.

Change-Id: I67f81ab5a206b3d42dd1e4e1eb52af88ac1a3adc
2013-08-02 22:48:51 +12:00
Steve Baker
7795c7b108 Package with pbr.
Uses pbr-0.5.20 glob support to install elements, lib and scripts.

Change-Id: Ie466e753f6e75a943ced1a2939c9bb27aec4a6b0
2013-08-02 12:32:25 +12:00
Derek Higgins
4651913e55 Add a new break on error
A user running di-b several times while developing an element may not
want to drop to a shell in all cases but may only want to do so if one
of their in target hooks failed.

This patch gives them the ability to do so, If break=after-error is set
then a user will be provided a in target shell taking over from where the
last failed command left off.

Change-Id: Ia2f7ac4c21b64b971f87f4ae9cb867981b13eb5e
2013-08-01 17:08:54 +01:00
Jenkins
4284936084 Merge "Ensure $TMP_BUILD_DIR is actually created." 2013-08-01 00:41:52 +00:00
Jenkins
9233ca4056 Merge "Enable running disk-image-create on SUSE Linux" 2013-08-01 00:36:36 +00:00
James Slagle
de6efe2b2c Ensure $TMP_BUILD_DIR is actually created.
Previously if mktemp failed in function mk_build_dir, die was not
called.  The check for the exit code was actually checking that the
previous export statement completed successfully, not the call to
mktemp.

Change-Id: I477a2ce75c87c8167883ce7aa342e93c40770e29
2013-07-31 14:26:05 -04:00
Jenkins
5b37036c88 Merge "Modify relative paths of lib, elements for packaging" 2013-07-31 11:04:03 +00:00
Jeff Peeler
87fb533979 Modify relative paths of lib, elements for packaging
(Based on review https://review.openstack.org/#/c/36009)

Scripts test for existence of ../share/diskimage-builder and
fall-back to ../ if not found. This allows scripts to run unmodified
from a packaged installation or a local archive/repository.

Change-Id: I0cf4c1fdb8e42ec284c56860cb15818632b93b9e
2013-07-31 13:15:33 +12:00
Jeff Peeler
6ac894cae6 Add binaries for setuptools
Binaries disk-image-create, disk-image-get-kernel, and
ramdisk-image-create were missing.

Change-Id: I1d9e25fe49d23e9e66fd2dcbde44e17505a3d7d7
2013-07-30 14:25:45 -04:00
Clint Byrum
5a69fa879e Make Ubuntu 13.04 (raring) the default release
Ubuntu 13.04 has been released now for 3 months. The updated libvirt,
openvswitch and kernel are all beneficial to various OpenStack components,
and many other software is updated beyond the versions in Ubuntu 12.10.

Change-Id: I358aed8bf906c3ff5103f19b1f9e6ac689b5d5ee
2013-07-29 17:02:23 -07:00
Jenkins
fb81fe8742 Merge "Provide hint for what package contains qemu-img" 2013-07-29 13:39:42 +00:00
Jenkins
45f32b4b82 Merge "Ignore emacs autosave files in source-repositories" 2013-07-29 05:04:12 +00:00
Jenkins
1146fe63fc Merge "Add needed symlink for Fedora deployment ramdisk." 2013-07-28 19:58:59 +00:00
Monty Taylor
4535c5323a Ignore emacs autosave files in source-repositories
source-repositories uses find to discover things it should import. If
someone happens to use emacs, and happens to have had an autosave file
in a directory with a wildly broken version of a source-repositories-
file, then dib will attempt to execute the contents of the autosave file
and the user will be sad. Adding a filter to the find command for files
ending in ~ will prevent pain and suffering.

Change-Id: I27c8a4b53d3c9b913ca926db16b4271941f6fb5d
2013-07-27 14:53:29 -04:00
Clint Byrum
f57e8c66c2 Provide hint for what package contains qemu-img
While following install instructions in README.md it should be more
clear how to install qemu-img on your system.

Change-Id: Ibb8828806bf46716b91b83a4eea696f4fe478206
2013-07-27 17:05:20 +00:00
Monty Taylor
6b74b65449 Fix hacking errors
As another step in aligning further with OpenStack practices, stop
ignoring the hacking style checks.

Change-Id: I16c9f0ca3be5790176467377303817249e7643ea
2013-07-27 12:17:02 -04:00
Monty Taylor
c179d9b04d Add downloadonly flag to fedora
devstack-gate needs to be able to make an image that has pre-downloaded
but not installed a sequence of packages. To support that, add a -d flag
to install-packages to allow downloading but not installing. -d passes
through on a dpkg install, and with this patch should be transformed
to --downloadonly which is the yum version.

Change-Id: Ia4e61fa4304df86afbee7dfc8067ea1bdf528f3e
2013-07-27 12:17:02 -04:00
Monty Taylor
36a46649ed Fixed up test-requirements
As a first step to OpenStack alignment, get test-requirements.txt in
line with current OpenStack standard practices.

Fixes: Bug #1205546
Change-Id: I899c3dc167df191b84008250671c58520c436a30
2013-07-27 12:16:52 -04:00
James Slagle
92fa96b2f6 Add needed symlink for Fedora deployment ramdisk.
Symlink /usr/lib -> ../lib for Fedora deployment ramdisk.  This
symlink is needed for systemd-udevd.  Otherwise, the network device is
not loaded and initialized when the deployment image is booted.  This
logic was previously contained in
elements/deploy/root.d/50-redhat-prepare-fs, but now that ramdisk
images are built in a chroot, the change needs to be moved so that it
is applied to the ramdisk build.

Change-Id: Icea43230126956ccf5fb8a6a96ca706b75d5c32f
2013-07-26 11:09:29 -04:00
Jenkins
b4a9fceea7 Merge "Install which on Fedora" 2013-07-26 13:51:13 +00:00
Derek Higgins
d8ba46f8e3 Install which on Fedora
Create an install hook where core utilities can be installed
on Fedora. In this case we are installing which.
Which is being used by
  ramdisk/post-install.d/01-ensure-binaries
  vm/finalise.d/51-grub
The absence to which caused the ramdisk build to fail on F19.

On Ubuntu which is installed with the package debianutils, this pretty
much has to exist on Ubuntu images, so a corresponding package install for
ubuntu isn't needed.

Fixes Bug 1202612

Change-Id: I847c1619cdab021c9348caed3fa32db64e663e8e
2013-07-26 11:04:58 +01:00