Commit graph

1416 commits

Author SHA1 Message Date
James Slagle
57ba300be4 Symlink correct element install type
The source-repositories element will now symlink the install scripts
corresponding to the set install type for an element into the install.d
directory.

Different install types are implemented by elements by writing scripts to do
the install type under install.d/<element-name>-<install-type>-install
directories.

For example, the nova element would provide:

nova/install.d/nova-package-install/74-nova
nova/install.d/nova-source-install/74-nova

source-repositories will create the following symlink for the package
install type:

$TMP_HOOKS_PATH/install.d/74-nova -> nova-package-install/74-nova

Or, for the source install type:

$TMP_HOOKS_PATH/install.d/74-nova -> nova-source-install/74-nova

Change-Id: I1bfaf39e5a98b2af904fbc6d674dbada30b27ed9
2014-01-15 14:57:08 -05:00
James Slagle
79f31d0476 dib-run-parts should dereference symlinks
dib-run-parts should dereference symlinks, and if they point to an
executable file, execute that file as a script. This can be accomplished
by using the xtype predicate in the find command instead of the type
predicate.

This change is needed so that we can dynamically symlink hook scripts
into hook directories such as install.d at runtime to support different
install types.

Change-Id: I933e7f4b4dcf16956841d8c14aa63b0f9a18fc5d
2014-01-15 14:57:08 -05:00
Clint Byrum
611a8b701a Add debian-upstart for experimenting with upstart
Upstart support in Debian is somewhat experimental, but having it would
be especially useful for TripleO which already generates upstart scripts
which will work fairly well on Debian based systems.

Change-Id: I267da5ae7bc4a484fac152a14963d96ca2ad2693
2014-01-14 22:41:25 -08:00
Clint Byrum
4ce573c582 Allow adding packages to debootstrap
Also allows setting the "distro name" so that things that change the
debootstrap signature will have a different tarball.

This is being done in advance of a debian-upstart element to allow
experimenting with debian and upstart.

Change-Id: I1a558fdaf5924c65f1cced9f9bf29d73643c30af
2014-01-14 22:41:25 -08:00
Clint Byrum
d0fd8915f6 Add support for Debian
Change-Id: Ibc6bc57dea1eab295fb23cccf40b38f3fcb285c3
2014-01-14 22:41:16 -08:00
James Slagle
8d910d10f8 Wait for tgtd socket to be available
tgtd returns execution control and backgrounds itself almost immediately
and before it has made it's listening socket available. This can cause a
race condition as the tgtd socket is not available when tgtadm is run,
resulting in an error:

failed to send request hdr to tgt daemon

Add a function to check if the socket is available before moving on to
calling tgtadm, and a wait_for helper function we can use.  We'll
check for the socket every 0.5 seconds, for up to 5 seconds.

I'm seeing this issue on almost every deploy using a ramdisk built from
Fedora 20. I'm not sure if something has changed in tgtd, but this
behavior is documented since Fedora 18 at least. In the systemd script
for tgtd, there is actually "sleep 5" to work around the problem.

See Also: https://bugzilla.redhat.com/show_bug.cgi?id=848942

Change-Id: Iffa9fc63393309ca653d592dff17316ecbea3e09
2014-01-14 12:53:03 -05:00
Jenkins
de7deb7ce9 Merge "Add new modprobe-blacklist element." 2014-01-14 13:05:55 +00:00
Jenkins
d9303ac9af Merge "Add package support to source-repositories" 2014-01-14 12:42:29 +00:00
Jenkins
85bb142641 Merge "Retry link check up to 10 times." 2014-01-14 11:12:25 +00:00
Jenkins
cccb870d92 Merge "Removing the config-applier element" 2014-01-14 11:09:14 +00:00
Dan Prince
ab9d5d3cc3 Add new modprobe-blacklist element.
This element can be used to provide a custom list
of modules via DIB_MODPROBE_BLACKLIST which
will be disabled via modprobe.d/blacklist.conf.

I'm using this to disable network adapters on the
Red Hat TripleO rack where the devices can't be
explicitly disabled via the BIOS.

Change-Id: I8a0a8ee05fa62628434d7f6422577dbf5cdd7a2e
2014-01-13 15:43:29 -05:00
Dan Prince
5ae9c5d24c Install traceroute on Fedora.
Useful for debugging, etc.

Change-Id: Ia06f8d7a76de2f082bea09c57fbb8150c9706afc
2014-01-13 15:12:39 -05:00
Dan Prince
ae28c99b5f Retry link check up to 10 times.
Updates the dhcp-all-interfaces element so that the link
check tries up to 10 times.

On some of my machines 3 times (seconds) doesn't appear to
be quite long enough...

Change-Id: Ibf7015162fc890d4de8a417b868b0301146944c0
2014-01-13 13:58:42 -05:00
Dan Prince
ae22d39472 Implement serial-console for systemd.
Update install.d/20-stty so that we try to start
a getty on ttyS0 and ttyS1 when using systemd.

Change-Id: Id5b605c74bf82855af0af4d0f95659cdd3a7b3a0
2014-01-13 13:02:54 -05:00
Derek Higgins
63a4c1e9d5 Removing the config-applier element
This is pretty much a duplicate of t-i-e/elements/os-apply-config, isn't used
in tripleo and the upstart script is out of date. We don't need to carry this.

Change-Id: I2b23d5930afae42cff28e2d5ce3d6c84224afa4a
2014-01-13 14:10:10 +00:00
James Slagle
d253ed16d8 Add package support to source-repositories
Adds a new "package" type to source-repositories. When the package type
is specified in an origin-repository-* file the package name and type
"package" need to be specified like so:

nova package

The existing map-packages mechanism can be used to map the specified
package name to the actual distribution package names.  The
pre-configured package repositories on the image will be used to install
the packages during the install.d phase of the image build.

If the repo type is package, an environment.d hook script is updated
with an environment variable which is sourced by dib-run-parts. This is
so that other hooks, particularly in install.d, will know what repo type
was used and can make the correct decision about doing a source install
or not.

Change-Id: Ief4e524664cd32d2087aeb3b1766f05e017de91e
2014-01-13 08:23:53 -05:00
Jenkins
77370c6740 Merge "Remove old versions of grub2 from the yum cache" 2014-01-13 13:06:33 +00:00
Clint Byrum
d2cb979ddf Increase padding to allow for smaller images
With smaller base images, such as Debian, the padding value used to
size the root filesystem does not leave enough room for grub which is
installed after the filesystem is created.

Change-Id: Ic2ab9e2efc9bf1b02f802ee47a36e3fff9c3512e
2014-01-11 18:06:33 -08:00
James Slagle
380df31aac Remove old versions of grub2 from the yum cache
Before downloading and caching the latest version of grub2, we need to
first remove all old versions. Otherwise, the find command that writes
the grub2 version to /tmp/grub/install to install during the finalise
phase could pick one of the older versions of grub2 in the cache. This
will cause the install during the finalise phase to fail since the older
version of grub2 will conflict with the newer installed version of
grub2-tools.

Change-Id: I5b43e359db3ba81ed6283dfb41c7e89629516d38
2014-01-10 17:12:13 -05:00
Jenkins
843214246c Merge "Install tcpdump on Fedora" 2014-01-10 17:28:28 +00:00
Dan Prince
7511faa2b1 Implement dhcp-all-interfaces for systemd.
Includes:

 * Enhancements to the generate-interfaces-file.sh script
 so that it generates ifcfg network-scripts for
 distros like Fedora/RHEL.

 * Includes a new dhcp-all-interfaces.service systemd
 script which ensures network interfaces get generated
 before the network service starts on Fedora.

 * Add a new disable_interface function to
 generate-interfaces-file.sh which (on Fedora)
 will delete the ifcfg config for an interface which
 isn't plugged in.
 This is important because ifcfg-eth0 exists by default
 and we want to remove it if the NIC isn't connected.
 Previous behaviour on Ubuntu is unchanged and the
 generate-interfaces-file.sh just logs it as skipped.

 * General doc updates...

I tested this on a multi-nic machine w/ Fedora where the first NIC
exists but was not connected.

Change-Id: Ia99e312539da43caefd72aa60398d43dac5dcc8f
Closes-Bug: 1239880
2014-01-10 10:14:07 -05:00
Derek Higgins
864e209bfa Install tcpdump on Fedora
Tcpdump has a fairly small footprint and should be installed to all
images (it currently is on the ubuntu image by default). Sometimes the
reason you need it is also the reason its to late to get it.

Change-Id: I4849157515b389d534bdee755cdac6b0768fcb29
2014-01-10 14:16:30 +00:00
James Slagle
f5d4d8c711 Mark install-packages +x.
install-packages is an executable script, it should be +x.  The install
command that was installing it is setting permissions to 755, so we were
fine there.  However, I was trying to test the script outside of an
image build to test a change, and it's helpful to have it +x in the
repository for that context like all the other scripts under bin.

Change-Id: Ic300cc56d463640d119db4e310d375c66ed133ca
2014-01-07 07:54:47 -05:00
Jan Provaznik
c9cceeeb40 Make sure a loop device exists before kpartx is called
If "kpartx -av" is called and there is no unused /dev/loop* device,
then kpartx fails. Calling "losetup -f" finds first unused device
and creates it if it doesn't exist.

Fixes bug #1195264

Change-Id: I5d59b519fd8e2a7380b71a788f9d3f2331b6567f
2014-01-06 11:30:51 +01:00
Jenkins
09a8fca8d0 Merge "Refactor unmount_image with unmount_dir" 2014-01-03 10:17:12 +00:00
Jenkins
c79c05910c Merge "Fixed device or resource busy issue in EXIT trap" 2014-01-03 10:17:11 +00:00
Jenkins
e85bab58ec Merge "Fixed command dib-init-system not found error" 2014-01-03 10:11:25 +00:00
JUN JIE NAN
3968e5bd2c Fixed command dib-init-system not found error
dib-init-system is installed in 04-dib-init-system. Before it is
installed, 10-dib-init-system.bash may be sourced so command not found
issue found.

The patch invokes "/tmp/target.d/dib-init-system" directly in
10-dib-init-system.bash.

Change-Id: I9843224497c7dab8392261b26d62b6d79d1795ee
2014-01-03 17:42:21 +08:00
JUN JIE NAN
893211f19a Refactor unmount_image with unmount_dir
To replace the similiar umount logic in unmout image.

Change-Id: Ia2105a48bbed00ba2571b442e216fccfbcc17f01
2014-01-03 16:55:17 +08:00
JUN JIE NAN
6b7cf26689 Fixed device or resource busy issue in EXIT trap
cleanup for ramdisk failed to umount TMP_BUILD_DIR with device or
resource busy error.

The patch unmounts all the mountpoints under TMP_BUILD_DIR and
detaches loop devices associated with TMP_IMAGE_PATH.

The unmounts are applied with both force(-f) and lazy(-l)
options. Force option is only for NFS mounts, it's kept here since no
harm for lazy option.

Change-Id: I84035e6a003d8135186b2fda3facbd2c37967529
2014-01-03 16:53:38 +08:00
Tomas Sedovic
f176a78486 Add a package mapping for libvirt-dev
This is a dependency for the libvirt-python Python package that
ceilometer-agent-compute uses. libvirt-python is already installed globally in
the system via apt/yum, but that's not available to ceilometer-agent-compute's
virtualenv.

Change-Id: I07a12226889c6cda3bd04021240ee9714a396522
2014-01-02 12:01:03 +01:00
Chris Jones
4794df890a Log unsupported source repository types.
If we don't recognise git/tar/etc as source-repositories repotypes, we
now log the repotype, for debugging purposes.

Change-Id: I7b870807658441c928c7880291eb0284b6f97778
2013-12-20 14:53:53 +00:00
Chris Jones
4c7a3bf79f Improve source-repositories git caching.
We now unconditionally cache the repositories being fetched with
source-repositories.

Additionally, by improving the way we handle repository refs, we are now
able to build images with code taken directly from gerrit reviews.

Change-Id: Ifd4c5691f6761eb5551663e6d0aa1c0e42afced3
2013-12-20 14:48:30 +00:00
Jenkins
c1eb44c9b2 Merge "Add element to modify /etc/apt/sources.list in dib" 2013-12-19 11:01:13 +00:00
Kui Shi
5daa989fb8 Add element to modify /etc/apt/sources.list in dib
Add apt-sources element for Ubuntu OS to override the default
/etc/apt/sources.list in cloudimg.

Set DIB_APT_SOURCES with a proper sources.list file to replace
the default one.

If you want to use this element in tripleo project, set NODE_DIST or
EXTRA_ELEMENTS / UNDERCLOUD_DIB_EXTRA_ARGS / OVERCLOUD_DIB_EXTRA_ARGS
to make it take effect at build time and run time.

e.g. before running devtest.sh:
export DIB_APT_SOURCES=/etc/apt/sources.list
export NODE_DIST="ubuntu apt-sources"

Change-Id: I9ce0d03b506c2948b96382e4d6e85f0aff906450
2013-12-18 06:01:36 -08:00
Ralf Haferkamp
2da3235b82 Fix package-mappings for openSUSE
Change-Id: I6b59d62a9b3df89d8eef7aa2d7eb0b65308428d0
2013-12-18 14:23:33 +01:00
Jenkins
207f64cbaa Merge "Update login.defs on openSUSE" 2013-12-18 10:25:05 +00:00
Jenkins
d667324cc0 Merge "Make sure sbin paths are in $PATH" 2013-12-18 09:43:26 +00:00
Ralf Haferkamp
641840b2c7 Update login.defs on openSUSE
Some of the tripleo image elements rely useradd a for creating a group
with the same name as the user that is added. On openSUSE is feature
is turned off by default. So enable it here.

Change-Id: I120067e5e522dc6e704e7f09b724d60dee0c7f22
2013-12-18 10:19:08 +01:00
James Slagle
56f4ef0851 Make sure sbin paths are in $PATH
The sbin paths (/sbin, /usr/sbin, /usr/local/sbin) need to be in $PATH.
Since $PATH is set in the build environment from the host, if the host
does not have these paths set, they need to be added to $PATH.  In
particular, they must be set for apt-get which calls out to ldconfig,
start-stop-daemon, etc.

The previous check was insufficient because if you had /usr/sbin or
/usr/local/sbin in your $PATH, the addition to $PATH was not done since
the "if" statement evaluated to true.

Change-Id: I52b41d2eb9794e7ee0ae600215cb3ceca191582f
2013-12-16 14:42:59 -05:00
Ralf Haferkamp
7da1425438 Merge force-text-mode-console into the bootloader script
52-force-text-mode-console, running after 51-bootloader, calls
grub2-mkconfig (on Ubuntu that will happen via the update-grub call).
This call will overwrite all the changes that 51-bootloader did to the
grub.cfg. By merging the 52-force-text-mode-console code into
51-bootloader we avoid that. This also means that 51-bootloader will now
call grub2-mkconfig regardless of whether there is already and existing
grub2 configuration or not (52-force-text-mode-console did that in the
past anyway).

Additionally this commit enables the force to textmode code for openSUSE
(was previously only working correctly for Ubuntu and Fedora).

Closes-Bug: #1260323
Change-Id: Ida8e1a69df22d41ad70eb7154707c50613b536a8
2013-12-16 13:58:42 +01:00
Jenkins
37f3c4fe90 Merge "Quieten disk-image-get-kernel" 2013-12-12 16:31:54 +00:00
Jenkins
cc44d394f9 Merge "Add -U to pip install command in tox.ini" 2013-12-12 03:24:28 +00:00
Jenkins
b89cf37104 Merge "Add zypper element" 2013-12-11 20:45:54 +00:00
Jenkins
b4b69f37b5 Merge "Increase the size heuristic for images" 2013-12-11 20:45:16 +00:00
James E. Blair
e6f31981f3 Add -U to pip install command in tox.ini
Ask pip to upgrade packages in the virtualenv so that a long-lived
virtualenv is still kept up to date as external dependencies change.

Change-Id: I5a57debd5c4bc03e6d00d3801be614d9b895c508
2013-12-11 09:38:14 -08:00
Ralf Haferkamp
f8a0715c56 Add zypper element
This provides some customizations for zypper based distributions (e.g.
openSUSE and SLES). It is heavily inspired by the yum element and sets up
zypper to cache downloaded package outside of the chroot so that they can be
reused to speed up subsequent builds.

Change-Id: I775c921ee19cf7d1180fb68c1b7857ea6245a77d
2013-12-11 17:09:24 +01:00
Derek Higgins
d5b30d126e Increase the size heuristic for images
Occasionally cloud-init fails to resize the disk on first boot, this is
occuring because the filesystem has only 10% free. Nudging the disk size
up a bit more should give us a little more headroom so this is less
likely to occur.

I havn't been able to find out what value ensures we wont hit this
problem but I can say for sure this fixes all casses of the problem I
have seen.

Change-Id: Ib23f3a654151338ad91839e49b323b65b4054245
2013-12-11 14:38:24 +00:00
Ralf Haferkamp
c719e5b4ee opensuse: Update README.md
Remove the comment about the rpm-distro element. The only tiny bit we copied
from it is 01-install-bin. 01-override-zypp-arch was unneeded and has been
removed from the opensuse element before the initial merge.

Change-Id: I9886bb6e7d723690f2d2ac476d6d3aca5a367607
2013-12-11 15:17:08 +01:00
Jenkins
ca667d0a33 Merge "local-config: Configure proxy for zypper repos" 2013-12-11 09:50:54 +00:00