Commit Graph

509 Commits

Author SHA1 Message Date
Steve Kowalik
58c755cf4c Create a new baremetal element
Rather than using a script to mount the image using nbd to extract the
kernel and ramdisk, make a new element called baremetal, which contains
a cleanup.d script that will copy them out to <image name>.{vmlinuz,initrd}.

Closes-Bug: 1224669
Change-Id: I8f3569aa12148d18b1c8242b6fbbd8857894b26f
2014-01-21 16:48:20 +11:00
Jenkins
ed8264c9d9 Merge "Update to Fedora 20" 2014-01-20 21:30:11 +00:00
Jenkins
6397fd3b5f Merge "Add debian-upstart for experimenting with upstart" 2014-01-20 21:27:24 +00:00
Jenkins
037e798625 Merge "Fix mysql package mappings for opensuse" 2014-01-20 21:27:01 +00:00
Jenkins
ca2a430375 Merge "Allow adding packages to debootstrap" 2014-01-20 21:26:51 +00:00
Jenkins
c21c82bacf Merge "Convert -dev to -devel in fedora/map-packages" 2014-01-20 21:26:16 +00:00
Jenkins
cf7af05fff Merge "Fix the curl command in the ironic-deploy element" 2014-01-20 18:58:54 +00:00
Jenkins
92ab5ab5af Merge "fedora/RHEL: use env from /usr/bin" 2014-01-20 18:54:26 +00:00
Ralf Haferkamp
2e2b19fd4f Fix mysql package mappings for opensuse
Change-Id: I231df61a79e6c8686afbbd203fa6bba96497f228
2014-01-20 18:52:22 +01:00
Dirk Mueller
2def5b7727 fedora/RHEL: use env from /usr/bin
/usr/bin/env is the new canonical location of "env", use
it.

Change-Id: Ifc5fbd942d409b1f54057579c5f57ccac63d4b72
2014-01-20 16:16:41 +01:00
Jenkins
1831a2bfbd Merge "Allow use of mirrors when building Ubuntu images" 2014-01-20 11:35:31 +00:00
Steve Kowalik
48f5359efb Allow use of mirrors when building Ubuntu images
Add a script to ubuntu's pre-install.d that will change
archive.ubuntu.com to a mirror specified in $DIB_DISTRIBUTION_MIRROR.

Change-Id: Idfd5a40ec8a1912a5de5dcdcaf9795946b0b53b8
2014-01-20 15:07:04 +11:00
James Slagle
82668aacc0 Update to Fedora 20
Update the default version of Fedora to 20 for the cloud image. A few of
us have run through devtest using Fedora 20 and it's working.

To continue to use 19, just specify DIB_RELEASE=19 for your image
builds.

Change-Id: I3d48272dedcd424747db5485ae0d8e9953f04191
2014-01-17 11:12:58 -05:00
Angus Salkeld
a021b110ad Convert -dev to -devel in fedora/map-packages
If there is not a special case in package_map then
convert all *-dev packages into *-devel

Change-Id: I411935c1f3221016b56263ac9502221cd75d130d
2014-01-17 18:10:13 +10:00
Lucas Alvares Gomes
af8eeebeb8 Fix the curl command in the ironic-deploy element
We need to use the eval command together with curl so that the
$TOKEN_HEADER variable is seems by curl as two arguments: "-H" and
"'X-Auth-Token: ...". Without the eval bash would interpret the variable
as only one argument "-H 'X-Auth-Token: ...'" making the curl command
to not understand that parameter and fail to pass the auth_token to the
Ironic API.

Change-Id: I4dcfc323d6ab9b7fa207328386ef65a146a93617
2014-01-16 13:16:47 +00:00
Jenkins
016fc76d84 Merge "Symlink correct element install type" 2014-01-16 11:56:23 +00:00
Jenkins
5108838071 Merge "dib-run-parts should dereference symlinks" 2014-01-16 11:56:17 +00:00
Jenkins
71fb3e0f3c Merge "Wait for tgtd socket to be available" 2014-01-16 00:03:12 +00:00
Jenkins
e152e27937 Merge "Implement serial-console for systemd." 2014-01-16 00:03:11 +00:00
James Slagle
57ba300be4 Symlink correct element install type
The source-repositories element will now symlink the install scripts
corresponding to the set install type for an element into the install.d
directory.

Different install types are implemented by elements by writing scripts to do
the install type under install.d/<element-name>-<install-type>-install
directories.

For example, the nova element would provide:

nova/install.d/nova-package-install/74-nova
nova/install.d/nova-source-install/74-nova

source-repositories will create the following symlink for the package
install type:

$TMP_HOOKS_PATH/install.d/74-nova -> nova-package-install/74-nova

Or, for the source install type:

$TMP_HOOKS_PATH/install.d/74-nova -> nova-source-install/74-nova

Change-Id: I1bfaf39e5a98b2af904fbc6d674dbada30b27ed9
2014-01-15 14:57:08 -05:00
James Slagle
79f31d0476 dib-run-parts should dereference symlinks
dib-run-parts should dereference symlinks, and if they point to an
executable file, execute that file as a script. This can be accomplished
by using the xtype predicate in the find command instead of the type
predicate.

This change is needed so that we can dynamically symlink hook scripts
into hook directories such as install.d at runtime to support different
install types.

Change-Id: I933e7f4b4dcf16956841d8c14aa63b0f9a18fc5d
2014-01-15 14:57:08 -05:00
Clint Byrum
611a8b701a Add debian-upstart for experimenting with upstart
Upstart support in Debian is somewhat experimental, but having it would
be especially useful for TripleO which already generates upstart scripts
which will work fairly well on Debian based systems.

Change-Id: I267da5ae7bc4a484fac152a14963d96ca2ad2693
2014-01-14 22:41:25 -08:00
Clint Byrum
4ce573c582 Allow adding packages to debootstrap
Also allows setting the "distro name" so that things that change the
debootstrap signature will have a different tarball.

This is being done in advance of a debian-upstart element to allow
experimenting with debian and upstart.

Change-Id: I1a558fdaf5924c65f1cced9f9bf29d73643c30af
2014-01-14 22:41:25 -08:00
Clint Byrum
d0fd8915f6 Add support for Debian
Change-Id: Ibc6bc57dea1eab295fb23cccf40b38f3fcb285c3
2014-01-14 22:41:16 -08:00
James Slagle
8d910d10f8 Wait for tgtd socket to be available
tgtd returns execution control and backgrounds itself almost immediately
and before it has made it's listening socket available. This can cause a
race condition as the tgtd socket is not available when tgtadm is run,
resulting in an error:

failed to send request hdr to tgt daemon

Add a function to check if the socket is available before moving on to
calling tgtadm, and a wait_for helper function we can use.  We'll
check for the socket every 0.5 seconds, for up to 5 seconds.

I'm seeing this issue on almost every deploy using a ramdisk built from
Fedora 20. I'm not sure if something has changed in tgtd, but this
behavior is documented since Fedora 18 at least. In the systemd script
for tgtd, there is actually "sleep 5" to work around the problem.

See Also: https://bugzilla.redhat.com/show_bug.cgi?id=848942

Change-Id: Iffa9fc63393309ca653d592dff17316ecbea3e09
2014-01-14 12:53:03 -05:00
Jenkins
de7deb7ce9 Merge "Add new modprobe-blacklist element." 2014-01-14 13:05:55 +00:00
Jenkins
d9303ac9af Merge "Add package support to source-repositories" 2014-01-14 12:42:29 +00:00
Jenkins
85bb142641 Merge "Retry link check up to 10 times." 2014-01-14 11:12:25 +00:00
Jenkins
cccb870d92 Merge "Removing the config-applier element" 2014-01-14 11:09:14 +00:00
Dan Prince
ab9d5d3cc3 Add new modprobe-blacklist element.
This element can be used to provide a custom list
of modules via DIB_MODPROBE_BLACKLIST which
will be disabled via modprobe.d/blacklist.conf.

I'm using this to disable network adapters on the
Red Hat TripleO rack where the devices can't be
explicitly disabled via the BIOS.

Change-Id: I8a0a8ee05fa62628434d7f6422577dbf5cdd7a2e
2014-01-13 15:43:29 -05:00
Dan Prince
5ae9c5d24c Install traceroute on Fedora.
Useful for debugging, etc.

Change-Id: Ia06f8d7a76de2f082bea09c57fbb8150c9706afc
2014-01-13 15:12:39 -05:00
Dan Prince
ae28c99b5f Retry link check up to 10 times.
Updates the dhcp-all-interfaces element so that the link
check tries up to 10 times.

On some of my machines 3 times (seconds) doesn't appear to
be quite long enough...

Change-Id: Ibf7015162fc890d4de8a417b868b0301146944c0
2014-01-13 13:58:42 -05:00
Dan Prince
ae22d39472 Implement serial-console for systemd.
Update install.d/20-stty so that we try to start
a getty on ttyS0 and ttyS1 when using systemd.

Change-Id: Id5b605c74bf82855af0af4d0f95659cdd3a7b3a0
2014-01-13 13:02:54 -05:00
Derek Higgins
63a4c1e9d5 Removing the config-applier element
This is pretty much a duplicate of t-i-e/elements/os-apply-config, isn't used
in tripleo and the upstart script is out of date. We don't need to carry this.

Change-Id: I2b23d5930afae42cff28e2d5ce3d6c84224afa4a
2014-01-13 14:10:10 +00:00
James Slagle
d253ed16d8 Add package support to source-repositories
Adds a new "package" type to source-repositories. When the package type
is specified in an origin-repository-* file the package name and type
"package" need to be specified like so:

nova package

The existing map-packages mechanism can be used to map the specified
package name to the actual distribution package names.  The
pre-configured package repositories on the image will be used to install
the packages during the install.d phase of the image build.

If the repo type is package, an environment.d hook script is updated
with an environment variable which is sourced by dib-run-parts. This is
so that other hooks, particularly in install.d, will know what repo type
was used and can make the correct decision about doing a source install
or not.

Change-Id: Ief4e524664cd32d2087aeb3b1766f05e017de91e
2014-01-13 08:23:53 -05:00
James Slagle
380df31aac Remove old versions of grub2 from the yum cache
Before downloading and caching the latest version of grub2, we need to
first remove all old versions. Otherwise, the find command that writes
the grub2 version to /tmp/grub/install to install during the finalise
phase could pick one of the older versions of grub2 in the cache. This
will cause the install during the finalise phase to fail since the older
version of grub2 will conflict with the newer installed version of
grub2-tools.

Change-Id: I5b43e359db3ba81ed6283dfb41c7e89629516d38
2014-01-10 17:12:13 -05:00
Jenkins
843214246c Merge "Install tcpdump on Fedora" 2014-01-10 17:28:28 +00:00
Dan Prince
7511faa2b1 Implement dhcp-all-interfaces for systemd.
Includes:

 * Enhancements to the generate-interfaces-file.sh script
 so that it generates ifcfg network-scripts for
 distros like Fedora/RHEL.

 * Includes a new dhcp-all-interfaces.service systemd
 script which ensures network interfaces get generated
 before the network service starts on Fedora.

 * Add a new disable_interface function to
 generate-interfaces-file.sh which (on Fedora)
 will delete the ifcfg config for an interface which
 isn't plugged in.
 This is important because ifcfg-eth0 exists by default
 and we want to remove it if the NIC isn't connected.
 Previous behaviour on Ubuntu is unchanged and the
 generate-interfaces-file.sh just logs it as skipped.

 * General doc updates...

I tested this on a multi-nic machine w/ Fedora where the first NIC
exists but was not connected.

Change-Id: Ia99e312539da43caefd72aa60398d43dac5dcc8f
Closes-Bug: 1239880
2014-01-10 10:14:07 -05:00
Derek Higgins
864e209bfa Install tcpdump on Fedora
Tcpdump has a fairly small footprint and should be installed to all
images (it currently is on the ubuntu image by default). Sometimes the
reason you need it is also the reason its to late to get it.

Change-Id: I4849157515b389d534bdee755cdac6b0768fcb29
2014-01-10 14:16:30 +00:00
James Slagle
f5d4d8c711 Mark install-packages +x.
install-packages is an executable script, it should be +x.  The install
command that was installing it is setting permissions to 755, so we were
fine there.  However, I was trying to test the script outside of an
image build to test a change, and it's helpful to have it +x in the
repository for that context like all the other scripts under bin.

Change-Id: Ic300cc56d463640d119db4e310d375c66ed133ca
2014-01-07 07:54:47 -05:00
Jan Provaznik
c9cceeeb40 Make sure a loop device exists before kpartx is called
If "kpartx -av" is called and there is no unused /dev/loop* device,
then kpartx fails. Calling "losetup -f" finds first unused device
and creates it if it doesn't exist.

Fixes bug #1195264

Change-Id: I5d59b519fd8e2a7380b71a788f9d3f2331b6567f
2014-01-06 11:30:51 +01:00
Jenkins
e85bab58ec Merge "Fixed command dib-init-system not found error" 2014-01-03 10:11:25 +00:00
JUN JIE NAN
3968e5bd2c Fixed command dib-init-system not found error
dib-init-system is installed in 04-dib-init-system. Before it is
installed, 10-dib-init-system.bash may be sourced so command not found
issue found.

The patch invokes "/tmp/target.d/dib-init-system" directly in
10-dib-init-system.bash.

Change-Id: I9843224497c7dab8392261b26d62b6d79d1795ee
2014-01-03 17:42:21 +08:00
Tomas Sedovic
f176a78486 Add a package mapping for libvirt-dev
This is a dependency for the libvirt-python Python package that
ceilometer-agent-compute uses. libvirt-python is already installed globally in
the system via apt/yum, but that's not available to ceilometer-agent-compute's
virtualenv.

Change-Id: I07a12226889c6cda3bd04021240ee9714a396522
2014-01-02 12:01:03 +01:00
Chris Jones
4794df890a Log unsupported source repository types.
If we don't recognise git/tar/etc as source-repositories repotypes, we
now log the repotype, for debugging purposes.

Change-Id: I7b870807658441c928c7880291eb0284b6f97778
2013-12-20 14:53:53 +00:00
Chris Jones
4c7a3bf79f Improve source-repositories git caching.
We now unconditionally cache the repositories being fetched with
source-repositories.

Additionally, by improving the way we handle repository refs, we are now
able to build images with code taken directly from gerrit reviews.

Change-Id: Ifd4c5691f6761eb5551663e6d0aa1c0e42afced3
2013-12-20 14:48:30 +00:00
Jenkins
c1eb44c9b2 Merge "Add element to modify /etc/apt/sources.list in dib" 2013-12-19 11:01:13 +00:00
Kui Shi
5daa989fb8 Add element to modify /etc/apt/sources.list in dib
Add apt-sources element for Ubuntu OS to override the default
/etc/apt/sources.list in cloudimg.

Set DIB_APT_SOURCES with a proper sources.list file to replace
the default one.

If you want to use this element in tripleo project, set NODE_DIST or
EXTRA_ELEMENTS / UNDERCLOUD_DIB_EXTRA_ARGS / OVERCLOUD_DIB_EXTRA_ARGS
to make it take effect at build time and run time.

e.g. before running devtest.sh:
export DIB_APT_SOURCES=/etc/apt/sources.list
export NODE_DIST="ubuntu apt-sources"

Change-Id: I9ce0d03b506c2948b96382e4d6e85f0aff906450
2013-12-18 06:01:36 -08:00
Ralf Haferkamp
2da3235b82 Fix package-mappings for openSUSE
Change-Id: I6b59d62a9b3df89d8eef7aa2d7eb0b65308428d0
2013-12-18 14:23:33 +01:00
Ralf Haferkamp
641840b2c7 Update login.defs on openSUSE
Some of the tripleo image elements rely useradd a for creating a group
with the same name as the user that is added. On openSUSE is feature
is turned off by default. So enable it here.

Change-Id: I120067e5e522dc6e704e7f09b724d60dee0c7f22
2013-12-18 10:19:08 +01:00
Ralf Haferkamp
7da1425438 Merge force-text-mode-console into the bootloader script
52-force-text-mode-console, running after 51-bootloader, calls
grub2-mkconfig (on Ubuntu that will happen via the update-grub call).
This call will overwrite all the changes that 51-bootloader did to the
grub.cfg. By merging the 52-force-text-mode-console code into
51-bootloader we avoid that. This also means that 51-bootloader will now
call grub2-mkconfig regardless of whether there is already and existing
grub2 configuration or not (52-force-text-mode-console did that in the
past anyway).

Additionally this commit enables the force to textmode code for openSUSE
(was previously only working correctly for Ubuntu and Fedora).

Closes-Bug: #1260323
Change-Id: Ida8e1a69df22d41ad70eb7154707c50613b536a8
2013-12-16 13:58:42 +01:00
Jenkins
b89cf37104 Merge "Add zypper element" 2013-12-11 20:45:54 +00:00
Ralf Haferkamp
f8a0715c56 Add zypper element
This provides some customizations for zypper based distributions (e.g.
openSUSE and SLES). It is heavily inspired by the yum element and sets up
zypper to cache downloaded package outside of the chroot so that they can be
reused to speed up subsequent builds.

Change-Id: I775c921ee19cf7d1180fb68c1b7857ea6245a77d
2013-12-11 17:09:24 +01:00
Ralf Haferkamp
c719e5b4ee opensuse: Update README.md
Remove the comment about the rpm-distro element. The only tiny bit we copied
from it is 01-install-bin. 01-override-zypp-arch was unneeded and has been
removed from the opensuse element before the initial merge.

Change-Id: I9886bb6e7d723690f2d2ac476d6d3aca5a367607
2013-12-11 15:17:08 +01:00
Jenkins
ca667d0a33 Merge "local-config: Configure proxy for zypper repos" 2013-12-11 09:50:54 +00:00
Ralf Haferkamp
ba5aa8725e Fix $TARGET_ROOT usage in yum element
Fix the test for $TARGET_ROOT in root.d/50-yum-cache.

Don't use $TARGET_ROOT in pre-install, it's only set for "root" and "cleanup".

Change-Id: I048364ea08ef503a4466f3494f18f72ebf99b5c2
2013-12-10 17:56:42 +01:00
Ralf Haferkamp
f6c2901b0a local-config: Configure proxy for zypper repos
This allows using the local-config element with openSUSE based images.

Closes-Bug: #1256982
Change-Id: I670c938a44095ea7e7c31b81194acc1555212f06
2013-12-10 15:54:59 +01:00
Tim Serong
54c8508d45 Add support for building openSUSE images
This is currently experimental/incomplete.  See
elements/opensuse/README.md for details.

Change-Id: Ie5bcab15c9703cf15ceb642fb986a8afc34f96fb
Co-Authored-By: Ralf Haferkamp <rhafer@suse.de>
2013-12-05 15:18:46 +01:00
Jenkins
e8e7f45fb2 Merge "Log the repository fetches in source-repositories" 2013-12-05 13:06:17 +00:00
Mark McLoughlin
10eac21043 Log the repository fetches in source-repositories
Make the source-repositories element log which repo it's cloning from
so it's easier to tell what's going on.

Change-Id: I224f7bd98bc3d0ba94313dae9063d4ba4e25bdaa
2013-12-05 06:28:13 +00:00
Jenkins
0e536ba062 Merge "Default name for ramdisks to image." 2013-12-04 23:37:24 +00:00
James Slagle
6b220a044f Default name for ramdisks to image.
If you build a ramdisk with ramdisk-image-create and forget to specify
-o, you end up with a .kernel and .initramfs file as the image outputs.
This commit defaults the $IMAGE_NAME value to image so you'd end up
with image.kernel and image.initramfs

Change-Id: I10f3ac9ad178c32119523e4da930070e3ade4f73
2013-12-04 15:31:46 -05:00
Yuriy Zveryanskyy
bf222fe7bc Add deploy ramdisk element for Ironic
Ironic requires callback from deploy ramdisk via API with json data.
Auth token also required if Keystone auth enabled for Ironic API.
This patch adds separate element 'deploy-ironic' for Ironic deploy
ramdisk creation.

Change-Id: I5ef96711763e50307cfa17c406a9e16bffb937bc
2013-12-04 17:57:44 +02:00
JUN JIE NAN
d7e43c0861 Modify /etc/selinux/config if it exists
When `/etc/selinux/config' does not exist, although selinux has
already been disabled, the image creation will fail.

Change-Id: I9e4a9a006073fd3f708049407ef98f82c3f399d1
2013-12-03 15:35:49 +08:00
JUN JIE NAN
08797d44b5 Remove dot after TMP_HOOKS_PATH
"$TMP_HOOKS_PATH." is always be nonzero so we should remove the dot.

Change-Id: I0432bb3e21cc29806e1e2ae245dd4b6301ee9ba6
2013-12-02 12:44:11 +08:00
Jenkins
8a38b14ecf Merge "Switch Ubuntu element to installing saucy." 2013-11-27 13:30:24 +00:00
Chris Jones
a84b2763fc Switch Ubuntu element to installing saucy.
In tests, saucy has been able to match raring for the ability to bring
up clouds, so we can roll forwards and switch the default to saucy.

Change-Id: I4df35d40f902c0daa7b2761d7afea8db25526753
2013-11-27 10:10:39 +00:00
JUN JIE NAN
afc07e2b2d Enhance dib-run-parts usage message
and fix $1: unbound variable issue when run dib-run-parts --list.

Change-Id: Id18e21683f785f318bce11d401cb4f826bed1b01
2013-11-27 17:42:37 +08:00
James Slagle
476ec8feec Add mapping for tgt to RHEL element.
tgt maps to scsi-target-utils on RHEL.  This mapping already exists for
the Fedora element, so no need to add it there.

Change-Id: Ifc9a8976299458cdef50c2fbe00ba6a0c8d22f00
Closes-Bug: #1252975
2013-11-26 15:21:21 -05:00
James Slagle
f5d5667b32 Update openssl for Fedora.
We need to update openssl before proceeding as part of the image build
because the version on the Fedora cloud image is no longer compatible
with new python environments installed by virtualenv.

Change-Id: I3ed889f7382e7ca0de052ef5bdd6afeefc8bbdd8
Closes-Bug: #1254879
2013-11-26 09:09:47 -05:00
Mark McLoughlin
4f97b087d5 Fix typo in source-repositories README
Change-Id: Iaf5367099544fc5d4699140af642dfd392c7090f
2013-11-25 10:54:18 +00:00
Chris Jones
09eb230ea3 Detect udevd version and behave accordingly.
We can no longer rely on the name of the udevd binary as a hint for its
version, so instead we query the binary and behave accordingly.

Change-Id: Id4c869dfeeca5ed970169343b6b5f0f2c4499ca3
2013-11-20 11:13:06 +00:00
Jenkins
1373ed5374 Merge "Fix issue with Ubuntu grub pre-install step for ARM" 2013-11-20 09:24:45 +00:00
Ripal Nathuji
8f118af7d3 Fix issue with Ubuntu grub pre-install step for ARM
The Ubuntu build for ARM breaks when the pre-install step attempts to remove
grub components. This change gates those package removals based on whether
they are currently installed.

Change-Id: Ie169dbf12213e69b3713b3b1f9a68ae224f9fd31
2013-11-19 15:26:17 -06:00
iberezovskiy
41caf597a5 Check existence of rhel rpm key
The key 'RPM-GPG-KEY-redhat-release' may not exist in image
e.g. when you use non-default image (for example, if you build CentOS
image using rhel element).

Fixes bug #1252697

Change-Id: Iae642abb7a7f76c5528750dafedd4fd4bda56fe2
2013-11-19 20:02:12 +04:00
Chris Jones
f093d4d38d Fix troubleshooting override.
We can now actually override the $TROUBLESHOOT environment variable in
ramdisks, by not declaring it as readonly. Yay for insufficient
end-to-end testing.

Change-Id: I026c9dbdd411bf3eb945d5b18188eb1eedf3850a
2013-11-19 11:25:58 +00:00
Jenkins
9eb2719998 Merge "Add mapping for gearmand" 2013-11-18 22:06:59 +00:00
Chris Jones
a19d8ceee7 Document ramdisk troubleshooting.
We now document how to trigger a troubleshooting shell when booting a
di-b ramdisk.

Change-Id: Ief44347893fa81a7f2669143e1746a5ee99599af
2013-11-18 19:45:59 +00:00
Jenkins
45a0082fa7 Merge "Add troubleshooting override to deploy ramdisk." 2013-11-18 19:08:45 +00:00
Chris Jones
1f328087d9 Add troubleshooting override to deploy ramdisk.
We now allow operators to drop into a troubleshooting shell while
booting a deploy ramdisk, without having to have pre-configured their
desire to do so ahead of time, by means of a 10 second pause for them to
press a key.

Change-Id: I05ffa1ebaa95c83dee7bd2a2f52ba4c08928bb10
2013-11-18 17:22:26 +00:00
Derek Higgins
257a9dd139 Add mapping for gearmand
The ubuntu gearman-job-server package maps to the Red Hat gearmand
package.

Change-Id: I3ec90b2759498488fce2ecaed03effc18313d1e6
2013-11-18 16:44:14 +00:00
Jenkins
6f7eae51b2 Merge "Split network bringup out of base ramdisk init." 2013-11-18 11:38:43 +00:00
Jenkins
6fb7a22166 Merge "/bin/bash all the things." 2013-11-18 09:36:01 +00:00
Jenkins
670f1f7312 Merge "Don't block the upstart daemon if it doesn't exist" 2013-11-18 09:35:03 +00:00
Victor Lowther
dbacf3e8df Make pxe_mac accurate in two common cases.
If we are booting pxe booting using syslinux, and it has IPAPPEND 2 in
the boot stanza, then it will append the mac address of the device we
are booting from to the kernel parameters where we can get at it
pretty easily.

If we are booting physical hardware via UEFI over the network, we can
rely on the BootCurrent EFI variable to point at the boot entry for
the NIC we booted from, which will include the MAC address of that
nic.

If neither of those cases are in play, we can just fall back to the
all-physical-devices-with-links code.

This currently uses the Bash 4 support for associative arrays to handle
the netboot-in-UEFI case, if needed I can rewrite it to be Bash 3 compatible.

Change-Id: I5e50e30c60d6d732a09ab61251cbb9be08bb6113
2013-11-17 21:43:59 -06:00
Victor Lowther
43b70ce224 /bin/bash all the things.
Since we are using bash syntax in some of the element fragments,
we should make sure we use bash for all of them, so that things don't
break on systems where /bin/sh != /bin/bash.

Change-Id: If2f043c57aa4e1492b7f9839213ef6123f683612
2013-11-17 16:59:31 -06:00
Chris Jones
c9f432d45b Split network bringup out of base ramdisk init.
We now run the network bringup portion of the ramdisk init from an
init.d script, thus allowing image builders to inject code into the init
script before that point (e.g. configure network hardware).
The environment variables used by the network bringup code are retained
in the base init script so they can be overridden by init.d fragments.

Change-Id: I1ef0bb21e7f26c0ff3f02266f853ce5402bcb94d
Closes-Bug: #1252023
2013-11-17 20:37:26 +00:00
Russell Sim
b9348447b9 Don't block the upstart daemon if it doesn't exist
Check if Upstart exists before blocking the daemon.  Not all dpkg based
OS's have Upstart installed so this should help increase compatibility.

Closes-Bug: #1251949
Change-Id: I2dcb1ff3641778b5653ca5762a728398adb57da1
2013-11-17 14:46:14 +11:00
Chris Jones
5387b01af9 Make sorting of ramdisk init elements explicit.
We assemble the init script of ramdisks with the fragments from included
elements, alphabetically.
We now place leading numbers on all of the fragments we ship, to make it
more obvious to element authors and downstream users, how the process
works.

Closes-Bug: #1251706

Change-Id: I56b0d42971c8c462eddcfe1769f8124405e1233c
2013-11-15 17:18:34 +00:00
Roman Podoliaka
b01c73aadc Provide a way of determining init system used
dib-init-system script is installed into $PATH. Called
without arguments it will print the name of init system
used to stdout.

Additionally, set DIB_INIT_SYSTEM environment variable to
the init system used.

Tested on ubuntu+upstart, centos+upstart, fedora+systemd,
debian+sysv.

Closes-Bug: #1251610

Change-Id: I29668079091f6060dab66d8259890384d3bbd653
2013-11-15 17:10:38 +02:00
Derek Higgins
726a816cac Add mapping for libffi-dev
libffi header files are needed to pip install xattr, which is a
requirement of swift.

Change-Id: I69560425f7106c35ec432825eab94ca8424eb89e
2013-11-12 20:32:52 +00:00
Derek Higgins
64f4323ae7 Allow heat-admin to sudo without tty
This is needed to run scripts from devtest, in particular so that
swift ring files can be distributed.

Change-Id: Icbe8c5b24a0d494730357983f230a37a7f078de7
2013-11-12 20:26:11 +00:00
Jan Provaznik
f6f4e808d1 Remove BUSYBOX variable from ramdisk-defaults
ramdisk-defaults file is sourced before chroot to an image being created
is executed. Busybox path should be set inside chroot instead.

Change-Id: Ic0e1d3093ca9d489dd96b14157e93b336cf0baee
2013-11-11 14:36:15 +01:00
iberezovskiy
f23babe971 Check existence of directory 'lost+found'
The command `sudo rmdir $TARGET_ROOT/lost+found` will fail
if `$TARGET_ROOT/lost+found` directory doesn't exist,
e.g. when you use non-default image.

Fixes bug #1245856

Change-Id: I48c8f2f201b29912a726249023ca7d20893cc958
2013-10-29 18:51:49 +04:00
Derek Higgins
d91ececa06 Remove mapping of atftpd to tftpd-server
These are different tftp implementations, tftpd-hpa and tftpd-server are
equivalent, this mapping was added in a previous commit.

Change-Id: Idd1f554d4ee8e1d3c515d5f2f191e90abe0ff396
2013-10-25 12:27:39 +01:00
Jenkins
0e4ddaa586 Merge "Add redhat mapping from tftpd-hpa to tftp-server" 2013-10-25 11:24:20 +00:00
Derek Higgins
4508a685de Remove dependency on dracut-network
Now that we have moved onto neutron-dhcp-agent. dracut-network is no
longer needed, infact it was getting in the way. Dracut wasn't
requesting the classless-static-routes dhcp parameter but was
configuring networking. Removing this causes the network service to
redo dhcp and set the appropriate static route (specifically so
overcloud nodes are routed to the correct metadata server). This element
is still required for undercloud nodes so needs to be included there.

Change-Id: Ic23560164aff7791dd87ed2c3ad008efd248ae93
2013-10-25 09:15:40 +01:00
Derek Higgins
66b293af1c Add redhat mapping from tftpd-hpa to tftp-server
This will be needed by a patch I'll be submitting to t-e-i to
change the tftp implementation used by ubuntu, so that both
Ubuntu and Fedora can use the same implementation.

Change-Id: Ie9581265dfa2c37342e01162fe580b1948dd3e03
2013-10-24 14:55:22 +01:00
Jenkins
3144cc63be Merge "Allow for bad Fedora image mirrors" 2013-10-23 23:10:29 +00:00
Tomas Sedovic
1c132f6aa5 Add apache and mod_wsgi to Fedora's package map
We need this for the Horizon element (which should work both on debian and
fedora-based distros).

Change-Id: I3a1dacbd1823abe051ba977b92b67ed27423f405
Signed-off-by: Tomas Sedovic <tsedovic@redhat.com>
2013-10-23 15:23:42 +02:00
Derek Higgins
9947cdebd9 Allow for bad Fedora image mirrors
I am getting a lot of http 404's from Fedora mirrors, in particular in
the US. Alternatively we could get cache-url to use previously existing
files if they exist and there is an error but this may not be the
correct thing todo in all cases.

So for now we do a retry if HTTP 404 is returned to the request for a
Fedora image.

Change-Id: I7634aeac3bdfe6930fafe6d276186c09fc8587b5
2013-10-23 10:49:29 +01:00
Derek Higgins
1adb4072b8 exit 44 if http 404 is returned
This will allow the fedora element to know that there was a http 404 so
it can retry.

Change-Id: I42100a175e42af1c53909508245695ac82204fc0
2013-10-23 10:44:47 +01:00
Jenkins
194df26096 Merge "Remove old-kernels for fedora" 2013-10-17 21:01:34 +00:00
Lucas Alvares Gomes
f707a6bd82 Dracut regenerate initrd w/ the right kernel
When picking the latest version of the kernel on i386 we should prioritize
PAE kernels. Debug kernels will be ignored.

Change-Id: Ic0fc5907074ee2a5ddfbbb1db2f1c8a6060cae9f
Related-Bug: #1240873
2013-10-17 13:50:54 +01:00
Lucas Alvares Gomes
0181b177c2 Remove old-kernels for fedora
Remove the old kernels on fedora to save some space in the image.

Change-Id: Ia7ace63c286052713ac1ef3092373cc2712db11a
2013-10-17 11:51:44 +01:00
Derek Higgins
d23ba2a8a8 Conditionally add dhcp-all-interfaces
If the /etc/init drectory gets created on Fedora, it causes
os-svc-daemon to add upstart init scripts instead of systemd.
As a result none of the openstack services can start.

Partial-Bug: #1239880

Change-Id: Iec317baa3eb9ff651fa66c582d2f614993cde45e
2013-10-15 13:43:50 +13:00
Jenkins
a019fcf470 Merge "Specify distro release in fedora image name" 2013-10-14 11:51:22 +00:00
Jenkins
52956a2c3a Merge "Allow for redirects to ftp servers" 2013-10-12 15:38:39 +00:00
Clint Byrum
f96f705559 Remove \r chars from dhcp-all-interfaces
For some reason we are adding CR's to/etc/network/interfaces, but it
does not require them. They are a bit annoying to see in an editor and
serve no known purpose there.

Change-Id: I9aeeff5533f418f09fcf33edd42e5d85cd486d23
2013-10-11 11:06:16 -07:00
Clint Byrum
28190d5954 Make dhcp-all-interfaces block all interfaces
Previously dhcp-all-interfaces was only blocking the first network
interface to come up. We add an instance stanza to make it run one
instance of generate-interfaces-file per interface. We then use flock to
serialize runs of generate-interfaces-file.

Fixes bug #1233577

Change-Id: Ib16bed6b37ce0789e315ef57e05ac561470a6f2a
2013-10-11 11:06:16 -07:00
Clint Byrum
2129d78124 Fix ifquery call in dhcp-all-interfaces
Ifquery does not print anything for interfaces which only have a single
iface line. It does, however, return non-zero if the interface is not
configured at all, so we can use that to indicate whether or not there
is a configuration.

Fixes bug #1233579

Change-Id: Ia2fdafbea57e806eba99ae8ddaf395ebdcc306e1
2013-10-11 11:06:16 -07:00
Derek Higgins
b67b850a7e Allow for redirects to ftp servers
The Fedora image can redirect to a ftp server, in which case outputs a
213 if not modified.

Change-Id: I714fc3562bf188994357e1918af309d8edeecde5
2013-10-11 10:16:51 +01:00
Derek Higgins
11b116977c Specify distro release in fedora image name
The fedora element downloads the latest available image so presumably
will jump to F20 once available. Probably causing several days (weeks?)
of busted stuff. Of course it will be impossible to know when all users
of the elements are ready to switch but the least we can do is allow a
little time as a buffer.

This commit ties it down to a specific version which can then be updated
when the consumers of this element are ready. This allso follows the
same pattern as the ubuntu element.

Change-Id: I15c8e15a66e8af1bd152c27144acbc55af9da88e
2013-10-10 08:21:52 +01:00
Tim Serong
43c0082aa7 Move /tmp/ccache setup to base element
Commit c7d80dd (Cleanup mount points automatically) removed the unmount
of $TMP_MOUNT_PATH/tmp/ccache in run_d_in_target() and moved the
"rm /tmp/ccache" to elements/base/finalise.d/02-remove-ccache.  There
are two problems with this:

1) Not unmounting at the end of run_d_in_target() results in tmp/ccache
   being bind mounted muliple times on top of itself (three times, if you
   just run `disk-image-create base`).  It is eventually unmounted, but
   somehow the auto unmount code is confused, and tries to unmount it
   one more time than it was mounted, which results in an error like
   "umount: /tmp/image.THQkZxQa/mnt/tmp/ccache: not mounted".
   This doesn't actually break anything, but it's a little messy.

2) "rm /tmp/ccache" in elements/base/finalise.d/02-remove-ccache never
   succeeds in removing /tmp/ccache, because that hook is invoked by
   run_d_in_target(), *while* /tmp/ccache is mounted.

This present commit solves the above by moving the ccache setup glue out
of img-functions and into the base element's root.d.  This has the
following implications:

1) lib/img-functions is a little cleaner.

2) /tmp/ccache is available in the chroot during the root, extra-data,
   pre-install, install and post-install stages.  It is not available
   during block-device, finalise and cleanup stages as it will have been
   automatically unmounted by then.

3) /tmp/ccache won't be setup if you're building an image that doesn't
   include the base element.

Change-Id: Ief4c0a6f4ec622db6c6f652776215684178d8943
2013-10-10 16:27:09 +11:00
Jenkins
126099cd6d Merge "Use --numeric-owner when extracting base image" 2013-10-08 22:54:08 +00:00
Tim Serong
6da49c6d49 Use --numeric-owner when extracting base image
When extracting the base image without --numeric-owner, user and group
names in the tarball are mapped to uid/gid by the host.  This can cause
problems when building an image for some other distro than you're
running yourself.  For example, building an Ubuntu image on openSUSE
ends up with /var/cache/man in the image owned by 'proxy' (uid 13)
instead of 'man' (uid 6), because the host (openSUSE) uses uid 13 for
the 'man' user.  This particular man/proxy discrepancy results in
"fopen: Permission denied" errors when apt-get does its "Processing
triggers for man-db" thing in the Ubuntu system.  I wouldn't be
surprised if there were other kinks caused by this uid/gid mapping
discrepancy too, but that's the one I found so far.

The same thing can also happen with Fedora, but seems to be less likely,
or at least less obvious to me when building Fedora images on openSUSE.
But, IMO, it's better to be safe and just use --numeric-owner on all
base image untarring outside the chroot.

Change-Id: I9da5ac66dd182e7278fe4fee932093f61d35673a
2013-10-08 22:45:51 +11:00
Matthew Farrellee
ddef437cd2 Make RHEL subscription optional
The code to handle unregister of RHEL subscriptions was buggy and
broke if no subscription credentials were supplied.

Change-Id: Iac29c45f207725e31eac6487a87367fcd3d34d49
2013-10-07 14:46:57 -04:00
Jenkins
21a81d8b79 Merge "Fix mellanox module loading." 2013-10-07 03:49:46 +00:00
Jenkins
59db52aa80 Merge "Ignore empty files returned by curl" 2013-10-07 00:36:55 +00:00
Chris Jones
8c17a5d040 Fix mellanox module loading.
My previous attempt at forcing the mellanox module to load was
completely bogus. This should not be (although I lack hardware to be
100% sure).

Change-Id: I22ff88181c9c9f0c024e021eeb7f16d79715241a
Closes-Bug: #1233949
2013-10-04 09:53:03 +01:00
Derek Higgins
50ea375154 Ignore empty files returned by curl
In cases where servers ignore the Modified time, curl cancels the
download, outputs a http 200 and leaves the output file untouched, we
don't want this empty file.

Fixes bug #1234926

Change-Id: I05b0dd95dcd53ca50d88ec07f2f1ee9958b6adb7
2013-10-04 09:26:43 +01:00
Clint Byrum
4a751ba162 Generate interfaces file before cloud-init runs
Cloud-init needs to query the metadata server after the network
interfaces are configured. The upstart job "cloud-init-nonet" is
specifically in place to provide a hook to block cloud-init from
running while we rearrange network interface configurations.

Fixes bug #1233577

Change-Id: Ib5cf75d858fdb670b2abcc082e912c4644d6b169
2013-10-03 16:00:19 -07:00
Robert Collins
2684cddd3d Fix typo in dhcp-all-interfaces.
It is meant to write to /etc/network/interfaces, not
/etc/network/interfaces].

Change-Id: I15ea677edf7129ffd222c39627f7f76356f68e11
2013-10-01 20:32:40 +13:00
Tomas Sedovic
d5afe2b1fa Set locale for the profiling printf command
When LC_NUMERIC is set to a format that doesn't use a decimal point,
`printf` will fail.

Change-Id: Ie6c4d075928f47b17cc413d537fc31c9d0734bdb
Signed-off-by: Tomas Sedovic <tsedovic@redhat.com>
2013-09-30 12:10:07 +02:00
Clint Byrum
f120e122bc Add mapping for atftpd in fedora/rhel
Upcoming tripleo-image-elements changes will need this.

Change-Id: I775dda78aaee892c8b47999707574054c3d9a95b
2013-09-26 10:38:44 -07:00
Jenkins
d744f7697c Merge "Make RHEL subscriptions optional" 2013-09-25 09:47:30 +00:00
Matthew Farrellee
1d4893b8bb Make RHEL subscriptions optional
The code to handle RHEL subscriptions was buggy and broke if no
subscription was supplied.

Change-Id: Ie4f30171f716312b69d760d45f25659606dadc1f
2013-09-24 17:05:40 -04:00
Matthew Farrellee
0add227af3 Add RHEL mapping for openssh-client
Change-Id: Ic0b5355c186dea403a9535497d28330f693e6843
2013-09-24 07:46:39 -04:00
Matthew Farrellee
18acacc26a Add RHEL mapping for augeas-tools
Change-Id: I296425504f5302e21bd4ba89e772c20c2be13838
2013-09-24 07:46:39 -04:00
Matthew Farrellee
50cb156369 Add RHEL mapping for default-jre
Change-Id: Icb474af27670d1266ce02cfd8179443496b3948f
2013-09-24 07:46:39 -04:00
Jenkins
6ede4c2016 Merge "Add DIB_IMAGE_CACHE" 2013-09-19 20:47:17 +00:00
Cody A.W. Somerville
a8d8e7a30d Do not prompt on removal of apt-xapian-index from ubuntu cloud images
Change-Id: I49582a0c51aafcda6c0ad7911c291d58bb6c52a6
2013-09-19 16:35:31 -04:00
Matthew Farrellee
1d2a85d5e4 Add DIB_IMAGE_CACHE
DIB_IMAGE_CACHE will be a user override for the location where images
are cached. Default location is ~/.cache/image-create

Change-Id: I3e9b9f970864d555c9ec9436344b53f6d3d66dfa
2013-09-19 14:34:05 -04:00
Clint Byrum
2d70812e81 Remove apt-xapian-index from ubuntu cloud images
This package recently caused us some very large headaches when it
was updated for a security issue. It is completely unnecessary and
should be removed.

Note that we have recommended that it be removed from the cloud images
in launchpad bug #1227425.

fixes bug #1227420

Change-Id: Ic0d4efa7b44c46271d19576f5191c9421d07c015
2013-09-18 17:22:59 -07:00
Monty Taylor
5e3b5320e4 Remove github references
OpenStack runs git.openstack.org which is more reliable and responsive
when projects operate within OpenStack Infra. Replace all of the
references to github with referneces to git.openstack.org.

Change-Id: Ib3ece85aba6451801487b0bdbd83147e39d9e155
2013-09-18 08:17:10 -05:00
Chris Jones
b50dbb0571 Move textmode forcer to vm element.
Changing the grub config makes no sense in a build not heading for a vm
and may fail because grub is removed from images not including the vm
element. Forcing textmode for those images would be better done in nova.

Change-Id: I1c5b89e551e62df2463200b1889cb2342498c7dd
2013-09-18 01:57:35 +01:00
Jenkins
bd632a92fd Merge "Fix grub/linux text mode override." 2013-09-18 00:05:12 +00:00
Chris Jones
d799ad5beb Fix grub/linux text mode override.
Apply a wider range of hammers to the recurring issue of images booting
with graphical consoles.

Change-Id: I96345bf8f2f13d615131557b572201b9d41b6c75
2013-09-17 22:03:02 +01:00
Ghe Rivero
6d1f03bda3 Add deploy-kexec element
Boots into the new image kernel once baremetal-deploy-helper signals
it is finished using kexec utilities.

Change-Id: I705787cc394ef14200d80404ee497762ab79b452
2013-09-17 14:01:20 -07:00
Dan Prince
fb33adc5ba Consolidate the checks for /etc/grub.
Create a single check for grub2 so we skip these steps on F19.

Change-Id: I5f05adcf409be5c2532478439bcb55ab397da20e
2013-09-17 11:08:04 -07:00
Jenkins
f5e191dc8e Merge "Check cached file size when downloading an url" 2013-09-16 21:24:53 +00:00
Jan Provaznik
9a5e14cbc5 Check cached file size when downloading an url
In some cases cache-url fails when downloading an image and leaves
an empty cached file. qemu-img then fails with "Wrong medium type"
error on next run.

Change-Id: I23e91c52094f27248cf8452f192ad63646051190
2013-09-16 13:32:36 -07:00
Dan Prince
15007cc184 Fedora 19 has no grub2 conf file.
This fixes ramdisk creation on Fedora 19.

Change-Id: Ica486473cecb93feac740ae905fba5f2a3723bc2
2013-09-16 12:21:59 -07:00
Steve Baker
9ba761ef27 Install fedora grub from cached rpm during finalise
When uninstalling grub2, leave all its dependencies
including grub2-tools installed to minimise the number of packages
which need to be installed in the finalise stage.

Since the yum cache is unmounted during finalise, installing
grub2 in finalise is slowed by re-populating the yum cache.

This change copies the grub2 rpm out of the yum cache so it can be installed
from file during finalise.

This should prevent disk becoming full during finalise on Fedora.

Closes-Bug: #1217185

Change-Id: If095adc4abb52a19a3aa0b1caebfb3e4d8f605ef
2013-09-15 09:11:58 +10:00
Jeff Peeler
ccd7b86b60 Remove -r option from kpartx for successful build
This option does not exist on RHEL hosts and matches what is
currently present in elements/rhel/root.d/10-rhel-cloud-image.

Change-Id: I578233c1f37d035c67600fc60e7c4eb4ff75cbb3
2013-09-12 17:56:30 -04:00
Matthew Farrellee
43b96d91c2 Add package mapping for default-jre
Change-Id: Idffbc84ea2afaf2584c1bb79729c3e0cf52b851e
2013-09-10 20:01:21 -04:00
Jenkins
d339f9b25e Merge "Add --list support to dib-run-parts." 2013-09-04 10:59:46 +00:00
Chris Jones
6311388892 Add --list support to dib-run-parts.
Our custom dib-run-parts now supports --list for better compatibility
with some OS-provided run-parts tools.

Change-Id: I2feb615b2094d9d6522bfe9b422362223bb8e652
Closes-Bug: #1190521
2013-09-04 09:44:48 +01:00
Steve Baker
c182fef45c On Fedora, use Linux Foundation bzr lsb_release
Installing redhat-lsb-core adds 87M to the install due
to bugzilla #1002342

lsb_release is a platform agnostic script, so this change fetches
a single file via source-repositories from linuxfoundation bzr http
and installs it to /usr/local/bin

Change-Id: I4c979d4eb5e34a753d9143d9dcfe4e5d08f74a0d
2013-09-04 10:16:49 +12:00
Steve Baker
9e5bc37265 Add support for file to source-repositories
Adds a file type to source-repositories to allow individual files
to make use of source-repositories caching.

REPO_DIRECTORY has been rename to REPO_DEST since it is now sometimes
a file.

This change also fixes the incorrect path to the cache-url in the
tar type.

Change-Id: Ib79e41969c982a02f0235318d9f254b39c3c6d93
2013-09-04 10:16:49 +12:00