Commit Graph

1676 Commits

Author SHA1 Message Date
Jenkins
d506c42ec4 Merge "Deprecate map-packages, replaced by pkg-map" 2014-11-20 01:41:45 +00:00
Ryan Brady
81b4c00fdc Update RHEL Registration
This patch updates the 00-rhel-registration script to not
not fail a stack when metadata isn't found.

Change-Id: Ie305fed79e4baadf1a03c4a3d06a23cf36e92f77
2014-11-19 18:05:24 -05:00
Jenkins
04e8ce0ece Merge "Enable RHEL Registration" 2014-11-19 22:34:37 +00:00
Jenkins
a80ff47dec Merge "Simplify Dracut cmdline script" 2014-11-19 20:26:10 +00:00
Ryan Brady
4178c0c69d Deprecate map-packages, replaced by pkg-map
This patch writes a warning out to stderr to notify element authors
that may be using map-packages to migrate to pkg-map.  This patch
also prints out a warning during image-create that lists the specific
elements that are not using pkg-map.

Change-Id: I7e2a7611dd5650815736ce998aa94a7833193d06
2014-11-19 12:04:12 -05:00
Jon-Paul Sullivan
8dd97597ea Allow multiple identical sources
The source-repositories element puts source in place in the generated image
for later installation.  It is possible for two elements to define the same
source to later install, and as long as the remote location and reference to be
installed is identical, we should not error.

Change-Id: Ic7dfb1961486c59d2005b5c1e7587267ea42e999
2014-11-19 12:03:16 +00:00
Tim Potter
018b1d4147 Add DIB_DEBOOTSTRAP_EXTRA_ARGS environment variable
This allows extra arguments to be passed to the debootstrap
command when creating an image using the debian element.

Change-Id: I87eb856365ff6a17f051e2e56587235648328d57
Co-Authored-By: Ghe Rivero <ghe.rivero@gmail.com>
2014-11-19 11:22:16 +00:00
Jenkins
8986283d41 Merge "Remove duplicate binary-deps from dracut-ramdisk" 2014-11-19 09:59:37 +00:00
Ghe Rivero
830de28e96 deploy-kexec depends on deploy
It was depending only on deploy-baremetal. Changing it to 'deploy' could
be used with both baremetal and ironic deployments

Change-Id: Ia2600080b488f7d259a9baf111767e82b6cc933d
2014-11-18 15:29:53 +01:00
Ben Nemec
8ee66ec8ec Remove duplicate binary-deps from dracut-ramdisk
These are already listed in the respective deploy-* elements, and
if we include them here then they get added to every ramdisk built
with dracut.  That causes issues for elements that don't need
something like tgtd because it won't be installed and the
binary deps script will fail.

Change-Id: Ibdcf7d200d4f7136396b63404cd966f7557043e0
2014-11-14 12:43:49 -06:00
Yuriy Zveryanskyy
676b1b738d Add element for building ramdisk with ironic-python-agent
This element creates kernel and ramdisk files based on Fedora,
example:
disk-image-create -a i386 -o test fedora ironic-agent disable-selinux

Change-Id: Ifa133d1680b81cb87d32a405aa7d7b40fe91f835
2014-11-13 19:30:44 +02:00
Jenkins
ce67ce87ea Merge "Support installing packages by default" 2014-11-13 10:27:56 +00:00
Ben Nemec
f528aa316e Simplify Dracut cmdline script
We don't actually need a real value for the root kernel param, and
requiring one causes problems for things like the discovery ramdisk
that don't pass in a disk= parameter.  Dracut seems to be happy to
take /dev/zero as the value, so we can just always use that.

Change-Id: Ia724f0214c26aa18c6f8f41f2c48d7f25b52ee6c
2014-11-12 11:18:53 -06:00
Ryan Brady
1aecb41b7e Add deprecation warning when using map-services
This patch writes a warning out to stderr to notify element authors
that may be using map-services to migrate to svc-map.

Change-Id: Ic80db16c607958d025e89b3a4058a9cbb568938e
2014-11-12 17:03:49 +00:00
Derek Higgins
c26a9ee4b1 Install lsb_release from package
The URL we were using to download lsb_release from no longer works,
install from package so this is not affected by disappearing URL's
in future.

This was originally installed from a URL because the package dependencies
caused a 87M increase in the size of the base image. For fedora the
increase is now 27M which is a little more tolerable.

Change-Id: I6e56a4a81786b33c5c6b92df2bd8236cd3f19670
2014-11-10 11:28:49 +00:00
Ryan Brady
ac8514c089 Enable RHEL Registration
This patch documents the 00-rhsm script as <= RHEL6 as it doesn't
work with the latest release of Satellite.  The Red Hat Network
(RHN) method of registration is being phased out in favor of Red
Hat Customer Portal and Satellite 6.  The subscription-manager
command line tool is the preferred method of registration.

The registration of RHEL is required to enable repositories for
software installation during image creation time.

This element provides functionality for registering RHEL images
during the image build process with diskimage-builder's
disk-image-create script.  The RHEL image will register itself
with either the hosted Red Hat Customer Portal or Satellite to
enable software installation from official repositories. After
the end of the image creation process, the image will unregister
itself so an entitlement will not be decremented from the account.

Boot-time registration is supported through metadata.  Please see
the configuration in the README for more information.

Change-Id: Ia9ef377cc4ed9595633888bfb248a1224e04b542
2014-10-31 12:36:43 -04:00
Jenkins
cafad0a148 Merge "Update Debian repo to retrieve signed Release file" 2014-10-31 01:27:19 +00:00
Jenkins
d72435a86b Merge "Optimize speed of deletion in find command" 2014-10-30 12:53:28 +00:00
Jenkins
50085793cc Merge "Unset trap before dracut ramdisk build script exits" 2014-10-30 11:43:09 +00:00
Jenkins
1e5ec9adb3 Merge "Use binary-deps.d for dracut ramdisks" 2014-10-30 06:25:30 +00:00
Dan Prince
022cb2c1c2 Support installing packages by default
Adds new disk-image-create --install-type option which
can be used to enable alternate install types. The
default install type is 'source'.

The motivation is to eliminate elements like
enable-package-installs which require coupling and also
don't work with elements in multiple element repositories.

This patch does not prevent you from using the previous
DIB_INSTALLTYPE_ variables to customize the install type,
rather it just changes the default so you don't have to
set it so often when using things like packages.

Change-Id: Icee98440fc2251728f2dca30e7c4789a0fd89b93
2014-10-28 15:05:53 -04:00
AzherKhan
bb38d0e325 Optimize speed of deletion in find command
I would like to recommend to use + instead of \; in the find
command. As this will ensure the removal of all selected
directories in a single invocation.

Hence improve the speed of deletion.

Change-Id: I409fe11aae217afb6f790491591005c679264ed4
2014-10-27 21:04:09 -07:00
Jenkins
dbe2846264 Merge "package-installs assumes packages have a pkg-map" 2014-10-28 00:15:19 +00:00
Gregory Haynes
c27e32f769 package-installs assumes packages have a pkg-map
Our package-installs script fails when installing a package which does
not have a pkg-map but a pkg-map file exist for the element.

Change-Id: I3dab802e23bccfc916efcc1c70c6ce6c4a9ccf67
2014-10-27 09:42:58 -07:00
Jenkins
3c8cf6a724 Merge "Move busybox binary-dep to ramdisk element" 2014-10-27 08:30:48 +00:00
Jenkins
34c273d8df Merge "Make diskimage-builder work in Docker." 2014-10-24 20:13:45 +00:00
Jenkins
2d80e53186 Merge "iso element to use 'search --label' for grub" 2014-10-24 15:55:44 +00:00
Jenkins
9d252ba318 Merge "Handle extra spaces in merge-svc-map-files" 2014-10-24 07:42:02 +00:00
Ben Nemec
0db6b9b401 Unset trap before dracut ramdisk build script exits
A similar change was needed in the normal ramdisk element as part
of ae928057bd to avoid running
cleanup immediately after the ramdisk build completes.

Change-Id: Ia96e2d8011b88ed96cc6727914c5a5d2dea59757
2014-10-23 20:02:08 -05:00
Ramakrishnan G
8c9ee78ab5 iso element to use 'search --label' for grub
Current iso element uses 'search --file' to find the
grub root.  This is a wrong approach as it may find
some other partition containing /vmlinuz. Instead modify
it to search for grub root by the label of ISO image.

Closes-Bug: 1384826
Change-Id: Id4217be3420597bed2f80a96788928259ec91582
2014-10-23 23:04:17 +05:30
Ghe Rivero
4543294487 Update Debian repo to retrieve signed Release file
After bootstraping a Debian image, the repository keys
are installed to verify the packages signatures, but the
Release signature file is missing. Updating the repo,
will retrieve a new InRelease file (inline signed).

Change-Id: I14f0d22cc9c72be9b07f3708270359bc8cff112d
2014-10-23 07:38:31 +00:00
Jenkins
1e55938148 Merge "iso element to build bootable ISO images" 2014-10-23 06:05:34 +00:00
Ben Nemec
bb8126e887 Use binary-deps.d for dracut ramdisks
Instead of hard-coding a list of binaries to include in the dracut
ramdisk, use the existing binary-deps.d functionality to provide a
list.  This will allow other ramdisks (such as discovery) to add
the binaries they need.

Change-Id: Ib7ffa15e08db1cc45e93a8f2a5c01369772c93ff
2014-10-22 15:13:48 -05:00
Jenkins
2337532954 Merge "svc-map requires PyYAML" 2014-10-22 08:15:54 +00:00
Jenkins
602ec41dd7 Merge "Revert "introduce $SYSTEMD_SYSTEM_UNIT_DIR"" 2014-10-21 21:15:25 +00:00
Ben Nemec
088f0eb7ac Move busybox binary-dep to ramdisk element
Busybox should be installed and managed from the ramdisk element,
since that is common to all elements that need busybox.  In
addition, we do not want busybox installed when building Dracut
ramdisks, so it can't be in the deploy-* elements.

Change-Id: I2656d20b466138f7f6dfcf558ba90c6909151d3c
2014-10-21 16:05:19 -05:00
James Slagle
ccd292c10a svc-map requires PyYAML
svc-map requires PyYAML, so install it. It's not always installed on all base
images.

Change-Id: I7ce5325e8afa0047f67153baa1fc7d4a90196c88
2014-10-21 16:13:16 -04:00
Ramakrishnan G
13aed64e97 iso element to build bootable ISO images
This commit adds a new element named 'iso' to build a bootable
ISO image for the kernel/ramdisk emitted by the 'baremetal' or
'ramdisk' element.

Change-Id: I89d175a29e2d0bc64b47fe527f0d0f6875f6849a
2014-10-21 14:37:57 +05:30
Michael Kerrin
730e7bd76e Handle extra spaces in merge-svc-map-files
During internal testing we are getting the following error from
merge-svc-map-files
File "/tmp/image.K2MYCphY/hooks/extra-data.d/10-merge-svc-map-files", line 54, in main
    with open(data_path, 'r') as dataFile:
IOError: [Errno 21] Is a directory: '...diskimage-builder/elements/svc-map'

Somehow IMAGE_ELEMENT contains a extra white space that manifests itself
as an element without a name. It is very hard to find where this is coming from so
instead this patch makes merge-svc-map-files more robost to this situation

Change-Id: Id1500ead8a77d691408617dcdc4e095bc5775be8
2014-10-20 18:16:39 +01:00
Jenkins
8ba6a26e0a Merge "Allow source-repositories to be disabled completely" 2014-10-20 11:35:40 +00:00
Jenkins
bb07de7fe9 Merge "Enable dracut deploy ramdisks" 2014-10-20 11:03:54 +00:00
Ben Nemec
eed30adc48 Enable dracut deploy ramdisks
The element builds dracut from source on Ubuntu because the
Ubuntu dracut package is broken and very old, so it can't be
installed properly and causes a number of other issues that
are fixed by using a newer version of Dracut.

This initial version should work in virtualized environments.
Further validation of its suitability for real baremetal
deployments will need to be done in the future, but this should
be sufficient to enable that work.

Regarding Dracut specifically, in order to limit the changes
needed in the existing scripts this element continues to use a
cut down version of the /init script that we were building for the
existing ramdisk.  However, instead of running it as pid 0 it is
run as a Dracut pre-mount hook.  This allows Dracut to set up all
of the hardware and system bits, while falling early enough in the
Dracut sequence to complete the deployment before Dracut would try
to boot off the hard disk.

bp tripleo-juno-dracut-ramdisks
Change-Id: I144c8993fe040169f440bd4f7a428fdbe3d745cf
2014-10-17 21:44:09 -05:00
Gonéri Le Bouder
a9006ee1e7 Revert "introduce $SYSTEMD_SYSTEM_UNIT_DIR"
This reverts commit 4bf38a829a.

This variable is pointless since we can always replace it with
/lib/systemd.

Commit I459f7514ab35082d31607968252a9005fa25de2a will fix the issue
in tripleo-image-elements.

Change-Id: I524badb6836a0d04a5f6e6a5b5d95920fd923ef6
2014-10-17 10:29:34 +02:00
Jenkins
2b60bea961 Merge "Allow for multiple image outputs from raw source" 2014-10-16 21:56:28 +00:00
Jenkins
e289b31d12 Merge "Consolidate lsb_release source-repositories" 2014-10-16 21:14:05 +00:00
Jenkins
5b71da8899 Merge "Remove first-boot.d support" 2014-10-16 18:36:59 +00:00
Jenkins
4ec80ee1f0 Merge "Don't re-install cloud-init for centos7 images" 2014-10-16 17:39:20 +00:00
Ghe Rivero
0b86194216 Disable all interfaces on eni systems
We are autodetecting and configuring devices in dhcp-all-interfaces
so having one configured in /etc/network/interfaces by default
is redundant and slows boot down.

Change-Id: Ic4e8a0668c793d21ed2dd96908649c9a77264f67
Closes-Bug: #1239480
2014-10-16 12:27:22 +00:00
Ghe Rivero
873de01491 Remove first-boot.d support
After being deprecated two releases ago, finally remove any reference
for the support of first-boot.d

Change-Id: I08d67404ef48cad61db3b18fb86e970abfa5d2b6
2014-10-16 09:48:14 +00:00
Ben Nemec
0eccd2808c Allow source-repositories to be disabled completely
In our official image builds we are only allowed to use resources
that are "blessed" by the build system.  This means that external
things like git repos and tar files are not allowed.  Currently,
even in offline mode source-repositories expects those things to
be available in the cache, so we need a way to disable it entirely.

This change adds an environment variable NO_SOURCE_REPOSITORIES
that does so.  It can be set in an environment.d script so elements
that might rely on a source repository will know it's not available.
The 02-lsb script in redhat-common is one such example and is
updated to handle this case.

Change-Id: I0de63bee6ad79733d6711478c707a9b41593e85f
2014-10-15 12:24:50 -04:00
Ian Wienand
97678dcda5 Move to final release of CentOS7 images
This is the final released image; it is the same one deployed in many
cloud provers such as HP and AWS.  It is also safe from all known
vulnerabilities up to its release

Change-Id: Ib8ba561662d2a98da55eb292b850dc019226dd59
2014-10-15 15:35:51 +11:00
Ben Nemec
0d08925892 Consolidate lsb_release source-repositories
Each distro had its own copy of the file, but they were all the
same and it's actually installed in redhat-common so it belongs
there.

Change-Id: Iad15f39fd28d1c4e20d30dcb2eb0ae6f2fa2b015
2014-10-14 15:09:44 -05:00
Clark Boylan
ae928057bd Allow for multiple image outputs from raw source
When uploading images to multiple clouds it is possible that the same
image will be needed in multiple formats to accomodate hypervisors
across clouds. Update disk-image-create's -t flag to take a list of
desired output image formats so that a single disk-image-create can
output all of the desired image formats.

Change-Id: If121b2342ae888855ba435aa3189f039e985b812
2014-10-13 09:16:42 -07:00
Ian Wienand
83656127da Don't re-install cloud-init for centos7 images
The centos7 images come with the base version of cloud-init already
installed.

---
$ virt-inspector CentOS-7-x86_64-GenericCloud-GA-7.0.1406_01.qcow2
...
      <application>
        <name>cloud-init</name>
        <version>0.7.5</version>
        <release>10.el7.centos.1</release>
        <arch>x86_64</arch>
      </application>
...
---

I think this is a hang-over from early rhel7 images that didn't have
it.

Change-Id: Idf488942222d9e6bca4d66f04f5af42e6707944d
2014-10-10 15:52:26 +11:00
Jenkins
4b05499384 Merge "Move install bin from rpm-distro to yum" 2014-10-10 01:20:51 +00:00
Ben Nemec
e5a2e07301 Check for epel before installing it
instack runs this script on a system where epel has already been
installed, which causes the yum command to fail.  Only installing
the epel-release rpm when it isn't already present addresses the
issue.

Change-Id: Ia179d7c7ed69ea5f785b5dcd16394c7663a145cc
2014-10-09 13:00:12 -05:00
Jenkins
3180c63338 Merge "Ensure epel7 is installed only on rhel7/centos7" 2014-10-08 14:00:12 +00:00
Ghe Rivero
6484b9b026 Move install bin from rpm-distro to yum
rpm-distro element was trying to install bin programs into the
image but it doesn't provide any. Move that logic into the yum
element which provides the install-packages bin.

Change-Id: Id5339fc7ffbef6327fe41f50a3e39ab30b6d0320
2014-10-08 07:43:19 +00:00
Jenkins
0f7d9af998 Merge "Move dpkg manifest creation to finalise" 2014-10-07 01:21:28 +00:00
Jenkins
927f437cd0 Merge "Save extended attributes when creating tar" 2014-10-07 01:16:33 +00:00
Jenkins
a9086da83f Merge "Fix openSUSE cloud image download and extraction" 2014-10-06 13:25:05 +00:00
Monty Taylor
3acc866f6d Use DIB_IMAGE_CACHE everywhere
disk-image-create processes a DIB_IMAGE_CACHE variable and exports it,
but there are several elements that ignore the value and wrote out
the base location themselves. Use the variable everywhere so that it
will get overridden everywhere.

Change-Id: I00fff354d6c931ad67cf3052d055f0e4604dfdc8
2014-10-05 13:58:32 -07:00
Ben Nemec
edd7477891 Save extended attributes when creating tar
The way redhat-common's extract-image script was creating the base
tarball caused file capabilities to get dropped, which meant that
things like ping in RHEL 7 images was unusable for regular users.
This change adds the necessary options to the tar call to maintain
as many extended attributes as possible.

--acls and --selinux are intentionally omitted, and the selinux
xattrs are filtered out because all of those items cause issues
in our chroot environment.  We restore selinux attributes at the
end of the build anyway so that shouldn't be a problem.

bz reference: https://bugzilla.redhat.com/show_bug.cgi?id=1144149

Change-Id: Ibff99ce9bde01bc5ecf95dc3a5d3e2cebe5015b9
2014-10-03 23:05:17 -05:00
Jenkins
07f11be024 Merge "Install openstack-selinux on RHEL" 2014-10-03 12:44:46 +00:00
Jenkins
a31bae8f60 Merge "Add unit test for cache-url" 2014-10-02 20:28:28 +00:00
Giulio Fidente
ee001cf4bd Ensure epel7 is installed only on rhel7/centos7
Change-Id: I5812812f7d2cb5ba2e310e33d6e01cda8823c1c4
2014-10-02 12:14:25 +02:00
Jenkins
42e79c4a6e Merge "Fix $DISTRO_NAME usage for centos7 element" 2014-10-02 10:10:04 +00:00
Richard Su
c1d4cd13e9 Update EPEL release rpm
EPEL 7 was released September 30. The beta release rpm was removed
from the repo and was replaced with the final release rpm.

Change-Id: I103a2f7abd7563869b9e39ccc1fbb3cfbd9ff6ae
2014-10-01 22:26:18 -07:00
Ryan Brady
c8037cc73f Fix $DISTRO_NAME usage for centos7 element
In both of the changed files in this patch, the $DISTRO_NAME usage
is checking against "centos".  The valid value for the centos7
element for $DISTRO_NAME is "centos7"[1].

[1] http://git.io/8IvuTw

Change-Id: I0c4a4b2ecf87c7a9283d10071f6feb494a6c716a
2014-10-01 21:42:47 -04:00
Ben Nemec
58d6c6478a Add unit test for cache-url
A sanity check that cache-url does the right thing when presented
with the appropriate curl return codes.  This change brings in the
test code for scripts from tripleo-image-elements, which needs to
be factored out into a place both projects can use it.

I'm stubbing out curl so we don't have a dep on curl in our unit
tests, and because I've seen some strange behavior out of curl in
the gate that caused random failures in this test.

Change-Id: I31e6b06b45415feec7285511d07e65eb78d0d045
2014-09-30 16:39:21 -05:00
Richard Su
8baf819b08 Install openstack-selinux on RHEL
The openstack-selinux package contains policy updates for OpenStack.
Updates are accrued in openstack-selinux before they are merged into
the upstream selinux-policy package.

Change-Id: I7c00e5cae17489202fe7aee3e656fca27ab630ba
2014-09-29 21:53:22 -07:00
Jon-Paul Sullivan
4d65e44472 Move dpkg manifest creation to finalise
As some finalise steps can install packages we need to generate the
dpkg manifest after that has occurred.

Change-Id: I2177db2e64d4d9c21deeac7cf017919888a2d524
2014-09-29 16:08:15 +01:00
Ben Nemec
dde3d24213 Allow custom rootfs labels
Per [1], our current root label of cloudimg-root does not work with
XFS because XFS only allows 12 character labels.  This change
addresses that by allowing a custom rootfs label to be specified
in the call to disk-image-create.  There should be no backwards
compatibility concerns as the default label is unchanged.  Any
external elements dealing with the label would need to be updated
to support this new feature, but should continue to work as before
as long as a custom label is not specified.

[1]: https://bugzilla.redhat.com/show_bug.cgi?id=1139584

Change-Id: I596104d1a63b5dc6549e8460a1ae3da00165ef04
2014-09-26 17:44:08 -05:00
Jenkins
89356c6d34 Merge "Refactor ramdisk element to allow alternate implementations" 2014-09-26 19:23:53 +00:00
Jenkins
78be2a21fe Merge "Add svc-map element." 2014-09-26 18:36:15 +00:00
Ryan Brady
e1853a7d5b Add svc-map element.
Adds a new svc-map element which can be used to install
services based on an in element 'svc-map' YAML config
file format.

This change is intended to decouple elements from DIB
and allow new elements to support multiple distributions
(with various service naming schemes) without having to
constantly maintain DIB's various bin/map-services files.

This patch ensures all of the YAML files are compiled into
a single file without namespaces.  The YAML compilation
process occurs during image creation time and errors if
conflicting mappings are found.

Change-Id: Id11433ea342aace71a358936a7ca3151ec11d506
2014-09-26 12:00:38 -04:00
Ben Nemec
5aca301693 Refactor ramdisk element to allow alternate implementations
In preparation for enabling Dracut-based ramdisks, this change
factors out functionality that is common to both busybox and Dracut
ramdisks.  Said functionality is moved to a ramdisk-base element
which is added as a dependency of the ramdisk element.  ramdisk now
only contains the functionality specific to building busybox-based
ramdisks.

bp tripleo-juno-dracut-ramdisks
Change-Id: Iad2907c8be491c88727d87ed5e5a720e5beb66c3
2014-09-26 00:13:55 -05:00
Jenkins
dbc60e8942 Merge "Change order for yum-repos file in centos7 element" 2014-09-23 11:52:45 +00:00
Jenkins
0b9ffe4a74 Merge "Provide ability to build images for other architectures" 2014-09-23 03:50:38 +00:00
Jenkins
b308c844ae Merge "Adds EPEL repo, cleans up rhel7 repos" 2014-09-20 19:12:05 +00:00
Jenkins
0ddd704e08 Merge "Update to newer Centos7 images" 2014-09-19 22:05:10 +00:00
Ryan Brady
dfd99460c7 Adds EPEL repo, cleans up rhel7 repos
This patch adds a new EPEL element for use with EL-based distros
in accordance with the official docs[1].  This patch also
modifies the rhel7 element to not use the hard-coded repo file.

[1] https://fedoraproject.org/wiki/EPEL

Change-Id: Ib2fa24b2d519837271a0e213a2dc5a012cc31f3c
2014-09-19 07:57:12 -04:00
Ian Wienand
45afb6d86c Update to newer Centos7 images
Update to newer Centos7 images as announced at [1].

Changes included:
 * update to GA image
 * respect TMP_DIR for image working directory
 * use suffix-match rather than prefix-match to build RAW_IMAGE name
   to be more robust against weird names
 * partition layout work-arounds not needed any-more, all one
   partition.

Resulting image booted locally with kvm and virtualbox (after
conversion to vdi)

[1] https://www.redhat.com/archives/rdo-list/2014-August/msg00045.html

Closes-Bug: #1363146
Change-Id: Ie24033468b78587ea87188ee1b843b26895798ff
2014-09-18 10:48:40 +10:00
Jenkins
41ba1d2b9c Merge "Add ceilometer service mappings" 2014-09-17 16:56:30 +00:00
Jenkins
6518876a5f Merge "Use dib-run-parts from dib-utils" 2014-09-17 14:31:49 +00:00
loki
414edd44c2 Provide ability to build images for other architectures
Adds binaries specified by user to chroot environment
allowing to build  images on amd64,i386,armhf,arm64
architectures.
Closes-Bug: #1332458
Change-Id: If6e63a472ee85559b93b5e6b96d9c3ddcf7bcc98
2014-09-16 22:50:17 +05:30
Jenkins
93be491c46 Merge "Reset yum configuration in post-install.d" 2014-09-15 23:03:26 +00:00
Jenkins
137becad69 Merge "Run environment.d hook for manifests earlier" 2014-09-15 21:23:41 +00:00
Jenkins
7ec2c7cdc4 Merge "Add vmedia boot support for deploy-ironic element" 2014-09-15 16:07:09 +00:00
Uwe Grawert
7b9b82a4d5 Change order for yum-repos file in centos7 element
Due to changes in centos7 element to use package-installs the order
of yum-repos in pre-install.d needs to be changed, to make yum repos
available before package-installs is called to install packages.

Change-Id: I47b584506465d90247f30bad515a32323e52180e
2014-09-15 14:49:46 +03:00
Victor Lowther
38b14df72c Make diskimage-builder work in Docker.
When running inside a Docker container, we cannot rely on devices in
/dev/mapper to be automagically created by udev, because we probably
don't have a udev at all.  To work around this, run dmsetup mknodes
after every kpartx run.

Change-Id: If7e30579224ce54c5ed26d08974d8293c144719a
2014-09-15 06:43:53 -05:00
James Slagle
9fa28a31f1 Add ceilometer service mappings
Add the service mappings needed for ceilometer elements.

Change-Id: Id139a7c208659ead6330842bec01f7466d7ba417
2014-09-12 09:00:18 -04:00
Jenkins
60fdfb26dc Merge "Fix ramdisk pkg-map for openSUSE" 2014-09-12 09:54:03 +00:00
Jenkins
e671dbb490 Merge "opensuse: Support pkg removal in install-packages" 2014-09-12 09:28:09 +00:00
James Slagle
7c0e30fac9 Run environment.d hook for manifests earlier
15-pip-manifests depends on the variables defined by the manifests
element, so we need to run this one earlier. I would have thought things
were alpha sorted after the numerical sort...but I just ran into this,
so apparently not.

Change-Id: Ifedf544222b5a8eb7630efda609eb6a5b9629de3
2014-09-11 21:09:13 -04:00
James Slagle
8175a7ef4d Reset yum configuration in post-install.d
When using the yum element, we should reset the changes we've made to
/etc/yum.conf during post-install.d. Otherwise, this build time
configuration is propagated into booted instances.

Change-Id: I1eea586ca0fefe9bc0cf91fedefcbd141a536fa2
2014-09-11 20:59:50 -04:00
Jenkins
f4011996ee Merge "Lock around extract-image downloads and extracts" 2014-09-11 11:56:40 +00:00
Ramakrishnan G
10d72ddb22 Add vmedia boot support for deploy-ironic element
deploy-ironic element currently retrieves token over tftp and
expects BOOT_IP_ADDRESS to be set.  This commit adds support for
retrieving token file from virtual media if node booted from
virtual media. Also corrects the issue for BOOT_IP_ADDRESS not set
for virtual media boot.

Change-Id: I3d5f1779b9b17842360860c7778baa01db1e1a52
Closes-Bug: #1356339
2014-09-11 15:41:46 +05:30
Thomas Bechtold
80b1f49f77 opensuse: Support pkg removal in install-packages
Add "-e" option to remove packages. This is needed for
elements/package-installs/bin/package-uninstalls which calls
"install-packages -e".

Change-Id: I673942f505d5278e6015324950f6e1455ba50805
2014-09-11 11:30:09 +02:00
Jenkins
9fb2c1425a Merge "debian: exclude contents of /tmp from debootstrap tarballs" 2014-09-10 03:10:04 +00:00
Jenkins
5694954959 Merge "Use package-installs in more elements" 2014-09-09 18:42:09 +00:00
Jenkins
bab742c854 Merge "Changed serial console setup" 2014-09-09 12:42:31 +00:00
James Slagle
07c22185bd Use package-installs in more elements
Updates additional elements in diskimage-builder to use the declarative
package install support provided by the package-installs element.
package-installs does not yet support the pkg-map functionality, so
elements using pkg-map are not migrated. That support will come in the
near future.

Change-Id: I3d36adad317ba44326eabd95243d45807e2a8a16
2014-09-08 15:16:14 -04:00
Jenkins
d78421642b Merge "Formatting fixes for dhcp-all-interfaces" 2014-09-05 03:28:52 +00:00
Jenkins
9785c27b15 Merge "Use package-installs in fedora" 2014-09-04 20:26:26 +00:00
Jenkins
c61aa82f14 Merge "package-installs for pre-install.d/post-install.d" 2014-09-04 20:26:18 +00:00
Jenkins
2399a51133 Merge "Add support for flashing ILO BMC's from ramdisks" 2014-09-04 20:00:18 +00:00
Sandhya Ganapathy
697e771e4d Fix for host env leak into grub
While building images in a dual boot host,
/dev/sda is leaking into build's grub config.
It is visible in the logs that the commands
GRUB-INSTALL and GRUB-MKCONFIG probes Host OS environment.
This is avoided by setting grub_disable_os_prober
to true in /default/grub config file.

Change-Id: I7408de7b3bc8a62f8bf103f00a88a596f988353c
Closes-Bug: 1166491
2014-09-04 13:59:57 +05:30
Jenkins
f4ba47467b Merge "Remove disable_interface from dhcp-all-interfaces" 2014-09-03 20:44:15 +00:00
Jenkins
e2d3bb0016 Merge "Add element-manifest" 2014-09-03 12:49:15 +00:00
Jenkins
1cc95b86d6 Merge "Map dkms package for SUSE in base element" 2014-09-03 11:31:22 +00:00
Thomas Bechtold
1757ef20ea Fix openSUSE cloud image download and extraction
Use the latest available version of the rootfs (instead of a qcow2
image) to simplify rootfs extraction.

Change-Id: Ic3a646b923738464b5217d799758b6980b2deaf2
2014-09-03 09:55:14 +02:00
James Slagle
d4bbc30a11 Add element-manifest
Add an element that writes a manifest of all the elements used in the
image build.

Change-Id: Icd1863fa9ece1c303208d20b2da46f59c23d7512
2014-09-02 19:34:25 -04:00
Abel Lopez
dd3ad7e014 Changed serial console setup
The previous method only configured serial console for Ubuntu or
Fedora, I noticed this wasn't working when I built RedHat 7.
After reading about how grub-mkconfig works, I realized that we
shouldn't be using sed to modify grub.cfg, and set up the proper
variables in /etc/default/grub

Change-Id: I02b73dc5074cb48a716849cb8a8a9b2f054bea31
2014-09-02 21:07:08 +00:00
Jenkins
7214ca1196 Merge "Extend/fix support for extlinux bootloader" 2014-09-02 21:06:00 +00:00
Thomas Bechtold
e57fe4a977 Map dkms package for SUSE in base element
The dkms package is not needed on SUSE.

Change-Id: Id8c0465f3d847efe9c1266c7c280c019d404fe04
2014-09-02 18:17:45 +02:00
Dirk Mueller
3bff0ab543 Fix capitalization of openSUSE
Change-Id: I10b9875633d29f4cc0b4e00c13bb5998a71bfc17
2014-09-02 17:55:28 +02:00
Dirk Mueller
d8e6b00130 Fix ramdisk pkg-map for openSUSE
The package is called dhcp-client on all (open)SUSE related
distributions.

Change-Id: Iefb1bb5d0710d05699c646b86914189cb55fa033
2014-09-02 11:20:08 +02:00
Jenkins
1f130f7e49 Merge "Cleanup apt cache after grub install" 2014-09-01 10:01:58 +00:00
Jenkins
7da217b8f1 Merge "Truncate instead of deleting log files" 2014-08-29 17:03:29 +00:00
Ben Nemec
417b511147 Truncate instead of deleting log files
For package installs we need to be able to fix up permissions in
/var/log at install time, and if we delete all of the files then
those permission fixes get lost.  To deal with that, we can just
empty the files instead of deleting them.

Change-Id: I70abb354e4dc726ec7a176694c7c6e9455d89d08
2014-08-28 19:18:30 -05:00
Jenkins
ceb9c431f0 Merge "Remove docs about deprecated pypi-mirror tool" 2014-08-28 22:47:33 +00:00
Jon-Paul Sullivan
ed833e628f Lock around extract-image downloads and extracts
This wil be needed for parallel image builds.

Change-Id: I6ba3a10b44fe37f8d8903ad06302a672c3e972c8
2014-08-28 17:42:25 +01:00
Jenkins
7d0e7fe5ad Merge "Centralize handling of /lost+found" 2014-08-28 16:33:12 +00:00
Robert Collins
027dcc2dbe Add support for flashing ILO BMC's from ramdisks
This is more than a bit hard to test, requiring as it does an actual
iLO BMC, so sadly I have no tests :(.

HP Hardware really wants to be running latest firmware, and its
commonly agreed that one place where it can be sanely applied is in
the deploy environment, so this is my long threatened implementation
of that.

Change-Id: I3117a250d0d0eb8ee33eb4b15c837485a5cbf476
2014-08-28 17:07:03 +01:00
Jenkins
06f7f40028 Merge "Make RHSM registration set -u safe" 2014-08-28 15:36:46 +00:00
James Polley
341c42ba83 Remove docs about deprecated pypi-mirror tool
The pypi-mirror tool is no longer supported by the infra team, or
reccomended for installs, and is known to have various bugs that will
not be fixed.

Remove mentions of it from the docs and point people at devpi-server
instead.

Change-Id: Ieedb8bf9bee2f5932edac6b9747e645d0f4eb08c
2014-08-28 11:33:52 +10:00
Ben Nemec
644598941c Use dib-run-parts from dib-utils
Now that dib-run-parts has been moved to the dib-utils project, we
need to update diskimage-builder to use it instead of the version
directly in diskimage-builder.

This change removes the old copy of the dib-run-parts script in
the element, adds dib-utils as a dependency of diskimage-builder,
and updates the uses of dib-run-parts to correctly handle the fact
that it is now external to the project.

Requires I0be1f876d0e4a7d38e0d5c6010a552a8ebb158a4

Change-Id: Ia0a0df7784a14c49b5c47ac0b03e6c2602c84b3b
2014-08-26 16:44:18 -05:00
Jenkins
c393ed3902 Merge "Revert "Remove the temporary deploy element after rename"" 2014-08-26 20:29:55 +00:00
Jenkins
597acccbd7 Merge "map-services: add openstack-nova-novncproxy" 2014-08-26 08:28:13 +00:00
Jenkins
c445a6c66a Merge "Support for UBoot." 2014-08-25 08:09:02 +00:00
Ben Nemec
5c69b7a683 Make RHSM registration set -u safe
There are a bunch of optional env vars references in the rhsm
registration script, but they weren't wrapped to be set -u safe so
they cause build failures if any aren't set.

Change-Id: I18326b618f0920efa4dbd805611c6ee66d33ebb0
2014-08-21 21:39:38 -05:00
Jenkins
45693f5762 Merge "Add service mappings for ironic" 2014-08-21 07:31:17 +00:00
James Slagle
0c92466f5b Add service mappings for ironic
Adds the needed service mappings for ironic-api and ironic-conductor.

Change-Id: I8b218f926561d8fba1644fe883bf8aae4bb6473f
2014-08-20 17:31:45 -04:00
lifeless
4ebee67650 Revert "Remove the temporary deploy element after rename"
This reverts commit ac73660c40.

This was a backwards incompatible change - it broke the stable tripleo-heat-templates job.

Change-Id: Ideffca203db7b860b9f9d96c1d9f73e046519ec7
2014-08-20 19:37:13 +00:00
Jenkins
b404682d1b Merge "Increase source-repositories support for tarballs" 2014-08-20 11:19:59 +00:00
Jonathan Brownell
6e4e2cfeca Increase source-repositories support for tarballs
The existing support for tar files in source-repositories saves
only the contents within subdirectories contained in the archive.

This change makes the selection of contents within the tarball user-
definable based on the REPOREF specified (or overridden in the env).

A REPOREF of '*' (the current default, but deprecated) will select
the contents of subdirectories within the archive, while '.' will
select the entire contents of the archive.

For reference:
http://lists.openstack.org/pipermail/openstack-dev/2014-August/043197.html
http://lists.openstack.org/pipermail/openstack-dev/2014-August/043249.html

Change-Id: I45db42ce66bf1d63d6ab5730090bf458b1b37ce9
2014-08-19 13:24:02 -07:00
Jenkins
8d165837c3 Merge "Allow to specify an empty package list in pkg_map" 2014-08-19 20:05:45 +00:00
Derek Higgins
bd83c2e699 map-services: add openstack-nova-novncproxy
Add a mapping from nova-novncproxy to openstack-nova-novncproxy, it was missing.

Change-Id: Ia163a967925087b9d20f7b8581f0f636e53328d8
2014-08-18 13:15:10 +01:00
Ghe Rivero
fecbb03e54 Allow to specify an empty package list in pkg_map
An empty package list can be provided in a pkg_map file for those
cases were there is no need to install the package.

Change-Id: I0df8c6848ec4ac9e0f90790f14e8eb5399650339
2014-08-18 06:24:55 +00:00
James Slagle
3d0e111479 Use package-installs in fedora
Updates the fedora element to use the declarative package install
support provided by the package-installs element.

Change-Id: Ib3eb1936aa5c4efc350828e004794b0a8bd6c6c4
2014-08-14 21:18:19 -04:00
James Slagle
17f8bda93f package-installs for pre-install.d/post-install.d
Packages are often also installed in both the pre-install.d and
post-install.d phases. This patch expands the package-installs element
to support declarative package support for these phases in addition to
the existing support for install.d. The actual install/uninstall logic
is moved to common scripts under bin/ so that it can be reused across
the different phases.

Change-Id: Id51d0bbad232737fc8b5ffaf016dec50cd5b66c9
2014-08-14 21:18:19 -04:00
Dan Prince
389669f687 Formatting fixes for dhcp-all-interfaces
Fixup some whitespace formatting issues in dhcp-all-interfaces.sh

Change-Id: Iac515f53e76e68356826cbcefb64142b2009f351
2014-08-14 16:32:50 -04:00
Dan Prince
4fac7ff925 Remove disable_interface from dhcp-all-interfaces
I don't actually think there is a way that disable_interface
would get called on Fedora... because if config_exists
it takes a different path. Furthermore we really don't
want to mess w/ configured interfaces in dhcp-all-interfaces
because that could be side effecty (and is arguably the
job os os-net-config anyways).

This function already did nothing on Debian so removing
shouldn't hurt anything.

Change-Id: Ic53bbab98aa03d487efd290ae8e076045781b5b1
2014-08-14 16:23:44 -04:00
Jenkins
4d4b104e91 Merge "Don't try to install if packages is empty" 2014-08-13 23:36:13 +00:00
Jenkins
8c72451345 Merge "Relabel filesystem if SELinux is available" 2014-08-12 16:30:34 +00:00
Jenkins
77fae1affc Merge "Tidy base image by removing /var/log files" 2014-08-11 17:59:05 +00:00
Victor Lowther
e92398a318 Relabel filesystem if SELinux is available
Relabel the filesystem during image builds if SELinux is supported
in the kernel of the build machine and userspace tools are available.

Otherwise touch /.autorelabel to schedule a relabel the first time
the image boots. We relabel when possible because it decreases first
boot time.

Change-Id: I0bec885d6e5d4f4e1106f3bd2a90ba5f86395b07
Partial-Bug: 1347845
2014-08-04 17:56:33 -07:00
Pino Toscano
9043871c6e debian: exclude contents of /tmp from debootstrap tarballs
Skip the contents of /tmp/ when creating tarballs with the results of
debootstrap, as that directory contains just ccache cache (and in
general temporary files).

Small (positive) side effect is that deboostrap tarballs gets slightly
smaller, something like 2-3 MB less.

Change-Id: Ib4adf3dc908f02deed408bf9188f032767eeb2bb
2014-08-04 15:48:12 +02:00
Pino Toscano
267e96f52c Centralize handling of /lost+found
Just remove globally /lost+found after the root and install phases, so
distribution elements don't need to do that by themselves.

Change-Id: Ic783e613bd8794aefd3f40c9a7c308d14cd04b8d
2014-08-04 11:48:21 +02:00
Jenkins
f303b6546c Merge "Remove package mapping for mariadb-rdo-galera-server" 2014-07-31 04:03:01 +00:00
Endre Karlson
ccce52fd21 Cleanup apt cache after grub install
Grub installation happens in finalize.d so cleanup should be done after this.

This reapplies I6322b8b529e31fd1b7251dd9e07cc85f442f7e81 after the revert,
but working.

Change-Id: Ie1873e64d407552da37a8dbedc13c3adbf79c085
2014-07-30 15:04:12 +01:00
Juerg Haefliger
340f38d014 Extend/fix support for extlinux bootloader
Some extlinux packages install mbr.bin under /usr/lib/extlinux or
/usr/lib/EXTLINUX so tese directories need to be checked as well.
Also, there are packages that create a /boot/extlinux directory
which needs to be taken into account. Finally, commit
bccffc8bfd dropped the /boot prefix
from the kernel and initrd references in the ext/syslinux config
file which is simply a bug that needs fixing.

Change-Id: Idb071c9b18ff557b0f0f76d0d754536f2deca904
Closes-Bug: 1335042
2014-07-30 09:49:34 +02:00
Jenkins
280513194e Merge "Avoid to install a blank list of packages" 2014-07-30 01:59:45 +00:00
Jenkins
e6c9ba3d1b Merge "Add a ramdisk-install.d hook path." 2014-07-30 01:31:29 +00:00
Jenkins
e818db25dc Merge "Adds RHEL common element" 2014-07-29 08:08:02 +00:00
Jenkins
32aa9da5c6 Merge "Fix sourcing of environment files" 2014-07-28 13:46:32 +00:00
Jan Provaznik
517c858275 Remove package mapping for mariadb-rdo-galera-server
This mapping is not needed anymore because mariadb-rpm element
(renamed from mariadb-rdo) uses pkg-map.

Relies on: I90e7ca4803516b86179b0f6639e307a60297960b
Change-Id: I19fb44ee6008c6c57dd331a780eec3234d46b15c
2014-07-28 10:36:24 +02:00
Jenkins
7485f4a080 Merge "Add virtual media boot support in ramdisk element." 2014-07-28 05:50:39 +00:00
Jenkins
673f6aae9f Merge "Initial centos7 support" 2014-07-25 19:54:53 +00:00
Jenkins
42bb303005 Merge "Handle non-script grub2-install" 2014-07-25 19:52:33 +00:00
Ryan Brady
7f78be9318 Adds RHEL common element
This element contains the installation steps common to RHEL
operating system versions.

Change-Id: I37cce0bd5b5d31525f1335ecd416be570d36ad61
2014-07-25 14:17:03 -04:00
Jenkins
72409a5fe9 Merge "Update RHEL7 element since release" 2014-07-25 16:41:22 +00:00
Jenkins
691e891c9c Merge "Revert "Cleanup apt cache after grub install"" 2014-07-25 15:57:06 +00:00
Om Kumar
58b5607004 Support for UBoot.
UBoot needs the kernel and ramdisk to be post processed
using mkimage utility. This element helps user do it as
part of the DIB process.

This element needs u-boot-tools to be installed.

The load address and entry point for UBoot kernel can
be specified as shown in the example below.

Example:
    export UBOOT_KERNEL_ADDR=0x80000
    export UBOOT_KERNEL_EP=0x80000

Change-Id: I0e1039bf57d1ef11a3f9831d82e24b1ec2136c13
2014-07-25 11:14:37 +05:30
Jenkins
d44fcdad95 Merge "Lock around source repositories setup" 2014-07-24 22:02:53 +00:00
Steve Kowalik
eac2d7f588 Revert "Cleanup apt cache after grub install"
This calls chroot as a non-root user, which will always fail with
EPERM.

This reverts commit ab2d1a31f2.

Change-Id: Ie674fef694ad66e1ebc22083dec2a0bc34371e7b
2014-07-23 21:56:40 -04:00
Alexis Lee
2f6fdb4142 Fix sourcing of environment files
Includes "Don't match editor backup files in environment" and "Order
execution of environment.d scripts". These had to be squashed because
the first change alone changes the order env.d scripts are sourced in.

The current glob match for environment files can source editor backup
files (foo.bash~) which will override the real changes you have made.
Other parts use the regex to avoid matching such files, so do
the same for environment file matching.  Note this has to match "."
unlike the other regex, as most env files are "foo.bash"

Also sort the files before sourcing them to ensure a reliable order.
This change should be in line with current expectations, given the
convention is to numerically prefix scripts in this directory.

Update of: I934486b3ff5884063d29c6d9b66fd9b11140464c
Subsumes: Icc509f695d7a15a8026d8c7e463f06acf65499d7
Change-Id: Ibfb562c5970b40598fc95da1e8d4beb9d51d7612
2014-07-23 17:18:28 +01:00
Ramakrishnan G
5e18777691 Add virtual media boot support in ramdisk element.
This change adds support in ramdisk element for recognising
boot from virtual media cdrom and read the parameters from
the configuration file in virtual media floppy.
Change-Id: I28ca888b4ead4905f7141b38ed4b9a3614d00d26
Closes-Bug: #1321564
2014-07-23 10:00:10 +05:30
Jenkins
22cf8e824b Merge "Explicitly name element enable-serial-console" 2014-07-22 22:23:35 +00:00
Jenkins
b44b798101 Merge "Lock around ubuntu tarball download" 2014-07-22 19:42:27 +00:00
Endre Karlson
ab2d1a31f2 Cleanup apt cache after grub install
Grub installation happens in finalize.d so cleanup should be done after this.

Change-Id: I6322b8b529e31fd1b7251dd9e07cc85f442f7e81
2014-07-22 09:28:06 +00:00
Ryan Brady
646da5b464 Update RHEL7 element since release
This patch updates the RHEL7 element to work
properly now that it has changed from beta to
release status.  It includes a repo update to
locate os-config rpms not in epel yet and an
update to the latest cloud image name.

Change-Id: I05607d62193c8f8fb1c1bece9872d456b19c5a08
2014-07-21 16:20:18 -04:00
Jenkins
d9d43c6c03 Merge "Update RHEL 6.5 Image name." 2014-07-21 13:47:17 +00:00
Jenkins
fc5e791a6b Merge "Add dhcp support for ramdisk element." 2014-07-18 12:23:45 +00:00
Jon-Paul Sullivan
94535ebb21 Lock around ubuntu tarball download
Make this safe for parallel builds

Change-Id: I578fa38e1d44bc205b70d087d161b471cd260fa1
2014-07-18 09:52:39 +01:00
Jenkins
a030387d6b Merge "Echo output when pkg-map fails" 2014-07-18 02:43:12 +00:00
Ghe Rivero
226e92fa4d Avoid to install a blank list of packages
There are some situations where the list of packages to install
in a image is blank (packages blacklisted).

Change-Id: I315f97fe619c25b36cf47ed1b7a65936b753312e
2014-07-17 20:34:20 +00:00
Jon-Paul Sullivan
94a0947a60 Don't try to install if packages is empty
After running through pkg-map we could have no packages to install,
if so, don't attempt to run a malformed command.

Change-Id: Ia8e0aed62bcf814bf85c86b54ff0837da49ae7dd
2014-07-16 11:17:40 +01:00
Robert Collins
4319730357 Add a ramdisk-install.d hook path.
We need to be able to do install.d like things for ramdisks
themselves, but install.d runs outside the ramdisk context - and its
likely to break peoples brains if we mangle the two together - so this
adds a new hook point, ramdisk-install, specifically for installing
things into the ramdisk.

Change-Id: I37d1660309cda6e28bd0b316b08f61db4e080613
2014-07-16 19:23:50 +12:00
Ian Wienand
5abb4a4f12 Initial centos7 support
Initial support for a centos7 image.

This is separate to rhel7 because the major differences are things
like repo and image locations, which are always going to be different.
We should merge any real changes into the redhat-common layers.

Apart from the added support files in centos7/*, the other change is
mostly modifications to redhat-common's extract-image to handle
different partition layouts of the centos7 image.

Change-Id: I943abe5ff0a803f36eda266a79af0d9220edcae7
2014-07-16 10:43:05 +10:00
Om Kumar
aff9b29c4d Update RHEL 6.5 Image name.
The default value points to old image name
which results in 404 error. So updating the
value to point to correct name.

Change-Id: Ia66f0cf4f4167926892a0786467412d277ebffc3
2014-07-15 14:17:06 +00:00
Ramakrishnan G
099493b62c Add dhcp support for ramdisk element.
This change adds support for retrieving the IP
through DHCP if 'ip' was not provided as a kernel
command-line argument by pxelinux.  This is used
when the kernel/ramdisk is booted by virtual media.

Change-Id: I1097ce5b56ad40f2d6dc3181681d54f924ec7145
Closes-Bug: #1321563
2014-07-15 16:30:09 +05:30
Ian Wienand
13eaa40e5e Handle non-script grub2-install
grub2-install has been re-written [1] so isn't always a shell script,
so just exec it directly.  Also use $() as that follows the rest of
the file.

[1] http://git.savannah.gnu.org/cgit/grub.git/commit?id=cd46aa6cefabd7b0fcb15cd614577e1ab8c7a841

Change-Id: Ib4fd7c6854fb485c0da6a79babf4a3d45a8a0f2c
2014-07-15 10:52:52 +10:00
Jenkins
c9bcce2030 Merge "Optimizing directory creation." 2014-07-11 16:36:38 +00:00
Jenkins
a595c7c4c7 Merge "Avoid to install a blank list of packages" 2014-07-11 15:28:49 +00:00
Jenkins
4e97def3d5 Merge "Correction: if then statement code style." 2014-07-10 07:38:49 +00:00
Jon-Paul Sullivan
fbb797ea42 Lock around source repositories setup
Problems were seen when building images in parallel with
git clones and fetches interfering with each other.

Stop it.

Co-Authored-By: Michael Kerrin <michael.kerrin@hp.com>
Change-Id: Iaa48f259755d8b22327459a5c6d224a342909aee
2014-07-09 19:37:06 +01:00
Jenkins
f58e50ee64 Merge "Remove fixfiles from rpm-distro finalize" 2014-07-09 08:51:27 +00:00
Jenkins
15b25423cd Merge "Really handle Ubuntu mirror cache skew," 2014-07-09 04:00:22 +00:00
Robert Collins
eca284cab7 Really handle Ubuntu mirror cache skew,
The last patch to do this only handled the case of SHA256SUM being
stale, but the image is more likely to be stale in the default
configuration as the hash file is downloaded over HTTPS.

Change-Id: I6531fa684e560cad48c3696394d48a600680c875
Closes-Bug: #1336067
2014-07-09 08:07:58 +12:00
James Slagle
3df0a08394 Echo output when pkg-map fails
When the call to pkg-map fails, it prints the error to stdout. However,
this output is lost when pkg-map is executed in a subshell, so the
actual error is never seen. This change adds an explicit echo so the
error is shown.

Change-Id: Id6f4fae6acadfe2839b408fb2dd11fb65d65df6e
2014-07-08 14:12:51 -04:00
AzherKhan
a22305de05 Optimizing directory creation.
Optimized the creation of directories by using the Brace expansion mechanism.

And corrected the dirname resolution to follow correct convention.

Change-Id: I048c04385d2706c4e38f8548555a59b0437e1fcd
2014-07-08 09:39:58 -07:00
Dan Prince
bbf62fc5b9 Revert "Don't match editor backup files in env...
This reverts commit 32eda4b92f.

I noticed today that environment.d files no longer seem to get sourced
by DIB. Reverting this commit fixes it.

Change-Id: I08a8feab541901c8fd0a803628f5aeb6c0eec598
Closes-bug: #1339116
2014-07-08 11:13:11 -04:00
Jenkins
3cdfd9565e Merge "Rename rhel element yum blacklist" 2014-07-08 01:38:05 +00:00
Jenkins
8a077a0b13 Merge "Last ditch effort to correct a wrong shasum." 2014-07-07 23:49:21 +00:00
Jenkins
92d4ed73db Merge "Don't match editor backup files in environment" 2014-07-07 11:51:08 +00:00
AzherKhan
d06a1caafb Correction: if then statement code style.
Corrected the if; then statement to follow bash convention.

Change-Id: Ia3f80520bb9446c4feb5ce36dccffa2b1c1d72c4
2014-07-07 03:43:15 -07:00
Chris Jones
a65d74f171 Last ditch effort to correct a wrong shasum.
Some users on corp networks are finding that cloud images appear in
their proxies before SHA256SUMS files.
We now try to force any to upstream proxies to update their copy and
retry the checksum, rather than immediately failing.
We also now add proper argument parsing to cache-url, and associated
help functionality, since we have now grown our first argument.

Change-Id: I44d44b62db8d1df990606fdf087ec1b837f491f6
Closes-Bug: #1336067
2014-07-05 12:06:28 -07:00
Ghe Rivero
24e33592c2 Avoid to install a blank list of packages
There are some situations where the list of packages to install
in a image is blank (packages blacklisted) on opensuse based
distributions.

Change-Id: Ib720473fa63671ed974d41a86fe10a9b10720f5d
2014-07-04 07:39:08 +00:00
Ian Wienand
32eda4b92f Don't match editor backup files in environment
The current glob match for environment files can source editor backup
files (foo.bash~) which will override the real changes you have made.

Other parts matche use the regex to avoid matching such files, so do
the same for environment file matching.  Note this has to match "."
unlike the other regex, as most env files are "foo.bash"

Change-Id: I934486b3ff5884063d29c6d9b66fd9b11140464c
2014-07-03 13:54:18 +10:00
Richard Su
4e68a7965b Remove fixfiles from rpm-distro finalize
Running fixfiles after setfiles is redundant. setfiles
already corrected the SELinux file security contexts.

Change-Id: I48067f06968c5add48fa91a1496b9bf36944546c
Closes-Bug: #1316241
2014-07-03 11:47:10 +10:00
Jenkins
61c192ea3c Merge "Set LC_ALL=C into dib-run-parts env" 2014-07-03 01:22:34 +00:00
Jenkins
012c974430 Merge "Add RHEL7 to Red Hat family in pkg-map" 2014-07-03 00:50:35 +00:00
Ghe Rivero
4fd84db547 Rename rhel element yum blacklist
Rename rhel 10-yum-blacklist to 10-rhel-blacklist to
avoid future name conflict with a yum element blacklist.

Change-Id: Ic2136a06e1ec8f19908ada978782733d5fa7cacc
2014-07-02 10:53:35 +00:00
Jenkins
451e571753 Merge "Create an element to allow using a dpkg manifest" 2014-07-01 20:39:16 +00:00
Jenkins
bd3b025e97 Merge "VM element: Enable serial console on Debian" 2014-07-01 20:24:37 +00:00
Jenkins
f7625ca6f6 Merge "update opensuse element" 2014-07-01 16:57:05 +00:00
Jenkins
57293fac89 Merge "Element to remove unwanted kernel/initrd." 2014-07-01 10:40:36 +00:00
Yolanda Robla
8ed645f046 Set LC_ALL=C into dib-run-parts env
Force LC_ALL to C to allow printf to work independently
from the locale. Gives invalid number error if run in non-english
locale.

Change-Id: Ia68853505485fefbf0890313456e7edb6097666b
Closes-Bug: #1335932
2014-07-01 11:49:48 +02:00
Jenkins
5ee5aa5c89 Merge "Remove uneeded code from pkg-map" 2014-06-29 13:08:32 +00:00
Jenkins
5151dc0a33 Merge "Always bind-mount pypi mirror if dir exists" 2014-06-28 00:14:13 +00:00
Jenkins
c307db119e Merge "Use $DISTRO_NAME instead of local lsb_release" 2014-06-27 23:45:42 +00:00
Chris Krelle
eed131c373 update opensuse element
This patch updates the openSuse element to work with the new
base image file format. It should be noted that the base image
file name has changed several times while updating this element,
and and may need to be set manually via the "BASE_IMAGE_NAME"
env var.

Change-Id: I4dac8bf9a4bf76a00d4a04cbf063fd245b11f3d6
2014-06-27 15:03:06 -07:00
Jenkins
a1aa1e92dc Merge "Update base element to make use of pkg-map" 2014-06-27 21:29:43 +00:00
Jenkins
75f6fd605f Merge "opensuse: support pkg-map in bin/install-packages" 2014-06-27 21:28:48 +00:00
Jenkins
850b7f16a9 Merge "dpkg: support pkg-map in bin/install-packages" 2014-06-27 21:25:59 +00:00
Jenkins
f866a6f240 Merge "introduce $SYSTEMD_SYSTEM_UNIT_DIR" 2014-06-27 08:38:37 +00:00
Jenkins
ec200af0f8 Merge "Use common element select-boot-kernel-initrd" 2014-06-27 07:00:14 +00:00
Jenkins
14741f3efc Merge "Add select-boot-kernel-initrd element" 2014-06-26 23:27:45 +00:00
Ryan Brady
4ca059f9d9 Use $DISTRO_NAME instead of local lsb_release
The $DISTRO_NAME var is now set in all of the OS elements
in environment.d.  This patch removes the call to
lsb_release and instead uses the $DISTRO_NAME variable.

Change-Id: I7088eb88b6a3611fef5a21a6b62975876549465f
2014-06-26 16:47:21 -04:00
Ryan Brady
4226bd66ba Add RHEL7 to Red Hat family in pkg-map
The current "redhat" os family omits RHEL7. The rhel7 element has the
corresponding $DISTRO_NAME set in environment.d.  This patch adds rhel7
so a pkg-map json file using "redhat" for the os family will work the
same as rhel, centos and fedora.

Change-Id: If6b4cbba618167669a499e6121f234942ca9367a
2014-06-26 16:29:59 -04:00
Ghe Rivero
156d3495e0 Use common element select-boot-kernel-initrd
Several elements were using almost similar code to
get the current kernel and initrd inside an image.
After factoring out this code into a common element,
modify the elements to make use of it.

Change-Id: Ib547cd96b2b354b1c97342cacf5f0d0a3f70a3da
2014-06-26 10:19:09 +00:00
Steve Baker
e822a0ff5a Always bind-mount pypi mirror if dir exists
It may be desirable to specify file://tmp/pypi as one of the list
of mirrors specified by PYPI_MIRROR_URL, so the local pypi mirror
should be mounted as long as the directory exists.

Even though the path is mounted, it is only used by pip if there is
no specified PYPI_MIRROR_URL, or if one PYPI_MIRROR_URL entry
specifies file://tmp/pypi.

The pypi element previously assumed that a local mirror would be
a complete offline mirror. This change allows for a mixture of
local mirror plus remote mirrors. This means that the local
mirror could (for example) contain a few locally built pip packages
from arbitrary git checkouts for gate testing of those projects.
In this scenario, most packages would be downloaded from the
configured remote mirrors.

This change is required for synchronous gating on the os-*-config
tools.

Change-Id: I0f6d153c06993ffd0d90bd8ca1717462e8b6b541
2014-06-26 14:49:06 +12:00
Ghe Rivero
4c977082c4 Add select-boot-kernel-initrd element
Encapsulate the new select-boot-kernel-initrd common function
into an element so other elements can make use of it.

Change-Id: Ie587bd5dae1afebfd5646190dfa951a7949938cd
2014-06-25 21:23:19 +00:00
Kiall Mac Innes
572b8b2b13 VM element: Enable serial console on Debian
Debian based distros do not enable the serial console by default
in grub.cfg, instead sending all boot messages to the display

Change-Id: I0e80859198e77862ab0301ab27ebe11702fba09d
2014-06-25 14:13:49 +00:00
Chris Krelle
6077bd8b06 Ensure cache directory exists before use
Ensure cache mount directory exists before its used. The zypper element
would error when attempting to mount the cache directory because the
mount point did not exist. This patch corrects that.

Change-Id: Iaa3c2a0254b12bd847643a61a99f5a234097fd21
2014-06-24 16:35:59 -07:00
Chris Krelle
336e286637 update zypper element
Scripts in pre-install.d run as root so there is no need to sudo commands.
This patch also helps avoid an error when sudo is not installed in the
default image.

Change-Id: Iab20edfb161189f8c4e0cfa7d6946f63163d6413
2014-06-24 16:20:55 -07:00
Jenkins
8422f46dc0 Merge "Drop ending slash from DIB_CLOUD_IMAGES" 2014-06-23 19:22:30 +00:00
Jenkins
83e35dfa49 Merge "map-services: add openvswitch" 2014-06-23 16:00:54 +00:00
Steve Kowalik
cf9870729d Drop ending slash from DIB_CLOUD_IMAGES
The DIB_CLOUD_IMAGES variable in ubuntu's root.d is already used
everywhere with a /, so we can safely drop it from the default
value.

Change-Id: I39dbd3f4004052fcccb4131dc838759f4c82312a
2014-06-23 16:22:51 +10:00
Dan Prince
4015097a31 Update base element to make use of pkg-map
With this change package name to distro mappings used
within the base element are now stored in the top
level pkg-map file.

Additionally, all bin/install-packages commands are updated
to use the -m option so that the base element pkg-map
namespace is used by default.

Change-Id: I3bdbf987c8525f9ef67ca0c263651d6613f29ef0
2014-06-20 16:02:28 -04:00
Dan Prince
99ebd302b0 opensuse: support pkg-map in bin/install-packages
Updates the opensuse element's bin/install-packages script
so that if supports using pkg-map instead of map-packages
for package mapping. To make use of the new change
simply add the -m <element name> to allow install-packages
to know which element namespace to use when installing packages.

Use of the new -m option is off by default.

As part of this change we also updated install-packages
to use getopt for in script argument parsing.

Change-Id: I56da8adf6c445c50c7642e221c6c11ba59330404
2014-06-20 16:02:28 -04:00
Dan Prince
9340ea309b dpkg: support pkg-map in bin/install-packages
Updates the dpkg element's bin/install-packages script
so that if supports using pkg-map for package mapping.
To make use of the new change simply add the -m <element name>
to allow install-packages to know which element namespace to use
when installing packages.

Use of the new -m option is off by default.

As part of this change we also updated install-packages
to use getopt for in script argument parsing.

Change-Id: Idfc40f2d75828a0f09d227f0332ccef8f0183efc
2014-06-20 16:02:28 -04:00
Om Kumar
42f8ef86a1 Element to remove unwanted kernel/initrd.
Remove unused kernel/initrd from the image.

Change-Id: If187ab9ff42b99f127897de761fd9685b37304fc
2014-06-20 20:57:33 +05:30
Jenkins
6744030be4 Merge "debian: fix network on Wheezy" 2014-06-20 08:55:31 +00:00
Jenkins
45d5bf45e9 Merge "Do not use DatasourceNone for precise cloud-init" 2014-06-19 23:31:54 +00:00
Jenkins
da9dfc854a Merge "dpkg: local cache for .deb files" 2014-06-19 19:46:46 +00:00
Gonéri Le Bouder
4bf38a829a introduce $SYSTEMD_SYSTEM_UNIT_DIR
export $SYSTEMD_SYSTEM_UNIT_DIR to know where .service files should be
moved.
So far those files are moved directly in /usr/lib/systemd/system whereas
Debian use /lib/systemd/system.

We load the value from the systemd pkg-config file.

The patch don't use pkg-config directly for three reasons:
 - environment.d files are run before install.d and the other similar
 targets, making hard to install an additional dependency
 - the .pc file can easily be parsed using awk
 - pkg-config package name is not standard across the different
 supported distributions

$SYSTEMD_SYSTEM_UNIT_DIR default value is /lib because, as pointed by James
Slagle, /lib is actually a symlink to /usr/lib/ on Fedora and will work
for Debian and Fedora.

See: https://bugs.debian.org/719634

Change-Id: I413a92284e6a79e7fcdde99c5138bc3ce8e85e80
2014-06-17 19:34:29 +02:00
Jenkins
12ae39b97b Merge "Ensure dib-run-parts profiling works with py3 and py2" 2014-06-17 07:26:27 +00:00
Nikhil Manchanda
d550087214 Do not use DatasourceNone for precise cloud-init
The 'None' Datasource does not exist in precise 12.04 (which uses
cloud-init 0.6.3), so conditionally ensure that we are not including
this in the datasource_list in case we're building a precise image.

Partial-bug: 1329068
Change-Id: I6efb91e17bf896bbf7d67c31c4eb5098d5d61a71
2014-06-16 10:58:39 -07:00
Jenkins
1b71aafc08 Merge "Add missing dollar" 2014-06-16 13:49:04 +00:00
Cian O'Driscoll
7e58968779 Explicitly name element enable-serial-console
We use an element remove-serial-console in image elements.
Renaming this element to match. Also renaming allows us to use
the serial console in devtest runs.
See - https://review.openstack.org/#/c/86272/

Change-Id: I8f1138f58d0046ccdeefa3decc45c36d95c6bf4b
2014-06-16 09:23:07 -04:00
Jenkins
c0501291c9 Merge "Refactor code to select boot kernel" 2014-06-14 04:01:45 +00:00
Jenkins
faa5ce8930 Merge "Name deploy-ironic and deploy-baremetal files uniq" 2014-06-13 19:40:02 +00:00
Jenkins
8d7e6dbec5 Merge "Fix package removal" 2014-06-13 17:17:04 +00:00
Coleman Corrigan
667248a970 Tidy base image by removing /var/log files
diskimage-builder/elements/base/cleanup.d/99-tidy-logs added
to tidy up (remove) initial logs files on disk images.

Makes for slightly smaller images, and easier times tracing post
boot issues.

Change-Id: Iee98b5d818e080bcd9635482bc29349a71938087
2014-06-13 15:23:13 +01:00
Gonéri Le Bouder
b822581d88 debian: fix network on Wheezy
source-directory ifupdown directive has been introduced in version
0.7.44. Wheezy comes with version 0.7.8.

Without this change, the eth0 interface will never get an IP address on
Wheezy.

Change-Id: If40ed3b82ca78280190b66c06a0a77bb71270d7d
2014-06-13 13:11:24 +02:00
Jenkins
3ee469f299 Merge "Name 10-rhel-cloud-image uniquely" 2014-06-13 09:50:42 +00:00
Jenkins
9e968c98ca Merge "Name 01-install-bin uniquely" 2014-06-13 09:50:36 +00:00
Jenkins
b541f0731c Merge "Delete redhat-common 00-usr-local-bin-secure-path" 2014-06-13 09:47:10 +00:00
Om Kumar
bccffc8bfd Refactor code to select boot kernel
The script to find best kernel from image (used by vm
and baremetal element) is duplicated and is not in sync
with each other. Moving the code to img-functions as a
function will reduce duplicated code and make it reusable
in future.

Since img-functions is not accessible in chroot env,
kernel selection is being moved from finalise.d to
cleanup.d in the vm element.

Change-Id: I8fbccc13a2c61a5191ef9ea5d2a8302a3e43b000
2014-06-13 11:46:39 +05:30
Jenkins
20ea513854 Merge "Yum: support pkg-map in bin/install-packages" 2014-06-12 23:40:12 +00:00
Jenkins
662782b66b Merge "Correct source-repository comments" 2014-06-12 20:56:35 +00:00
Jenkins
adefb7337a Merge "Name 99-setup-first-boot uniquely" 2014-06-12 18:31:40 +00:00
Jenkins
7489540b22 Merge "Correct the wrong rename in rhel element" 2014-06-12 13:24:55 +00:00
Jenkins
2a1c189069 Merge "Debian: Support additional debootstrap arguments" 2014-06-12 05:31:35 +00:00
Jenkins
f1437fe241 Merge "Tidy up SuSE kernel selection" 2014-06-12 04:52:16 +00:00
Jenkins
de8073d4d3 Merge "Parameterise PXE kernel and initrd selection" 2014-06-12 04:52:10 +00:00
Dan Prince
b146f94c9a Name deploy-ironic and deploy-baremetal files uniq
Updates the 51-install-tgt and 52-install-busybox files
in both the deploy-ironic and deploy-baremetal elements
so that they have unique names.

While an end user would never use these elements at the same time we
should always use unique names for things per DIB conventions.

Change-Id: I8926870e45849341c10b2d479ddee45f9b625e2b
2014-06-11 08:16:41 -04:00
Dan Prince
c2df583132 Name 10-rhel-cloud-image uniquely
Updates the rhel7 version to be named 10-rhel7-cloud-image

Change-Id: I8d3eec989ee60f48c4895423f23e0ddff5a9cb14
2014-06-11 08:16:41 -04:00
Dan Prince
3145e4a970 Name 01-install-bin uniquely
While an end user would never use these elements at the same time we
should always use unique names for things per DIB conventions.

Change-Id: I53b10522183e8566c62922b91878e5cf767fe2c6
2014-06-11 08:16:41 -04:00
Dan Prince
93d8d022d7 Delete redhat-common 00-usr-local-bin-secure-path
Both the rpm-distro and redhat-common elements had the
same exact 00-usr-local-bin-secure-path. This commit
deletes the one in redhat-common and leaves the one
in rpm-distro (which is also required by the same
elements).

Change-Id: Ie9ef29d268b3737a6741fce016ef335db6651b59
2014-06-11 08:16:41 -04:00
Dan Prince
50374fe4e7 Name 99-setup-first-boot uniquely
While an end user would never use both redhat-common and ubuntu
elements at the same time we should always use unique names
for things per DIB conventions.

Change-Id: I2cabc583a1fd41d4fe7fbf8cf93127d49e26aff4
2014-06-11 08:16:41 -04:00
Dan Prince
5254aedfe5 Yum: support pkg-map in bin/install-packages
Updates the yum element's bin/install-packages script
so that if supports using pkg-map instead of map-packages
for package mapping. To make use of the new change
simply add the -m <element name> to allow install-packages
to know which element namespace to use when installing packages.

Use of the new -m option is off by default.

As part of this change we also updated install-packages
to use getopt for in script argument parsing.

Change-Id: I19240e74b69859f7faf9cd80440633234a677d63
2014-06-11 08:09:27 -04:00
Robert Collins
d5e0c2d778 Correct source-repository comments
git revisions are not references.

Change-Id: If450e70e46cd30a2662b7246a2555c118ed33ad2
2014-06-11 19:32:27 +12:00
Jenkins
6ff1681915 Merge "Reinstate Trusty as default for Ubuntu" 2014-06-11 04:28:56 +00:00
Jenkins
336398d57a Merge "Only use Ec2 cloud-init data source for Ubuntu" 2014-06-10 23:06:27 +00:00
Gonéri Le Bouder
eca59b2e97 dpkg: local cache for .deb files
With this patch, /var/cache/apt/archives directory content is preserved.
The directory is actually a bind mount of the
~/.cache/image-create/apt/$DISTRO_NAME directory, much like what we do
for ccache.
You can use DIB_APT_LOCAL_CACHE=0 to disable this behavior.

This trivial change improve performance A LOT (>30%), even if a local
HTTP proxy because:
 - there is no need to copy again and again the same files
 - we avoid network latency

The patch has been tested with Debian and Ubuntu with every elements
from the tripleo-image-elements repository, the final size of the cache
directory is about 700MB per distribution subdirectory.

Change-Id: I4fab499493f734c7c546d4d23b1a98f0e7523a39
2014-06-10 17:20:37 +02:00
Jenkins
343f70cd23 Merge "Add new cloud-init-datasources element" 2014-06-10 06:43:47 +00:00
Gonéri Le Bouder
1de9612dad map-services: add openvswitch
OVS service is called:
 - openvswitch-switch on Debian/Ubuntu
 - openvswitch on RHEL/Fedora

This changes is needed to get the neutron element to work
on Debian/Ubuntu with systemd:
  I6e3df30dc3a6918f3a949a7dac47289ede5c3d1f

Change-Id: I2748a9db7f8ee456698187a9646bc7cda2256fe7
2014-06-09 09:18:02 +00:00
Giulio Fidente
3262e76248 Ensure dib-run-parts profiling works with py3 and py2
This change adjusts the "inline" python function used to profile
run-parts duration to make it run successfully on both py2 and py3

Change-Id: Ib5287a56b8e2b3e882dca489192f01d9f01cd8ba
2014-06-06 18:37:39 +02:00
Tomas Hancock
6b8a0408d8 Add missing dollar
Check is missing a dollar leading to an error on deploy ramdisk
consoles when booting .

Change-Id: Id61af0f7abbc353e0750580b80e448820cc17b10
2014-06-04 14:17:22 +01:00
Adam Gandelman
40914bd121 Reinstate Trusty as default for Ubuntu
Now that issues associated with this bump have been addressed
elsewhere, reinstate trusty as the default DIB_RELEASE for
the ubuntu element.  Also adds a note about memory requirements
to the ramdisk element.

Change-Id: Ia89c83ba0ee4ad3bd89b833ba0db08135eac63ed
2014-06-03 16:28:25 -07:00
Adam Gandelman
f61c1acf81 Only use Ec2 cloud-init data source for Ubuntu
Default to only having cloud-init query Ec2 on first boot for Ubuntu,
until cloud-init has been SRU'd to fix the CloudSigma data source issue
that causes Trusty boots to hang.

Change-Id: Icb3734d5ae78f4a0a6c0fae1af4a2ce3c809308c
Partial-bug: #1316475
2014-06-03 16:27:41 -07:00
Ryan Brady
dcba738541 Remove uneeded code from pkg-map
This patch removes a couple of lines of redundant code.  In Python,
the get method on a dictionary defaults the return value to None,
so the method never raises a key error.

https://docs.python.org/2/library/stdtypes.html#typesmapping

Change-Id: I180877f9be23d373cd63771bb3682d6587ec6fac
2014-06-03 11:37:15 -04:00
tanlin
ebd54b1010 Correct the wrong rename in rhel element
There is a substitution of suffix in rhel element, like
rhel-guest-image-6.5-20140121.0.x86_64.qcow2 to
rhel-guest-image-6.5-20140121.0.x86_64.raw

The convert is correct but the substitution of the suffix is wrong.

${PARAMETER#PATTERN} will match the string from the beginning and generate rhel-guest-image-6.5-20140121.0.x86_64.qcow2.raw

We should use ${PARAMETER%PATTERN} instead, which is from the end.

Change-Id: Ic45a3f800058ea9e5d746ff543c068a54632407d
2014-06-03 06:42:35 +00:00
Adam Gandelman
f645287ec4 Add new cloud-init-datasources element
This moves cloud-init data source configuration to a general purpose
cloud-init-datasources element that can be used to explicitly configure
the list of cloud-init sources that will be queried on first boot.

cloud-init-nocloud now depends on this new element to configure the
datasource_list while continuing to prep the image for a nocloud first boot.

Change-Id: Ibcc3b86d6ca567a23f89b7a1a36bc713e444ef68
Closes-bug: #1316475
2014-05-29 21:54:59 -07:00
Dan Prince
50cb019a25 Set DISTRO_NAME in OS environment.d
Updates the various operating system elements so that we
set the DISTRO_NAME variable for each distro.

This env var is used by bin/pkg-map to set the default
distribution name for package name mappings.

Change-Id: Ib4b05eb7191dd50d25799a0bac51fd15c01b74cb
2014-05-28 12:22:02 -04:00
Dan Prince
c55539b142 Add pkg-map element.
Adds a new pkg-map element which can be used to install
packages based on an in element 'pkg-map' JSON config
file format.

As part of this change we expose DIB's IMAGE_ELEMENT variable
so that we can have access to the element names which are being
installed in a clean manner.

This change is intended to decouple elements from DIB
and allow new elements to support multiple distributions
(with various package naming schemes) without having to
constantly maintain DIB's various bin/map-packages files.

Change-Id: Ic0a061995e2ae708c95a535cba6237bff58f7d93
2014-05-28 12:21:44 -04:00
Jenkins
ab8a0f4bdd Merge "add some missing \n at end of file" 2014-05-28 00:40:57 +00:00
Jenkins
e71e8efb51 Merge "Set -o pipefail new scripts" 2014-05-27 18:54:57 +00:00
Juerg Haefliger
562546ecb6 Fix package removal
Shift first to drop the first commandline argument (-e).

Change-Id: I73a0100484b6443ad5bdf500dd032af20eac7d3b
2014-05-27 07:50:42 +02:00
Jon-Paul Sullivan
ec234d356e Create an element to allow using a dpkg manifest
Enable reusing of a dpkg manifest to pin package versions at build
time

Change-Id: I23e83268d282c972b335a8cf0dfc45030a0ab4ff
2014-05-26 11:23:48 +01:00
Ben Nemec
79464cf00f Set -o pipefail new scripts
A few scripts were added without set -o pipefail since the original
change to add that everywhere.  This will get the dib-lint check
passing again.

Change-Id: I96bef45cc10ff9bbcf2c4f1b796b8cd188e10485
2014-05-23 15:57:27 -05:00
Jon-Paul Sullivan
76c59bfbde Add support for source-repos gerrit refs
Add the ability to supply a gerrit ref in DIB_REPOREF_* and
have that reference fetched from the remote and checked out for
building within the image.

Closes-Bug: #1314021
Change-Id: I5e5742c4cbff98e14121c50dde5e512f192b5415
Co-Authored-By: Robert Collins <rbtcollins@hp.com>
2014-05-23 16:00:22 +01:00
Jenkins
ce85042b9c Merge "Stop using bash arrays for whitelisting in yum" 2014-05-22 23:42:36 +00:00
Jenkins
0f6f533993 Merge "Export unbound variable DIB_RHSM_USER" 2014-05-22 17:02:19 +00:00
Jenkins
8737df75cf Merge "Remove the temporary deploy element after rename" 2014-05-22 10:48:50 +00:00
Steve Kowalik
e02dbd717b Stop using bash arrays for whitelisting in yum
Using -n for bash arrays was a little bit bad, and also resulted in
errors when running under set -u. Wean the yum element off the array
and switch to a straight string.

Change-Id: Ifd61fce7dadd5820022e9b2ab42020ba0cb71ce6
2014-05-22 19:05:28 +10:00
Jenkins
2afe36eca8 Merge "Add switch to turn on caching for debian element." 2014-05-22 09:05:17 +00:00
Jenkins
9941326be6 Merge "set -u doesn't permit bash arrays to test -n" 2014-05-21 20:21:11 +00:00
Jan Grant
b7aa04ed9d Tidy up SuSE kernel selection
This is just removing a C&P line that only ever returns a single result.

Change-Id: Iad37e982412a9199d86d9b4481cc14a85eef704e
2014-05-21 10:08:24 +01:00
Jan Grant
6df7039ba1 Parameterise PXE kernel and initrd selection
When using custom-built kernels, their filenames may not necessarily
follow the standard naming conventions for the stock systems in question.

This patch permits the specification of an alternative pattern to use
when looking for the correct kernel to construct the PXE images from.

Example:
  export DIB_BAREMETAL_KERNEL_PATTERN="kernel*jang"
  export DIB_BAREMETAL_INITRD_PATTERN="initrd*jang"

Change-Id: I86e5254051addd5e89dd4e363e9bba117e65353d
2014-05-21 10:08:23 +01:00
Jenkins
44922b8247 Merge "Ensure we can read the kernel out." 2014-05-20 03:18:42 +00:00
Jenkins
f70a9fa4e2 Merge "Add map-services for debian distros." 2014-05-19 21:23:00 +00:00
Jenkins
d69c0fce86 Merge "Fedora: Add support for configuring Yum mirrors" 2014-05-19 15:52:13 +00:00
Kiall Mac Innes
1757214196 Debian: Support additional debootstrap arguments
Add support for supplying a custom keyring and debootstrap script
using two new environment variables:

* DIB_DEBIAN_KEYRING - Path to a gpg keyring.
* DIB_DEBIAN_DEBOOTSTRAP_SCRIPT - Path to a alternative debootstrap
  script.

Change-Id: I42e8832b435b8a77b0681b5bc9d9332bbe860cd5
2014-05-18 19:40:38 +00:00
Dave Thomas
1a70916bca set -u doesn't permit bash arrays to test -n
Although set -u allows empty WHITELIST=() list to be set, you can't test
-n "$WHITELIST".  Looks like the intention of the commit that caused
this issue (Change-ID Iad2d490c86dceab148ea9ab08f457c49a5d5352e) is to
set a policy to prevent future use of bugs due to unbound variables.

Thanks StevenK for pairing with me and helping me hone my bash skills.

Change-Id: I2349c990ba53b0aaecd5119812582c7c848f1e4e
Closes-Bug: #1319987
2014-05-16 13:16:06 -07:00
Jenkins
a719b9fa5f Merge "4 spaces indent" 2014-05-14 08:26:08 +00:00
Robert Collins
072fee1a74 Ensure we can read the kernel out.
If the kernel is restricted read (as a security measure) we may not be
able to read it without chmodding it.

Change-Id: Ib577705c27d8c351288334de03a7848cf49f2de9
2014-05-14 04:25:05 +12:00
iberezovskiy
8995d6402a Export unbound variable DIB_RHSM_USER
If 'DIB_RHSM_USER' isn't set, the image build with rhel
element fails because of check 'set -u'. So, we should export
'DIB_RHSM_USER' in script to avoid error during build

Change-Id: Ic63a20e3d91a25bbe804440a5919b035489c60a3
Closes-bug: #1318717
2014-05-13 12:03:27 +04:00
Jenkins
dcb5f57a5a Merge "Move instead of copy the temporary git manifest" 2014-05-12 03:02:28 +00:00
Gonéri Le Bouder
ea3f4dd459 4 spaces indent
ensure 4 spaces indentation is used everywhere.

Change-Id: Ieb48faacb4c96b7b358771d70c17f2f22d0354f4
2014-05-10 11:17:29 +02:00
Trent Geerdes
771ed1903c Adding -x to set parameters for more output
When the cache tarball gets fail the output doesn't indicate
what url is failing.  Adding '-x' displays the urls attempted.

Change-Id: I71b2dbfd712e72c591410ca9b68edc76d493ad6e
2014-05-08 14:41:03 -06:00
Ben Nemec
16be6d7ce0 set -u and -o pipefail everywhere
As with the previous similar changes, this is intended to catch
problems as they happen instead of ignoring them and continuing on
to potentially fail later.  Setting this on all existing scripts
will allow us to enforce use via Jenkins.

Change-Id: Iad2d490c86dceab148ea9ab08f457c49a5d5352e
2014-05-06 15:51:07 -05:00
Dan Prince
131fb6e4db Fedora: Add support for configuring Yum mirrors
Updates the fedora element so that DIB_DISTRIBUTION_MIRROR can
be used to configure a custom Yum mirror on Fedora. This
implementation replaces the baseurl in each of the default
Fedora Yum .repo files so that a custom mirror URL/path
can be supplied. The URL should point to the directory
where the updates and release directories exist. Example:

  http://download.fedoraproject.org/pub/fedora/linux

As part of this change we also remove any existing
metalink settings in the .repo files to cover the case
where an end user runs their own createrepo command
to create the mirror... which means your metadata won't match that
of the normal Fedora mirrors and your baseurl won't get used.

Change-Id: I1a49a2941b1cc489e91a72b87430393cd32e865d
2014-05-06 10:11:26 -04:00
Gonéri Le Bouder
3ae4edc276 add some missing \n at end of file
This to avoid the “No newline at end of file” error from patch/git.

Change-Id: Ice82d4a36a0893e8f71dbffbdab154d1fbab8a09
2014-05-06 13:43:53 +02:00
Jenkins
dde3360057 Merge "Support declarative package installs/uninstalls" 2014-05-06 08:51:29 +00:00
Dan Prince
3a7800ec99 Add map-services for debian distros.
DIB's map-services is currently used in TIE's os-svc-config
binaries to support service name lookups. Many of the systemd
specific functions require bin/map-services to exist.

This change adds bin/map-services files that are noops
to both the debian and ubuntu elements so that as those
distros move towards using systemd they can make use
of the functions we already have in place.

Change-Id: If8d61f3858b11de86c3292e840d033e5e3cecedb
2014-05-05 16:08:40 -04:00
Ben Nemec
79ab95b16e Make sure all scripts are set -e
If scripts are not set -e then errors can be ignored, causing more
confusing failures later.

Also adds an exclusion comment to the ramdisk init script since we
don't want that to exit on failure.

Change-Id: Idf43993bd10b1ef16c1d3b0d9df8d0ad94c46458
2014-05-02 17:18:54 -05:00
Jenkins
928e6f237b Merge "debian: properly deal with Debian stable/unstable" 2014-05-02 15:37:26 +00:00
Jenkins
d99d8092a7 Merge "Add sysv support to elements/dhcp-all-interfaces" 2014-05-02 08:59:46 +00:00
James Slagle
7f9ebf2632 Support declarative package installs/uninstalls
Adds a new element, package-installs, that provides an interface for
declarative package installs and uninstalls. Packages to install can be
added to an install.d/package-installs-<element-name> file. The set of
packages listed across such files are installed in a single transaction
at the beginning of install.d.

Prefacing the package name with a "-" indicates that the package should
be uninstalled at the end of the install.d phase. Again, the full set of
uninstalls are done in a single transaction.

An element providing a package-installs file should add package-installs
to its element-deps file.

Change-Id: I5b540388eff1079c8dee933b869463371481152b
2014-05-01 08:47:14 -04:00
Gonéri Le Bouder
6da238485d debian: properly deal with Debian stable/unstable
This commit fixes the if/else block around Debian stable/unstable which
was badly merged during the last rebase.

Change-Id: Ia2eb806da3d7c8d28afffb026f35600c7125d8d7
2014-04-30 18:29:31 +00:00
Jenkins
b46ece4de9 Merge "debian: add systemd support" 2014-04-30 12:08:01 +00:00
Jenkins
6e1b64deb3 Merge "debian: support upstart on Wheezy" 2014-04-30 12:01:12 +00:00
Jenkins
71fafff215 Merge "Permit cache-url to work with fifos." 2014-04-30 08:30:44 +00:00
Jenkins
8bba3a2137 Merge "cleaning up 01-copy-manifests-dir" 2014-04-29 22:32:08 +00:00
Jenkins
301dc5cc0e Merge "set -e all the things" 2014-04-29 11:29:43 +00:00
Jenkins
1384f8aa16 Merge "Map openjdk-7-jre-headless to RHEL+SUSE" 2014-04-29 08:10:37 +00:00
Jenkins
59a2ccc6c3 Merge "Add package uninstall support" 2014-04-29 08:10:31 +00:00
Robert Collins
481f8de56a Permit cache-url to work with fifos.
This makes it easier to work with temporary files - less traps
scattered around higher layer code.

Change-Id: I2fdd93115a7b0d289c2e28f8c133d4059de75b87
2014-04-29 14:46:50 +12:00
Jenkins
8a566e67a6 Merge "Explicitly use bash" 2014-04-29 01:35:22 +00:00
Gonéri Le Bouder
9e60987e40 debian: add systemd support
This patch adds a debian-systemd element to get systemd support with
Debian.

A backport of systemd is used on Debian Wheezy.

Change-Id: I095c81b2c32844fccf940c97b79054923c25b8c2
2014-04-28 16:51:13 +02:00
Gonéri Le Bouder
155accb599 debian: support upstart on Wheezy
Upstart cannot be installed by debootstrap on Wheezy because of a
conflict with sysvinit. sysvinit is flagged a being “essential” and
apt-get will refuse to remove it.
See: https://bugs.debian.org/668001

This patch uses another strategy. We install a standard sysvinit
Debian chroot with debootstrap and during a second step, we
replace sysvinit with the wanted init system.

Change-Id: I081c81b2c32874fccf940c97b79054923c25b8c2
2014-04-28 16:51:13 +02:00
Jan Grant
90bbd1e98f Debian element should activate eth0
Cloud images expect the primary interface to run dhclient in order
to come up and acquire an IP address. Although we have technology
to manage without this for the undercloud and so on, the seed
still relies on this machinery in order to bring up its first
interface.

Change-Id: I079a679779de760ee9ca6f93cbb22e69a65e03af
2014-04-28 13:01:45 +01:00
Jenkins
b1e8c082c3 Merge "Add a mapping for kernel headers" 2014-04-26 00:38:12 +00:00
Ben Nemec
f6ba2aeaf4 set -e all the things
Using set -e in all of our scripts will prevent some subtle bugs
from slipping in, and will allow us to enforce use of set -e with
tooling.

This change also adds -u and set -o pipefail in the less complex
scripts where it is unlikely to cause problems.  A follow-up change
will enable those options in the complex scripts so that if it
breaks something it can be reverted easily.

Change-Id: I0ad358ccb98da7277a0ee2e9ce8fda98438675eb
2014-04-25 17:38:51 -05:00
Ben Nemec
0b367e919b Explicitly use bash
In the mailing list discussion, everyone seemed to agree that we
should just be explicit that these are bash scripts since that's
where we're testing them and there's no good way to ensure they
work on any other shells right now.

Change-Id: I86c41d2cddf6e91e7deeb4c2ab51635c80a1ce0c
2014-04-25 17:38:41 -05:00
Jenkins
db1ba39fed Merge "Remove All Rights Reserved" 2014-04-25 21:23:02 +00:00
Monty Taylor
1c07d4ba1a Remove All Rights Reserved
The phrase is no longer needed as of August 23, 2000 with Nicaragua's
joining of the Berne Convention.

Additionally, in at least one instance,
elements/cache-url/bin/cache-url, its existence in the file between
Copyright lines is just weird and feels misleading, even though it is
not.

Remove all of the lines, because sanity.

Change-Id: I24fd76c2b4f66b8036010b5079db39ead729abee
2014-04-25 11:25:10 -07:00
Jenkins
e1c28525fc Merge "Place /usr/lib64/ccache in PATH" 2014-04-25 17:17:50 +00:00
Jenkins
0ec786e5a1 Merge "Revert default Ubuntu release back to Saucy" 2014-04-25 02:01:12 +00:00
Jenkins
0303108857 Merge "indent: replace tab by 4 spaces" 2014-04-24 23:19:09 +00:00
James Slagle
6824181354 Add package uninstall support
Adds the ability to uninstall packages to our install-packages scripts.

Change-Id: I72e0252968a151d71176aa479c7233f6df0a10ac
2014-04-24 16:57:20 -04:00
Jenkins
23bb91dd5b Merge "Disable splashimage for legacy grub." 2014-04-24 20:20:33 +00:00
Adam Gandelman
927f80adaa Revert default Ubuntu release back to Saucy
Trusty ramdisks do not boot, failing to exec init.  This reverts
the default DIB_RELEASE back to Saucy until Trusty ramdisks have
been properly tested.

Change-Id: I5ac81822b78805275268cecc021a74a62c22ef58
2014-04-24 11:19:43 -07:00
Monty Taylor
ace82f16e9 Add a mapping for kernel headers
Nodepool needs to be able to install kernel headers, which means we
need a mapping for redhat systems here.

Change-Id: Ic83d642b5ebe85b042cf7b23b295be74bbe185fa
2014-04-24 10:44:40 -07:00
Alexis Lee
21f01e937c Map openjdk-7-jre-headless to RHEL+SUSE
Provide package name mappings for openjdk-7-jre-headless:
        RHEL/Fedora: java-1.7.0-openjdk-headless
        SUSE: java-1_7_0-openjdk-headless

See https://review.openstack.org/#/c/86314

Change-Id: I3a351d0f9854f6f320ba145a71ffc66fd02c231c
2014-04-24 07:49:12 +00:00
Alexis Lee
6b2a78f3ab Sort rhel/bin/map-packages
Unlike the redhat-common and opensuse map-packages maps, the RHEL one
appears to be unsorted. Fix that.

Change-Id: I75c5d2694525731660c2a3b5b58ee696c61923ba
2014-04-23 08:53:03 +01:00
Juerg Haefliger
e5f537ff2a Add sysv support to elements/dhcp-all-interfaces
This patch adds support for sysv init to the dhcp-all-interfaces element.
Specifically, it adds a sysv init script and the relevant install/config steps.

Tested with Debian jessie/testing.

Change-Id: I4ae1d5365bafda188cb65ea1fb72d8ffe2408910
2014-04-23 09:20:29 +02:00
Jenkins
621d63ce2a Merge "Make "trusty" (Ubuntu 14.04) the default release" 2014-04-23 02:46:55 +00:00
Derek Higgins
df78cd3dca Place /usr/lib64/ccache in PATH
ccache wasn't being used on fedora as the symlinks are in lib64.

Change-Id: I810558799dd2382b36b5112e435b74fd5caa8ccf
2014-04-23 01:18:29 +01:00
Steve Kowalik
0072a39b2b Small fixes for dhcp-all-interfaces
I3166951c6fbb2faec53ffc339edd2fd0c3512300 recently landed without two
fixes that I suggested, namely using rm -f rather than testing for
file existance, and a comment fix.

Change-Id: I68e187e20802f3482d4c524a6c435fec7526178e
2014-04-22 16:39:52 +10:00
Michael McCune
bcf529ce80 cleaning up 01-copy-manifests-dir
This change adds extra tests to the
elements/manifests/cleanup.d/01-copy-manifests-dir script to prevent it
from exiting with an error due to non-existant files.

The premature exit condition only seems to happen when an image is being
created without including the "base" element.

Changes
* adding file detection tests for dib_arguments and dib_environment
files

Closes-Bug: #1308224
Change-Id: Iaa8bafb6765241408b9f02d4896913d15f2f401d
2014-04-21 16:42:44 -04:00
Jenkins
b588e4af05 Merge "Use provides to note an element provides deploy" 2014-04-18 21:18:03 +00:00
Jenkins
6c966068b0 Merge "Remove dependency on /etc/lsb-release" 2014-04-18 20:20:34 +00:00
Jon-Paul Sullivan
ac73660c40 Remove the temporary deploy element after rename
This is dependent upon change in tripleo-incubator
I1ff4bb98c99dfe87ccc4fb19767b93e27707d3a7 and concludes the
renaming of deploy to deploy-baremetal

The rename was performed so that each element had a more descriptive
name, and so that each element could accurately set an element-provides
of "deploy", given that the elements enable tripleo deploys via the
ramdisk image.

Change-Id: Ic2ca5f95b8895fa0d6d7314cab9de30a7204541c
2014-04-18 20:18:54 +01:00
Jon-Paul Sullivan
0f12d45a50 Use provides to note an element provides deploy
Use "deploy" to denote that an element provides the deploy
capability for a ramdisk for tripleo installation purposes.

This will be a 3-part change to enable renaming deploy to
deploy-baremetal so that the generic "deploy" capability can
be used for the provides file, as a dependant change will have to
be made in tripleo-incubator.

The tripleo-incubator change is
I1ff4bb98c99dfe87ccc4fb19767b93e27707d3a7 and that swaps to using
deploy-baremetal in place of deploy for ramdisk creation when using
baremetal.

Change-Id: Ibb4026e8fc8ba6378061461c4796a91ab2fa991f
2014-04-18 20:10:46 +01:00
Jenkins
35a006cfa6 Merge "debian: simplify the way debian element is called" 2014-04-18 13:52:47 +00:00
Stephen Mulcahy
94d2cc42c6 Remove dependency on /etc/lsb-release
/etc/lsb-release doesn't always exist in cloud images and doesn't
serve any significant purpose in this element so removing the
dependency entirely.

Change-Id: I4d972074e6e10fea09a875cbc4ed4e60248d46d9
2014-04-18 11:43:17 +01:00
Gonéri Le Bouder
9cb0615f38 indent: replace tab by 4 spaces
As decided in I072cf8bf6748d0c910fecffdf2282bcc4656d038, tab indent
should be avoided.

Change-Id: I301b6319ef4f36878999b6109e1842f9e7cafd43
2014-04-18 10:32:43 +02:00
Jenkins
a9c0a7bf27 Merge "Add ability to add extra apt keys" 2014-04-17 22:47:58 +00:00
Jenkins
7b1c2b98e5 Merge "Document a little the concerns for operators." 2014-04-17 22:47:57 +00:00
Jenkins
4dc7d784b0 Merge "Change refspec used to fetch all branches and tags" 2014-04-17 20:23:05 +00:00
Jenkins
a4c48d78dd Merge "apt-conf: uninitialised variables fix" 2014-04-17 20:08:50 +00:00
Clint Byrum
404c695de9 Make "trusty" (Ubuntu 14.04) the default release
Ubuntu 14.04 was released today. Saucy will be EOL in 3 months.

Change-Id: I2043c3dfa8b795ae587b42cd80d3a0bf866bf82a
2014-04-17 11:20:06 -07:00
Jan Grant
fe33d34637 Add switch to turn on caching for debian element.
DIB_OFFLINE has wide-ranging effect. This patch adds a tunable that
only turns on the tarball-caching behaviour specifically for the
debian element.

Change-Id: Idd646d7909df6907afc0eeeec1129c3f1a1f92bc
2014-04-17 10:10:34 +01:00
Jenkins
95cdefebf6 Merge "Fix Grub configurations for Fedora images built on a UEFI host." 2014-04-17 07:48:06 +00:00
Dan Prince
0a797c86de Remove cloudy interfaces in dhcp-all-interfaces
Moves the code that removes default cloud interfaces
into the dhcp-all-interfaces element. Having it live in
stable-interface-names won't work right now because we
need this for virtualized testing.

Change-Id: I3166951c6fbb2faec53ffc339edd2fd0c3512300
2014-04-16 14:09:50 -04:00
Gonéri Le Bouder
eb26c1a0b1 debian: simplify the way debian element is called
“debian” element now source the ${TARGET_ROOT}/.extra_settings file. We
can use this file to adjust its behavior and specifiy what init system
should be used.

Change-Id: Id57ce879341184a57851d2578cf70978f409a48e
2014-04-16 17:39:18 +02:00
Jenkins
ed5a872adf Merge "Add mapping for mariadb rdo package" 2014-04-16 14:30:19 +00:00
Jenkins
c6c01fb237 Merge "Fix set -eu and pipefail failures" 2014-04-16 13:33:39 +00:00
Jenkins
c1af3770ec Merge "map-services: add apache2 in the list" 2014-04-16 02:22:00 +00:00
Jenkins
dd09afb670 Merge "Remove map-services from fedora element" 2014-04-16 02:21:55 +00:00
Ben Nemec
381ff6ab1d Fix set -eu and pipefail failures
Fixes problems found by set -eu and pipefail, including:
-Many unset variables
-Commands that can fail under normal circumstances, which breaks
with set -e.  This change swallows those expected errors to allow
our existing error code to handle them.
-The dkms element was not finding Fedora kernel versions correctly.
This may be an issue for other distros too, but since Fedora was
working fine without this functionality I only changed it to print
a warning message rather than failing the build when it happens.
-The ramdisk init script will not be set -eu because if it fails
the result is a kernel panic, which can be tricky to debug.
However, in testing with set -e a few failing commands were found
and have been fixed in this patch.

Change-Id: I44cf98dfc80cfcaec54b88cc83be80a3dbf2cec3
2014-04-15 20:53:15 -05:00
Jon-Paul Sullivan
b507cab2b5 Move instead of copy the temporary git manifest
This prevents a build-up of temporary files in the cache directory

Change-Id: Ice34b57ef4f24bbe69b43ee3b2e2e394894825e5
2014-04-15 11:24:43 +01:00
Jan Provaznik
40adbff681 Add mapping for mariadb rdo package
Mariadb package in RDO repository is mariadb-galera-server, but
this name is mapped to MariaDB-Galera-server because of
mariadb installed from mariadb.org.

To avoid mapping conflicts mariadb-rdo uses virtual package name
mariadb-rdo-galera-server which is mapped to mariadb-galera-server.

Change-Id: Idf77d91876baeb81d89eb688808b892ece2bf19a
2014-04-15 12:02:52 +02:00
Gonéri Le Bouder
707cb51c84 map-services: add apache2 in the list
apache2 service is called httpd on Fedora and RHEL unlike SUSE,
Debian and Ubuntu.

Change-Id: I125c46bebb0342ee3bdb957116e7c7e8e27a2ecc
2014-04-15 10:13:57 +02:00
Ben Nemec
2747e0139c Remove map-services from fedora element
This was supposed to have been moved to the redhat-common element,
and having it here too is both confusing and may break builds
depending on which version gets used.

Change-Id: Icae5cc633e9e49a9ee63bd9dfc94657c6795c6f1
2014-04-11 21:36:23 -05:00
Jenkins
ae9ccc4a23 Merge "Remove call to depmod in busybox" 2014-04-12 00:12:52 +00:00
Jenkins
b1a4eda4ee Merge "Standardise manifest creation and retrieval" 2014-04-11 22:10:52 +00:00
Jon-Paul Sullivan
793dae8fe6 Use provides to note an element provides an OS
Use "operating-system" to denote that an element provides that to the
image being created.  This can then be used to specify an abstract
dependency in other elements to assist in build-time image verification.

Change-Id: Ic125803da65e0f8f8c20ef90b08815f4f7d09762
2014-04-10 14:42:04 +01:00
Jon-Paul Sullivan
4c467964f5 Standardise manifest creation and retrieval
* Create a standard element to copy manifests out of the image and
  into the build area.
* Ensure all current manifests are copied into a standard manifests
  directory

Change-Id: I37eff5a13a14564e1adc33eb4f0144d571267617
2014-04-10 12:25:13 +01:00
Nicholas Randon
d11c10fb6a Remove call to depmod in busybox
The version of depmod in busybox does not work if two versions of
the same kernel version are available, as search paths are not
honoured correctly.  However, we don't need to actually call
depmod as it's done during image building anyway.

Change-Id: Idd3cd5854e7c86e7ca1f2b0e50758d7db79ab6e7
Co-Author: matthew.gilliard@hp.com
Closes-bug: #1302856
2014-04-09 10:10:11 +01:00
Stuart McLaren
06ccb7484f apt-conf: uninitialised variables fix
In the bash script "99-override-default-apt-conf" 'set -u'
is being used to check for uninitialised variables.

This resulted in generic 'unbound variable' messages rather
than the intended error messages.

Tweaked the check for DIB_APT_CONF to work if the variable is unbound;
so that we get the intended error message rather than the `unbound
variable` message.

Change-Id: Ib8e287060f32312ab80dd7f44e2792ecca48edbb
2014-04-09 09:12:31 +02:00
Jenkins
0fc211a82a Merge "Remove hardcoded /tftpboot/ from token's tftp path" 2014-04-08 17:24:59 +00:00
Jenkins
53561fc295 Merge "debian: use sudo to create file in the chroot" 2014-04-08 13:09:17 +00:00
Steven DuChene
2408569566 Add ability to add extra apt keys
Adds an environment variable DIB_ADD_APT_KEYS pointing to a folder of
apt repo GPG keys. Each key will be verified and installed with apt-key
to the apt keyring.

Change-Id: I4b15347a76aa3241e741931bb94f17598641ae42
2014-04-07 13:51:30 +01:00
Robert Collins
3b7d6694e9 Document a little the concerns for operators.
Change-Id: I2429371d3c5d931d46674c4e46d99599fda9f42e
2014-04-05 20:17:30 +13:00
Gonéri Le Bouder
4004985844 debian: use sudo to create file in the chroot
The ${TARGET_ROOT} chroot is owned by the root user.

Without this change, the build will fail for a standard user
with a “Permission denied” error.

Change-Id: I6eb87d3a14292da5620bdfd623ea7ef846fe2af6
2014-04-04 10:14:26 +02:00
Om Kumar
2fb1b91dec Disable splashimage for legacy grub.
There is a known issue in legacy grub. When using splashimage with
serial console, Legacy grub wont start properly. This commit is to
ensure that when serial console is used, legacy grub should not
contain splashimage. It also adds serial support to legacy grub
configuration file.

Change-Id: I2007916998dfe7c66ef51bf8c57c1698dc0fcb75
2014-04-03 15:16:51 +05:30
Jon-Paul Sullivan
f8416b1fe5 Change refspec used to fetch all branches and tags
When the repository references are sha1s the fetch command used will
not work.  This will ensure that all of the objects are still fetched,
and all of the local cache references for branches and tags will be
updated rather than just the specified reference in use.

Change-Id: Iad2554b3087a5da37dac6e9f6a72432d49fce83d
Closes-Bug: #1301435
2014-04-02 16:25:48 +01:00
Gonéri Le Bouder
341d7aaff7 fix grub2 installation on Debian Wheezy
--target has been introduced in grub2. This patch ensure we use
the minimal version before adding the --target XXX flag.

Debian Wheezy and Precise share the same limitation regarding the
root device.

Change-Id: I5b3ad42b9fb536d13fb65d84339bc15bc23d99a3
2014-04-02 11:37:49 +02:00
Om Kumar
50d1e09df1 Fix Grub configurations for Fedora images built on a UEFI host.
The grub install scripts on Fedora have hardcoded checks for
/sys/firmware/efi and if the directory exists, it uses efi
specific instruction like initrdefi, linuxefi etc. vm element
should fix it once the configuration is generated.

Closes-bug: #1231946
Change-Id: I888304cb9167b8e074b04ca6c5b2f9143b70d37d
2014-04-02 03:09:29 +00:00
Robert Collins
f49e8a42c4 Fix dhcp-all-interfaces upstart job
This was calling it for all interfaces previously, making the fixes
we'd put in place be ineffective.

Change-Id: I16bf81225a942e3b19a7e745d949e6ddea2fd50e
2014-04-02 09:14:03 +13:00
Robert Collins
c7875398b3 Fix resource exhaustion with upstart.
When high VM churn is encountered dhcp-all-interfaces got tangled up
and leaked instances until upstart run out of filedescriptors.

This happened because we were locking on every invocation, and the
lock logic was disabling the single-interface code - so we always
probed every interface every time - and this was serialized across
every instance of dhcp-all-interfaces.

Change-Id: I4ef1fb83d978cc99d4cb9cc2666bde302c6db7d1
2014-04-01 22:45:12 +13:00
Jenkins
6090f520d1 Merge "Fix "(None)" seed hostname on Debian" 2014-03-31 15:41:08 +00:00
Jon-Paul Sullivan
843085a7cd Write a dpkg manifest to list installed packages
To create an auditable log of packages installed in an image use
dpkg-query to create a json data file with the package names, version
and install status.

Change-Id: I59018221edcb944a876ad47b402538c63c426bc7
2014-03-28 18:12:02 +00:00
Jenkins
b9189a204f Merge "Uses policy-rc.d to prevent dpkg starting daemons" 2014-03-27 18:38:06 +00:00
Jenkins
eacdb2a4a5 Merge "dhcp-all-interfaces: correct ifquery return stmt" 2014-03-27 18:01:52 +00:00
Jonathan Brownell
5af25b5f46 Fix "(None)" seed hostname on Debian
The cloud-init package on Debian Linux overwrites the system
hostname to "(None)" unless a hostname is specified in one
of the /etc/cloud/cloud.cfg.d/* files. This change applies
a default hostname of "debian" using cloud-init.

Change-Id: I9b411df44cfa05f899e708900f95719c0bcd6267
2014-03-27 10:08:03 -07:00
Jenkins
5feeb06738 Merge "Make stable-interface-names its own element." 2014-03-27 16:14:08 +00:00
Dan Prince
6f764d9119 dhcp-all-interfaces: correct ifquery return stmt
Corrects an issue in the config_exists function that could
cause duplicate interfaces to be configured on Ubuntu.

Previously we used 'return ifquery' directly which caused
a silent 'numberic argument required' error to go unnoticed.
This would also return 1 meaning a new interface would get
configured.

The new logic should handle things properly.

Change-Id: I625225e15113d7e184e3bcb5054df1616dec008a
Closes-bug: #1298430
2014-03-27 10:47:55 -04:00
Jenkins
54d16fa9f8 Merge "Adds package mapping for mariadb packages" 2014-03-27 13:30:14 +00:00
Jenkins
57ff15ea80 Merge "rename udev.rules to dhcp-all-interfaces-udev" 2014-03-27 07:24:40 +00:00
Dan Prince
99c54323a0 Make stable-interface-names its own element.
Virtual baremetal instances have their NICs show up as ens4
which doesn't play nicely as a default in devtest at
the moment.

Ideally we'd want it to be em1 like real baremetal
(which is why we put this in the baremetal element
to begin with). Turns out the baremetal element is
required in devtest so we can extract the ramdisk and kernel
so lets leave it alone for now and move this elsewhere
until we get a better solution for fake 'baremetal'
testing.

Closes-bug: #1298152
Change-Id: Ia71e1d32b93db0c4c844a6dc1ebcd04ab0c13f05
2014-03-26 22:25:39 -04:00
Jenkins
f93147795c Merge "debian: install cloud-init on Wheezy" 2014-03-26 23:28:39 +00:00
Thom Leggett
46a14ae0e6 Uses policy-rc.d to prevent dpkg starting daemons
Currently the dpkg element fakes start-stop-daemon, initctl and
invoke-rc.d to stop dpkg from starting a bunch of daemons in the
chroot. This is problematic when packages use service, start, stop or
restart commands.

This patch uses a policy-rc.d instead of faking invoke-rc.d to achieve
the same thing. This approach now aligns exactly with debootstrap.
Without this patch DIB runs on some debian distributions fail
when trying to umount the chroot loop device as there are daemons
running.

The log will now show "invoke-rc.d: policy-rc.d denied execution of
start." instead of "Warning: Fake invoke-rc.s called, doing nothing."

Change-Id: I6db192127aca19b5b7915179b781f5192078bfc7
Related-Bug: #1211564
2014-03-26 20:59:49 +00:00
Ben Nemec
63230414d2 Alphabetize all element-deps
Change-Id: Idc31d60daeea61d6df0b70dac4a5671fa5a804ee
2014-03-26 15:23:18 -05:00
Jan Provaznik
507b717c61 Adds package mapping for mariadb packages
Change-Id: I6fb0e176da3f73cb2388cd833395f5880c4ce3c3
2014-03-26 21:12:58 +01:00
Trent Geerdes
8007882726 Adding pypi-mirror dependencies for redhat
Change-Id: I030f27460c92ce0b59188e9469206b293271697a
2014-03-26 06:44:15 -06:00
Jenkins
a78e33f617 Merge "Add console kernel parameters to extlinux configuration" 2014-03-26 09:47:40 +00:00
Gonéri Le Bouder
2514b822d2 debian: install cloud-init on Wheezy
cloud-init packages are not available in the Debian Wheezy repository.
However, they are available in wheezy-backports.

With this change, the cloud-init packages are installed from
wheezy-backports if DIB_RELEASE is wheezy.

Change-Id: I3daa3b0ba185ba3f7c1bf6b8f9b1f8c167fcb340
2014-03-26 10:21:13 +01:00
Jenkins
73c3b2c404 Merge "serial-console: Use udev rules to startup getty" 2014-03-26 05:33:52 +00:00
Jenkins
440de95f41 Merge "Set +x on executable files" 2014-03-25 20:54:41 +00:00
Jenkins
593680e936 Merge "DHCP: make udev rules want dhcp-interface@.service" 2014-03-25 20:23:14 +00:00
Jenkins
8c5aff82a2 Merge "baremetal: correct the path of ifcfg-eth0" 2014-03-25 15:50:31 +00:00
Luigi Toscano
be3efed80b Add console kernel parameters to extlinux configuration
Those parameters allow to access the console for VMs using
extlinux (CentOS 6 for example). They are already added to
Fedora images.

Change-Id: I035170bd101062e004fb3782b5f8e5a7944a4f03
Closes-Bug: 1296899
2014-03-25 14:24:43 +01:00
Robert Collins
ee18fe41da Permit use of wheel mirrors in pypi element.
pypi-mirror creates a separate mirror index for wheels (one per OS
that mirrors are built on). To be able to use it one then needs to be
able to export multiple mirrors for inclusion in pip.conf. As a drive
by I made it possible to disable the use of the pypi.python.org index
without using --offline (as --offline has larger impact).

Change-Id: I3e85a8069b18cafd7eae4cd0591821acc3b5a739
2014-03-25 16:48:18 +13:00
Dan Prince
6957bd8db3 baremetal: correct the path of ifcfg-eth0
Remove the correct ifcfg-eth0 file on Fedora.

This will require an associated incubator change as well
to ensure the undercloud/overcloud now use em1.

Change-Id: I4b4bb7b6a25b058eebe889fd8ce1a403aaf193fd
2014-03-24 09:20:40 -04:00
Jan Provaznik
2f24062753 Adds libmariadb-dev package mapping
libmariadb-dev is named mariadb-devel on Fedora. mariadb-devel
package is needed for upcoming mariadb element.

Change-Id: Ib62071a964a4449c4e32ef1646ab7a0fb0f5f7b7
2014-03-24 10:39:43 +01:00
Dan Prince
8378072446 rename udev.rules to dhcp-all-interfaces-udev
Renames to udev.rules files to avoid potential
install.d collisions in DIB.

Change-Id: I2d70aa8b6e3e5fe957cb81c57f378d9623809c03
2014-03-21 15:13:35 -04:00
Dan Prince
576fa64abf serial-console: Use udev rules to startup getty
Update the systemd serial port element implementation
so that it uses udev rules. Much nicer than
hard coding ttyS0 and ttyS1.

The udev rule only adds real serial device types
(i.e. == 4). See:

  define TTY_DRIVER_TYPE_SERIAL 0x0003

from include/linux/tty_driver.h.

Change-Id: I5d7de4815fd09c01693b7da4b8949f22fee20745
2014-03-21 15:01:39 -04:00
Ben Nemec
81957625e7 Set +x on executable files
Change-Id: I6007aa3f4e6326b1c7030ad0463c7e8f49fbd507
2014-03-20 15:38:30 -05:00
Jenkins
94805549d8 Merge "Add RHEL 7 image element" 2014-03-20 11:33:24 +00:00
Jenkins
bc3950c7a5 Merge "Use redhat-common in fedora element" 2014-03-20 02:13:25 +00:00
Jenkins
62f8f4b120 Merge "Add redhat-common element" 2014-03-20 02:10:52 +00:00
Dan Prince
00d853e5fd DHCP: make udev rules want dhcp-interface@.service
Updates the dhcp-all-interfaces element to fix a race
with the recent udev rules implementation on Fedora.

With the new approach we make the udev rule want (require
to startup) a generic dhcp-interface@.service template which
can be started individually for each interface that is
discovered.

The dhcp-interface@.service is setup such that it:

1) It calls dhcp-all-interfaces <iface> directly with
 a pre-exec script. This creates the ifcfg file right
 before we need it but avoids the case where network.service
 might get greedy and try to start it itself.

2) Only runs if the ifcfg script doesn't already exist. This
 is important because we only need to bootstrap the DHCP configs...
 Once they exist the network.service will take care of starting them
 on reboots, upgrades, etc.

3) On initial boot ensure that the initial DHCP interfaces come
 up after network.service. Since we really only want
 dhcp-all-interfaces to help bootstrap that haven't already
 been configured this seems reasonable.

4) We also try to ensure that cloud-init
 comes up after the DHCP interfaces. Cloud init has a decently
 long timeout that this wasn't a functional problem but it keeps
 log file spew down.

Change-Id: I71b026f027182aad49c3435bb903e5e38e524685
Closes-bug: #1294803
2014-03-19 14:33:14 -04:00
Jenkins
39964b975e Merge "Update openSUSE package mapping for libffi-dev" 2014-03-19 01:18:17 +00:00
Ben Nemec
3b77e1b3c5 Add RHEL 7 image element
This is being added as a separate element because RHEL 7 is too
different from either RHEL 6 or Fedora to share all of the
configuration they do.  It uses the redhat-common element for
pieces that are duplicated.

Some current limitations of the element:
-RHEL 7 is currently still in beta, so final image locations are
 not available.  This currently defaults to the RHEL 6.5 image,
 but that will need to be overridden using the DIB_* env vars.
-There are additional tripleo-image-elements changes that will be
 needed to be able to use this for building tripleo images.
-Open vSwitch packages are not currently available in the official
 RHEL 7 repos.
-RHEL does not support qemu without kvm acceleration, so at this
 time it cannot be used as a compute node for virtualized testing.

Change-Id: I705fd475303576525a27ce6801c673b5721792c9
2014-03-17 16:14:41 -05:00
Ben Nemec
94f07c5962 Use redhat-common in fedora element
Have the fedora element use the common steps in the redhat-common
element.

Change-Id: Ic71b11c03b5c3868cc9e82e049f2700c1b67263a
2014-03-17 16:14:41 -05:00
Ben Nemec
ed15edf165 Add redhat-common element
Add an element intended for use in both Fedora and RHEL.  This
allows them to share install steps that are common to both.

Change-Id: Ie4e820a7b777b8701514351b1f802cfe57c3812e
2014-03-17 16:14:36 -05:00
Jenkins
5af4c268f9 Merge "Support adding DHCP interfaces one at a time." 2014-03-17 19:52:41 +00:00
Jenkins
f2578f2614 Merge "Move install type enablement into base element" 2014-03-17 13:07:07 +00:00
Dan Prince
f10e614579 Support adding DHCP interfaces one at a time.
Refactors dhcp-all-interfaces.sh so that if an optional
INTERFACE argument (the first argument) is passed to the script
it only inspects that single interface. If no argument is
passed then the previous default behaviour is used which
causes all interfaces to be inspected.

To avoid a collision with the previous $1 we move to using
$FLOCKED for the exec flock command which runs on distributions
using ENI.

Also sets PATH so that the commands within the script
can all be found if it isn't set properly (/sbin/ip, /bin/cat, etc.)

This is a move towards using udev rules to add these types
of interfaces automatically.

Change-Id: I3ec8fd2cc2071bfc6943c744ca619e31b71146fc
2014-03-14 13:30:17 -04:00
Jenkins
3f9299e12b Merge "RHEL Package maps for build-essential, python-dev and libz-dev." 2014-03-14 17:03:57 +00:00
James Slagle
b4dfa6cb90 Move install type enablement into base element
Move whether the package or source install type is enabled out of the
source-repositories element and into the base element.
source-repositories wasn't a great fit for this functionality to begin
with.

Specify the install type for an element via the
DIB_INSTALLTYPE_<install_dir_prefix> environment variable.

Additionally with this approach, any new install type can be added
in an element, without having to update source-repositories or anything
in dib.

You could just write your install type implementation under
nova-foo-install, then define DIB_INSTALLTYPE_nova=foo in your
environment, and the scripts under nova-foo-install would get run during
the image build.

Source installs (any scripts under <install_dir_prefix>-source-install)
is the default install type for all elements.

Change-Id: I9414aca360c41e030e27d3d0c0a52d9d8e13d8b1
2014-03-14 12:25:11 -04:00
Alexander Gordeev
7d50c04bf7 Remove hardcoded /tftpboot/ from token's tftp path
Since file to be fetched from the root tftp directory it shouldn't
contain any hardcoded path. Additionaly it eliminates bearing with
map-file containing regular expressions for tftp path replacement.

Change-Id: Iee1672834f735d7dfefce43000540522e1d053b9
Closes-Bug: #1291533
2014-03-14 18:59:26 +04:00
Jenkins
14140ae180 Merge "Add apache2 mod_wsgi pkg map for suse" 2014-03-14 14:02:52 +00:00
Jenkins
09ad2ae482 Merge "Extract move cache logic to a function" 2014-03-14 08:08:24 +00:00
Om Kumar
dd7b7765e6 RHEL Package maps for build-essential, python-dev and libz-dev.
Some of the tripleo-image-elements were failing for RHEL due to lack of
build essential, python-dev and libz-dev.

This commit should fix those failure.

Change-Id: I080f9601faedb3259deada5cfbda7d00d3f13eb9
2014-03-14 08:54:48 +05:30
James Polley
ca95cf9f70 Extract move cache logic to a function
We use the same logic twice - extract it to a function.

Change-Id: I32a257663de37d5f3588c91e4164abceb12ee80d
2014-03-14 09:36:52 +11:00
Jenkins
bae43c91c6 Merge "98-source-repositories tries to return from script" 2014-03-13 16:17:51 +00:00
Jenkins
c9282cf2ee Merge "Better apt-sources docs." 2014-03-13 15:13:34 +00:00
Jenkins
7b0119808d Merge "Create a git manifest from source-repositories" 2014-03-13 15:11:40 +00:00
Jenkins
12463106c8 Merge "Enable extlinux support for (non-Ubuntu) Debian platforms" 2014-03-13 03:34:18 +00:00
Dan Prince
6f8f6b049b baremetal element: Enable stable interface names
By default most cloud images have disabled the
udev rules which give us stable network interface names. While
this is a reasonable default for something like EC2 (the target
for these images) this is very bad behavour on bare metal
and can cause a "musical NICs" situation to occur where
on reboot the NIC names get swapped.

In this commit we re-enable stable interface naming by
removing the dev nulled rules softlink and installing
the biosdevname package. Once this is done interface
names should again show up as em1, em2 instead of
the old eth0, eth1.

We also remove some the common, statically configured
eth0 config files on some distributions. Equivalent
files (if needed) may be auto-generated by using
the dhcp-all-interfaces element.

Change-Id: Idd67977342719e2f295e2fa2c0ed6cfa8602171a
2014-03-12 16:48:48 -04:00
Jenkins
66cf95c321 Merge "Fix unbound variable in debian element" 2014-03-12 10:59:46 +00:00
Jenkins
6dd90ae476 Merge "Add libaio1 to libaio pkg map for fedora" 2014-03-12 10:59:00 +00:00
Jon-Paul Sullivan
179aa31d41 Fixup all occurrences of REPONAME for replacing '-'
There was a single REPONAME variable missed in change 76679.
Fix it up.

Change-Id: Ibc6cd59cb3f3d949fd7679ee18d85c7574d9acd5
2014-03-11 13:13:10 +00:00
Jenkins
ffdd57b8a5 Merge "Replace more then just "-" in REPONAME" 2014-03-11 12:51:31 +00:00
Jenkins
d0ba5a9dfd Merge "Additional swift storage service mappings" 2014-03-11 12:48:21 +00:00
Jon-Paul Sullivan
e4c43e608c Create a git manifest from source-repositories
As git repositories are copied into the image make a note of their
details and add them to a manifest on the build system.

This allows the code used in building an image to be found after
the image build is completed without needing to open the image up
for inspection.

Change-Id: I7dbe9e163ad38a418cf2869a81e720de2c27dfb1
2014-03-11 12:38:52 +00:00
Gregory Haynes
7b550e3c96 Add apache2 mod_wsgi pkg map for suse
Horizon installs libapache2-mod-wsgi which is called apache2-mod_wsgi on
suse.

Change-Id: Id80b6f9bd2131906a25722e515aa9f14720f323b
2014-03-10 19:44:27 -07:00
Dan Prince
ef379f05e4 98-source-repositories tries to return from script
In 8b2325118f we added a line
so that 98-source-repositories tries to return from the top
level script (not from within a function).

Also, remove the return 1 within the function as well
because we just want to continue in these cases.

Change-Id: I66eddc12208b278594a0a8d8676c38d72045ca75
2014-03-10 17:21:20 -04:00
Gregory Haynes
0bd84775e7 Add libaio1 to libaio pkg map for fedora
The libaio1 package for ubuntu is called libaio on fedora

Change-Id: I81c7baba4ab50e2fa05a950b0773cfe07bb0b3e9
2014-03-10 12:14:15 -07:00
Jenkins
1080b40368 Merge "Add rsync to the package list used by debootstrap" 2014-03-10 19:02:37 +00:00
Jenkins
e83d4dcbb2 Merge "Bash eval the lines in source-repository scripts" 2014-03-10 18:55:06 +00:00
Richard Su
f4c8075768 Additional swift storage service mappings
Change-Id: I07ad7a5b175bf7730e9425a66122c63477fb5e21
2014-03-07 09:16:11 -08:00
Ralf Haferkamp
5044113fe8 Update openSUSE package mapping for libffi-dev
Change-Id: I9c5ec5bd839a51a5292be37a7696499554c6ae16
2014-03-06 10:51:05 +01:00
Robert Collins
e5f120274b Better apt-sources docs.
Minor layout tweaks, fix markdown syntax errors and mention the new
SEED_DIB_EXTRA_ARGS variable.

Change-Id: I18104c80e64d06b084df57ee5adc15519bc3ddf6
2014-03-06 13:29:13 +13:00
Ryan Brady
d96740dc8a Add nfs-common package mapping
NFS client utilities for Debian are found in nfs-common, and the
Fedora equivalent is found in nfs-utils.  The NFS server components
are already accounted for in the nfs-kernel-server mapping.

Change-Id: I2a43208f54b7fb625c239ce22231c43e194a97ed
2014-03-05 14:31:28 -05:00
Derek Higgins
0226cf2246 Replace more then just "-" in REPONAME
Looks like shells only support alphanumeric characters and "_"'s in
environment variables. We were substituting "-" characters but in
order to be able to set overrides in projects with names containing
other characters we need to substitute more, I'm looking at you
oslo.config.

Change-Id: I3e2b1b0bc5871e4ec4ffd8117906cd077aa2cb0d
Co-authored-by: James Polley <jp@jamezpolley.com>
2014-03-06 04:56:50 +11:00
Jenkins
5bcc1365e3 Merge "Alphabetize openSUSE services dictionary" 2014-03-05 09:16:07 +00:00
Jenkins
9edd996056 Merge "Alphabetize openSUSE packages dictionary" 2014-03-05 09:16:06 +00:00
Jenkins
3f4e44436b Merge "Alphabetize Fedora services dictionary" 2014-03-05 09:14:01 +00:00
Jenkins
af1369ee99 Merge "Alphabetize Fedora packages dictionary" 2014-03-05 09:13:55 +00:00
Jenkins
e602863642 Merge "Replace use of show-ref with name-rev" 2014-03-04 18:05:46 +00:00
Jenkins
d55d54d9cf Merge "Fix syntax error in GRUB_OPTS env var handling" 2014-03-04 18:04:29 +00:00
Gregory Haynes
6a6d913459 Bash eval the lines in source-repository scripts
This will allow us to add environment variables to source-repository
scripts. Useful for things like arch-specific download urls.

Change-Id: I4d046825a317dc88db3d7673ad261cedacd1a104
2014-03-03 16:37:43 -08:00
Jenkins
9595197474 Merge "Improve usability of the source-repositories cache" 2014-03-04 00:21:52 +00:00
Jenkins
5b7f6dcc1d Merge "Fix spelling error in "vm" element README.md" 2014-03-03 22:12:12 +00:00
Jenkins
27421cb257 Merge "Add $EXTRA_ARGS back to yum call" 2014-03-03 18:37:17 +00:00
Jonathan Brownell
9c05300728 Fix spelling error in "vm" element README.md
Change-Id: I7b5e38277843da1ee6496292a7d6ff147c97cf31
2014-03-03 09:58:12 -08:00
James Slagle
30d1668c03 Add $EXTRA_ARGS back to yum call
This got removed accidentally in a recent consolidation of
install-packages.  Adding it back so that -d support works again.

Closes-bug: #1287142

Change-Id: If17e613bc9bc0f808517edfa1506858d3b9dce54
2014-03-03 17:42:18 +00:00
Perry Myers
b9e77477bb Update the image name for RHEL Guest Image
The RHEL 6 Guest Image has been updated with a new release that
addresses a number of bugs.  The diskimage-builder element that
references this file needed to be updated to reference the correct
image name.  In addition, some notes to help users download the
image from RHN have been added to the README.

Change-Id: I3c9e64a1887afd9c0b370d70eff4b7f9598cebd1
Closes-bug: launchpad-bug-1286568
2014-03-01 12:23:02 -05:00
Newell Jensen
1d3ceef5c2 Alphabetize openSUSE services dictionary
This patch alphabetizes the openSUSE services dictionary in
map-services.

Change-Id: Ib3b596e7c25475ecdc2a6bbeb98b84bb60e03ac3
2014-02-28 10:12:03 -08:00
Newell Jensen
80e07afab4 Alphabetize openSUSE packages dictionary
This patch alphabetizes the openSUSE packages dictionary in
map-packages.  The openstack mappings were left
at the bottom as it looked deliberate.

Change-Id: I3661fba4d4ed68c836893e21fb4a8728c691af20
2014-02-28 10:10:22 -08:00
Newell Jensen
1646d30d1d Alphabetize Fedora services dictionary
This patch alphabetizes the Fedora services dictionary in
map-services.

Change-Id: If35b0249c33e553937e7016c2480aee45f1c7353
2014-02-28 09:50:46 -08:00
Newell Jensen
8896af2cc7 Alphabetize Fedora packages dictionary
This patch alphabetizes the Fedora packages dictionary in
map-packages.  The openstack mappings were left
at the bottom as it looked deliberate.

Change-Id: I4874c19f2a6826062874698e8f61bc6e2685ab19
2014-02-28 09:47:56 -08:00
Jonathan Brownell
1eabcdf35f Enable extlinux support for (non-Ubuntu) Debian platforms
Extlinux bootloader configuration in "vm" element requires vmlinuz and
initrd.img files in /boot/ with "generic" suffix. When these don't exist,
bootloader configuration should fall back to using whatever does exist.

Change-Id: I79d426e8f008e9f23db899f0b8f8cf2f23643e28
2014-02-28 06:56:41 -08:00
Jon-Paul Sullivan
41f1ca2ad4 Replace use of show-ref with name-rev
Testing showed that show-ref did not accurately determine whether a
git repository contained the given reference.  Testing with name-rev
showed that it did.

Change the HAS_REF determination appropriately to avoid unnecessary
cache updates when using fixed references.

Change-Id: Iece213348516358af84417807f8dd36e959139eb
2014-02-28 14:35:07 +00:00
Jenkins
e5afa4d6f9 Merge "Add NFS package mapping" 2014-02-28 10:47:24 +00:00
Jenkins
4a45704edf Merge "Enable simple modification of git repo location" 2014-02-28 04:32:29 +00:00
Jenkins
4dfd81a70b Merge "Generalize install-packages for yum" 2014-02-28 04:20:17 +00:00
Jenkins
142dfad122 Merge "Enable custom apt.conf in apt-conf element" 2014-02-28 04:17:51 +00:00
Jonathan Brownell
2189b627de Fix syntax error in GRUB_OPTS env var handling
Reference to ${GRUB_OPTS:""} in 'vm' element causes syntax error
if GRUB_OPTS is set. Fix by changing to ${GRUB_OPTS:-""}.

Change-Id: Iae3ee8b7c4064c92d0cfc1b5a8caa22e53d6d112
2014-02-27 13:53:07 -08:00
Robert Collins
2843e8a802 Improve local-config proxy handling.
We were applying proxy settings after apt-get install runs are done
in the Ubuntu element, which made it ineffective. Secondly, we were
not applying no_proxy settings to apt, which caused it to fail in
corporate settings. More explicit control may be facilitated in future
but local-config is a DTRT tool, and so we should help it DTRT.

I think I've correctly setup Zypper for this, and I have no idea
how to make the same improvement for Yum (and also Yum currently
doesn't set HTTPS proxies, which may lead to similar reports there).

Change-Id: Idbb9781d64d44b717ea22a25429383fa45d84148
2014-02-28 00:01:11 +13:00
Jon-Paul Sullivan
852cec3789 Enable custom apt.conf in apt-conf element
Allow DIB_APT_CONF to be set to the apt.conf file to be inserted
into the built images.

Change-Id: I5d148ae7286a93e81b3e984c94ba4f089d3a3280
2014-02-26 19:10:23 +00:00
Jonathan Brownell
263e0310dc Add rsync to the package list used by debootstrap
Debian builds fail due to missing rsync in the baseline set
of packages that debootstrap defaults to. Rsync is used by a
variety of tripleo-image-elements and should be included in
disk images by default.

Change-Id: I8d83a53dfab279075e11f9825f15ee960395dd17
2014-02-26 10:53:55 -08:00
Jon-Paul Sullivan
0edfcf6587 Enable simple modification of git repo location
Add the DIB_GITREPOBASE variable to the source-repositories element
to enable global changing of git repository location.

Change-Id: Icfac5e103fce6e5a3f1a2db094144a0ed89fcc33
2014-02-26 13:52:03 +00:00
Jenkins
61d474f2cd Merge "Update pypi element to suggest pypi-mirror" 2014-02-26 08:52:19 +00:00
Ryan Brady
90d416e857 Add NFS package mapping
In Fedora, the package is nfs-utils and in Debian the package is
'nfs-kernel-server'.

Change-Id: I34f793bb6eb43f749bf383a082e90371f8e28219
2014-02-25 19:28:06 -05:00
Jon-Paul Sullivan
8b2325118f Improve usability of the source-repositories cache
* Rename the cache directory for sources

Make the cache directory name match that of the element that creates
it.  This makes it easier to find and more logically implies what
element created the cache directory.

* Add REPONAME to repository cache directory

For better usability add the reponame to the directory created
to cache it.

* Add migration code

Move pre-existing cache directories of the old name to the new name.

Change-Id: Ic62f19acb1746888ccbdf95eebf6e653f3e66357
2014-02-24 20:42:48 +00:00
Ghe Rivero
48e6ceebe9 Fix unbound variable in debian element
Change-Id: Ie4bf86fa0ce387a55f1c390f58e58d17267ae61d
2014-02-24 10:19:30 +01:00
Steve Kowalik
a84611d766 Replace security.ubuntu.com when setting mirror
The current method to rewrite the mirror URL in the ubuntu element will
only rewrite archive.ubuntu.com, which leaves the security URL alone.
This is probably not intended for people who are creating images behind
firewalls.

Change-Id: I54a867e0e9d5a7c94324a5c00ff1a83b4e6e4524
2014-02-24 11:00:24 +11:00
Ben Nemec
004474e93b Generalize install-packages for yum
All of the yum-based install-packages scripts are doing essentially
the same thing, so let's use one script for all of the elements
that depend on the yum element.

Change-Id: I49e8c9b44e41bcf4cb9fa820e8a9179754694a97
2014-02-21 12:00:24 -06:00
Jenkins
6360b46229 Merge "Add 00-fedora-fixup-vim" 2014-02-21 14:31:54 +00:00
Jenkins
86cd539496 Merge "Permit using arbitrary PyPI urls." 2014-02-21 04:43:26 +00:00
James Slagle
959aed3f86 Remove accidental merge marker
Change-Id: If98066c418f64be9f1bdce16558969d800e602c0
2014-02-20 15:10:32 -05:00
Dan Prince
4f795b91ed Add libvirt-bin -> libvirtd to map-services.
We'll need this soon for the Nova TripleO image element.

Change-Id: I9f4284aa503f422828e2ccb56597852b5a0c9591
2014-02-20 14:38:56 -05:00
Jenkins
66ae41756b Merge "Pep8/Pyflakes fixing" 2014-02-20 04:28:41 +00:00
James Slagle
4072ae232f Add 00-fedora-fixup-vim
We need to update vim-minimal, otherwise when rabbitmq-server gets
installed and pulls in vim-common, we run into:

https://bugzilla.redhat.com/show_bug.cgi?id=1066983

Change-Id: I910cd6c5c5fcfc6f6780b86029b1b4e660dd8d05
2014-02-19 08:32:25 -05:00
Clint Byrum
d03b14b4eb Update pypi element to suggest pypi-mirror
jeepyb no longer contains pypi-mirror.

Change-Id: Ifa905181119b30b1184dde53deaf0a97e2cb2ab2
2014-02-19 12:10:38 +00:00
Robert Collins
1adf3a377e Permit using arbitrary PyPI urls.
The pypi element is cool, but some folk have local network mirrors
which we should permit them to use.

Change-Id: Ie840ad1184e72b0e01966eee0298cfd6511b6c19
2014-02-19 14:05:31 +13:00
Dirk Mueller
0ab2a1300f Pep8/Pyflakes fixing
Just avoiding the error popups in the editor that way.

Change-Id: Ic0cb903dfb0e9eff6b12d004da720919f6f1a009
2014-02-17 22:37:34 +01:00
Steve Kowalik
98fa6c882f Shift debian element to DIB_DISTRIBUTION_MIRROR
The ubuntu element supports simple replacement of sources.list by using
the DIB_DISTRIBUTION_MIRROR environment variable. Drag the debian
element into line by supporting both DIB_DEBIAN_MIRROR and the new shiny
DIB_DISTRIBUTION_MIRROR.

Change-Id: I9d5f8d1e3251965b34e55929182aa601a524fe8f
2014-02-17 14:16:25 +11:00
Jenkins
e9b4ec2104 Merge "Teach cache-url to handle file:// URIs." 2014-02-14 12:23:40 +00:00
Robert Collins
1dfd4b3e44 Revert "Add Fedora DHCP interfaces via udev rules."
This reverts commit 201c9e8527.

This depended on the revert for one-at-a-time.

Change-Id: I05f1f85b8f13a1eea3d5e4b144ff92e0e791bdac
2014-02-14 14:43:09 +13:00
Robert Collins
39e6f7a5c4 Revert "Support adding DHCP interfaces one at a time. "
This reverts commit a1b469b10b.

This broke testenv deployments - it does not yet work on Ubuntu.

Change-Id: I0f74abc6b99da4b8cabca8ed673c606bae7de0e1
2014-02-14 14:42:58 +13:00
Chris Jones
fde4572764 Teach cache-url to handle file:// URIs.
We can now support file:// URIs in source-repositories files and other
places, by ignoring curl's inability to determine an HTTP status code
for them.

Change-Id: Ie226ce05ecb912b23cedc73fd7ba53c0b9c0fe35
2014-02-13 19:43:50 +00:00
Jenkins
b4e2968aea Merge "Add Fedora DHCP interfaces via udev rules" 2014-02-13 16:44:12 +00:00
Jenkins
0d52928a53 Merge "Support adding DHCP interfaces one at a time." 2014-02-13 16:43:47 +00:00
Jenkins
14ad0df6e7 Merge "Don't hardcode environment.d" 2014-02-13 08:45:43 +00:00
Dan Prince
201c9e8527 Add Fedora DHCP interfaces via udev rules
Updates the dhcp-all-interfaces element so that
we add new DHCP interfaces using a udev rule
instead of a systemd service.

The previous approach (using the systemd service)
was proving to be problematic when used with
the new Fedora constistent network naming
convention rules (em1, em2, etc.). We aren't using
these device names as a default yet... but we will
need to soon and when we do udev rules seem to be the
best way to avoid racy behaviour when calling
dhcp-all-interfaces.sh from the systemd service.

Change-Id: I87e0aed1f34718e896bd414388886a1f1073b0c0
2014-02-12 12:00:50 -05:00
Dan Prince
a1b469b10b Support adding DHCP interfaces one at a time.
Refactors dhcp-all-interfaces.sh so that if an optional
INTERFACE argument (the first argument) is passed to the script
it only inspects that single interface. If no argument is
passed then the previous default behaviour use used which
causes all interfaces to be inspected.

Also sets PATH so that the commands within the script
can all be found if it isn't set properly (/sbin/ip, /bin/cat, etc.)

This is a move towards using udev rules to add these types
of interfaces automatically.

Change-Id: Ia482c1d3ddce0f0d8d77f9bc3ac76d6924640715
2014-02-12 11:57:56 -05:00
Jenkins
eab085af02 Merge "ifquery doesn't exist on Fedora" 2014-02-12 03:11:43 +00:00
Jenkins
983f918804 Merge "Make the MIRROR_TARGET directory if it didn't exist" 2014-02-11 14:38:10 +00:00
Jenkins
0a678da6b6 Merge "Remove unneeded service mapping for mysql on openSUSE" 2014-02-11 14:17:37 +00:00
James Slagle
f760f11050 Don't hardcode environment.d
Don't assume environment.d is always under /tmp/in_target.d. This makes
dib-run-parts more flexible, and less specific to diskimage-builder. For
instance, dib-run-parts is already used by os-refresh-config, but
/tmp/in_target.d/environment.d is never going to exist when it's called
by os-refresh-config.

This is useful b/c elements can now install os-refresh-config scripts
under /opt/stack/os-config-refresh/environment.d to influence how
further scripts are run. Specifically, the OpenSuse packages prepend the
OpenStack user accounts with "openstack-". Instead of checking for which
users exists with an if/else in all the os-refresh-config scripts that
need to do user related things, we can just update the scripts to use
$NOVA_USER, etc. The OpenSuse element can then install a file under
/opt/stack/os-config-refresh/environment.d to set NOVA_USER to
openstack-nova. We could have files that declare defaults as well within
each specific element, or just set a default variable at the top of each
os-refresh-config script.

Change-Id: Iadbfad995da657e2965fd55fc4ba3a88138b4cfc
2014-02-11 08:33:45 -05:00
Jenkins
014e3bf875 Merge "Adjust neutron package install for Fedora" 2014-02-11 09:39:54 +00:00
Jenkins
e1d58046f0 Merge "Add lsof package to all Fedora images" 2014-02-10 12:32:05 +00:00
Jenkins
9295162946 Merge "Add more package name mappings for openSUSE." 2014-02-10 10:49:19 +00:00
Dirk Mueller
af3cf11b27 Add more package name mappings for openSUSE.
Adjust for naming difference on openSUSE.

Change-Id: Ieda36c14b94add4d9877a25224b15563c648d122
2014-02-07 22:56:12 +01:00
Dan Prince
72ca729c67 Add lsof package to all Fedora images
Really useful when debugging.

Change-Id: I0862ad2b97f5fa061aea592f4a267856ac124f65
2014-02-07 16:29:37 -05:00
Dan Prince
01e25c378b ifquery doesn't exist on Fedora
Updates dhcp-all-interfaces so that we avoid using
ifquery on Fedora to determine if an interface exists.

Adds a new config_exists function which simply looks for
the ifcfg config file on distro's that use netscripts.

Change-Id: Ie55524b83820fe4fab28cc9d3f7e08ca2e42c182
2014-02-07 13:33:35 -05:00
Ralf Haferkamp
b3ab436a62 Remove unneeded service mapping for mysql on openSUSE
The service is called mysql on openSUSE and that's what the mysql element uses.
Change-Id: I73cb6a1c06c3096aa8a9f515fd79014e5b2962d2
2014-02-07 11:15:09 +01:00
Shane Wang
4aeee175d8 Fix misspellings in diskimage-builder
Fix misspellings detected by:
* pip install misspellings
* git ls-files | grep -v locale | misspellings -f -

Change-Id: I42acbcce7611ff3e2e120168acccc518299c2c42
Closes-Bug: #1257295
2014-02-07 14:42:45 +08:00
Dirk Mueller
abc3bcd2f3 Adjust neutron package install for Fedora
With https://review.openstack.org/#/c/70721 installing
the dhcp-agent subpackage was introduced, which is needed
for openSUSE. It seems on Fedora this is part of the main
package, so we need to map it back to that one.

Change-Id: I80f8412d291b3571d220fd26cfece86a03479440
2014-02-06 23:11:56 +01:00
Derek Higgins
c2ccfa6504 Make the MIRROR_TARGET directory if it didn't exist
Just adding an element shouldn't make a disk image build crash, even if
we're not using the element for its intended purpose.

Change-Id: I2ec91ce4975b3b0deee7c85bb223f854c694553f
2014-02-06 14:10:42 +00:00
Jenkins
5964fec479 Merge "Update openSUSE package mappings for OpenStack" 2014-02-06 12:45:41 +00:00
Ghe Rivero
03a7c4a117 Add tgt service mapping for Fedora
tgt service is called tgtd in Fedora
(https://review.openstack.org/#/c/71466)

Change-Id: Ic345b4a228cd665063a6753d05b3a7e29796039f
2014-02-06 08:10:45 +00:00
Jenkins
22cc3900a5 Merge "Add mysql mapping for Fedora" 2014-02-05 22:50:21 +00:00
Dirk Mueller
f76873fd61 Update openSUSE package mappings for OpenStack
There are some subtle naming differences between
Fedora and openSUSE, adjust.

Change-Id: I286e656ad980443abe0d578e9390c2a989cfdee7
2014-02-05 20:32:27 +01:00
Jenkins
79ab588592 Merge "Update map-services for Fedora" 2014-02-03 14:19:20 +00:00
James Slagle
e7118cec1d Update map-services for Fedora
Additional service names added to map-services for cinder and swift.

Change-Id: I0c4afc505ccb80d7e2e397614792d74b3294264a
2014-01-30 20:22:07 -05:00
Ana Krivokapic
609746f1f3 Add Fedora packages mappings for snmpd
Change-Id: I61ee59ee6e9ffc8391dbdd3e459df4f0cb369801
Closes-Bug: #1272234
2014-01-30 18:18:07 +01:00
Jenkins
97c7e8fd5e Merge "Add ability to use local cloud image" 2014-01-30 15:51:45 +00:00
Jenkins
5658b85d8a Merge "Add package map for stunnel4" 2014-01-30 10:23:15 +00:00
Ralf Haferkamp
28e0d2dd98 Add mysql mapping for Fedora
To be able to use the os-svc-* helpers for mysql. (See:
https://review.openstack.org/#/c/68278/)

Change-Id: I0af105f6d4bc02a165c5004859c1823f508d43ec
2014-01-30 11:18:24 +01:00
Jenkins
f43f0eea85 Merge "Skip relabel unless SELinux is enforcing" 2014-01-30 08:03:01 +00:00
James Slagle
dfde5c434b Add package map for stunnel4
stunnel4 is called stunnel on Fedora/RHEL.

Change-Id: Ic27692e485792c87a9e54c5baa6b7a75410e8cf2
2014-01-28 19:47:32 -05:00
James Slagle
15960f01cd Add ability to use local cloud image
Adds the ability to set $DIB_LOCAL_IMAGE to use as the base cloud image
for a Fedora image build. There are many repetitve tasks that are done
every image build. With this change you can build an image with the
fedora and vm element and then reuse the resulting image as input to
future image builds. This greatly reduces future image build times.

For instance 99-up-to-date is already taking almost 2 minutes (even with
rpm downloads already cached) for Fedora 20, and is only going to keep
taking longer. By having a local up to date Fedora cloud image, this
time can be saved on each image build.

There is one minor change to support this in that /tmp/grub needs to get
cleaned up at the end of image builds so that the image can be reused.
Plus, there is no reason for it to stick around anyway.  (didn't think
this was worth a seperate commit).

Change-Id: Ic74d138da922ecc99c38c27f105170d90009a84a
2014-01-28 11:01:30 -05:00
Jenkins
fb5a25f379 Merge "Fix tftp mapping on openSUSE" 2014-01-27 12:22:29 +00:00
Jenkins
1420386541 Merge "Add bash as a dependency to the deploy ramdisk" 2014-01-27 10:26:44 +00:00
Jenkins
d2a0098d01 Merge "Use /usr/bin/env, not /bin/env" 2014-01-27 10:15:34 +00:00
Dirk Mueller
d8a045c813 Add a service mapping for openSUSE
Change-Id: I90fcb217e39ac5ef9d5e821a9dd59a1283419843
2014-01-27 10:17:00 +01:00
Jenkins
82080313db Merge "Mount root filesystem readonly during boot" 2014-01-27 08:56:10 +00:00
Ralf Haferkamp
cb26d4478c Add bash as a dependency to the deploy ramdisk
Otherwise some dependend libraries might be missing in the ramdisk. E.g.
on SUSE bash is dynamically linked against libreadline.

Change-Id: I8b36bee6aa5a1e1da419e748e8bba046c29f3ae2
2014-01-27 09:32:24 +01:00
Jenkins
b948d4a23f Merge "Fix ramdisk element for openSUSE" 2014-01-27 02:45:35 +00:00
Dirk Mueller
6ecc8ab766 Fix tftp mapping on openSUSE
the atftp package is not compatible with what TripleO expects

Change-Id: I3309de2bb445bba1ab5ed000e87583387b519f2d
2014-01-24 17:46:10 +01:00
Dirk Mueller
0c2c2ab2d9 Use /usr/bin/env, not /bin/env
/usr/bin/env is more portable (/bin/env does not exist on SUSE).
Also address minor nit on review about python 3.x compatibility.

Change-Id: I6bc1a579bb43664f4d2be35a96f45189a6df0e27
2014-01-24 15:55:39 +01:00
Jenkins
fac607e8d8 Merge "Rename generate-interfaces-file.sh..." 2014-01-23 21:58:31 +00:00
Jenkins
6a50d3cdd6 Merge "Only configure DHCP for real interfaces" 2014-01-23 21:55:13 +00:00
Jenkins
8810011028 Merge "Add map-services" 2014-01-23 21:49:53 +00:00
Dan Prince
5da1f1324b Rename generate-interfaces-file.sh...
Rename generate-interfaces-file.sh to dhcp-all-interfaces.sh
so that it is easier to find/remember in the $PATH on a deployed
image.

Also, on some distros the script actually generates many files
so the previous name was a bit confusing.

Change-Id: I7152fa7c28e8ade251311da2cd5f75972423b66c
2014-01-23 16:38:41 -05:00
Dan Prince
f58f859303 Only configure DHCP for real interfaces
Updates the generate-interfaces-file.sh script in
the dhcp-all-interfaces element so that we only
add interfaces that have real MAC addresses.

The generate-interfaces-file.sh script is run early enough
in the boot process (before OVS is initialized) that this
isn't usually a problem unless you execute it manually by hand
after booting. Then you'll end up with network/DHCP
configs for all of your OVS bridges, etc. This
patch avoids configuring all of the virtual interfaces
which have generated MAC addresses.

Change-Id: I7a705084aa5b11305ac0ec5ca37fd2e87a2ae8b7
Closes-bug: 1239479
2014-01-23 16:37:44 -05:00
James Slagle
c53d58dd11 Add map-services
map-services is used by os-svc-restart from Change-Id:
I15eb2a5db4b0a08e1fb40bda640cd8f224939a92

map-services is not only distro specific, it also needs to be able to
handle different service names for a given distro based on source vs
package installs, since the service names may be different. This is done
via a simple check if the requested service is defined, if it is, we
print the requested name, otherwise, we print the mapped name.

Currently only implemented for Fedora. Support for other distros is
easily implemented just like map-packages.

Change-Id: Ie7b2dcfa7cabd887d5c212df110d90f9d00a7f65
2014-01-23 15:32:27 -05:00
James Slagle
ea257c96d9 Skip relabel unless SELinux is enforcing
The SELinux relabel of the filesystem is taking almost 2 minutes and
isn't needed unless you actually plan to run with SELinux enforcing.
Plus, it appears to "leak" out of the chroot, referencing filesystems on
partitions that aren't even mounted in the chroot.

Note you just can't use getenforce or selinuxenabled here to get the
state of SELinux because those commands are not accurate inside a
chroot.

TBH, a downside of this is that if someone goes to try to enable SELinux
in an image where it was built with it not enabled, the file contexts
are going to be wrong. So they'd need to relabel themselves at that
point. However, this saves me quite a bit of time during image builds,
so I thought I'd submit to get other folks opinion on it.

Change-Id: I2132060d573fc93cf974f3560fdc651ff8ba38b4
2014-01-23 15:29:29 -05:00
Jenkins
1e04079d4f Merge "Fix kernel/initrd extraction for SUSE based distros" 2014-01-23 17:26:01 +00:00
Derek Higgins
2c8d4b2271 Mount root filesystem readonly during boot
If creating an image of a disto without grub2 extlinux is instead used,
extlinux was being configured to mount the root filesystem as
read/write which in turn causes e2fsck to fail to test the filesystem,
on RHEL 6.5 this causes the boot process to fail.

Change-Id: I07859e0df9a2bc1ddd8c5f2ed4509ba14312e140
2014-01-23 16:03:03 +00:00
Ralf Haferkamp
de2becc9ff Fix kernel/initrd extraction for SUSE based distros
Change-Id: Ib4c8ddfe3592242ce684676e9dcb6428466ca66e
2014-01-23 15:30:52 +01:00
Ralf Haferkamp
5fb556f44d Fix ramdisk element for openSUSE
The openSUSE base image does not contain /sbin/dhclient-script, install it.

Change-Id: Ib0db15139930d42bd60213437ef63d7915d8134a
2014-01-23 14:26:11 +01:00
Dirk Mueller
b12c28dde8 Workaround broken udev update on openSUSE
Delete stale symlinks created by current openSUSE 13.1
for kpartx loopback partitions. Those break bootloader
installation otherwise.

Change-Id: I13a7387e341b4426b459610fd1d3c49ce5730f46
2014-01-23 08:34:22 +01:00
Jenkins
e882bb5836 Merge "Setup ccache symlinks on openSUSE" 2014-01-22 21:03:15 +00:00
Jenkins
35aa61617a Merge "Add support for Red Hat Satellite" 2014-01-22 14:02:32 +00:00
Jenkins
f67a3805e2 Merge "Support list of Red Hat channels and repos" 2014-01-22 14:02:31 +00:00
Jenkins
0a5b5159ad Merge "Add support Red Hat Network (RHN)" 2014-01-22 14:02:30 +00:00
Jenkins
f1704ed421 Merge "Update default RHEL guest image" 2014-01-22 11:37:19 +00:00
Clint Byrum
0c6b346c15 Add Copyright and License header to debian element
These were forgotten in the initial commit.

Change-Id: I42ef745cc56d0a6983b1cc9c870d5a81d1f61940
2014-01-21 15:57:43 -08:00
Ralf Haferkamp
831cd69790 Setup ccache symlinks on openSUSE
openSUSE's ccache package doesn't include symlinks to enable ccache
for gcc by default, so we set them up manually.

Change-Id: I79dbe814ff2dfad8158fb1f06d68cc07eae19a41
2014-01-21 22:01:17 +01:00
aweiteka
bb04ff1ef2 Add support for Red Hat Satellite
A Red Hat Satellite URL may be provided as the source for
Red Hat packages. A Satellite key may be provided for a pre-
defined set of RHN channels to be automatically applied. Support
for installation of a Satellite certificate also provided.

Change-Id: Iae5b97d6f4732a28efa08ab778a56c2f9b6b6bd9
2014-01-21 14:26:35 -05:00
aweiteka
ce14214a7a Support list of Red Hat channels and repos
When registering to RHN or RHSM a list of space-separated
channels or repositories may be provided. This allows control
over which packages are available to the system during build.
Note, the optional channel or repository is added by default
since diskimage-builder requires access to its packages. This
list is for additional channels or repos.

Change-Id: I9cf90d47ab1d09f5491f574132410438f9e294e3
2014-01-21 14:26:35 -05:00
aweiteka
78dbbe68fe Add support Red Hat Network (RHN)
This provides basic support to register to the Red Hat Network
(RHN Classic) for packages. The image is unregistered from RHN
before completing image build. The rhnsd service is prevented
from starting since it is not needed for packagage install.

Change-Id: I2dc124cdc6ae59a63fd94d82d77108acccc3f668
2014-01-21 14:26:35 -05:00
aweiteka
3dc40c925d Update default RHEL guest image
Red Hat periodically updates the qcow2 guest image available
for download. This sets the default image name to the latest.

Change-Id: Iba3075bbee3b41918d5cd3da9721fcbf98ff3bcd
2014-01-21 14:26:35 -05:00
Steve Kowalik
58c755cf4c Create a new baremetal element
Rather than using a script to mount the image using nbd to extract the
kernel and ramdisk, make a new element called baremetal, which contains
a cleanup.d script that will copy them out to <image name>.{vmlinuz,initrd}.

Closes-Bug: 1224669
Change-Id: I8f3569aa12148d18b1c8242b6fbbd8857894b26f
2014-01-21 16:48:20 +11:00
Jenkins
ed8264c9d9 Merge "Update to Fedora 20" 2014-01-20 21:30:11 +00:00
Jenkins
6397fd3b5f Merge "Add debian-upstart for experimenting with upstart" 2014-01-20 21:27:24 +00:00
Jenkins
037e798625 Merge "Fix mysql package mappings for opensuse" 2014-01-20 21:27:01 +00:00
Jenkins
ca2a430375 Merge "Allow adding packages to debootstrap" 2014-01-20 21:26:51 +00:00
Jenkins
c21c82bacf Merge "Convert -dev to -devel in fedora/map-packages" 2014-01-20 21:26:16 +00:00
Jenkins
cf7af05fff Merge "Fix the curl command in the ironic-deploy element" 2014-01-20 18:58:54 +00:00
Jenkins
92ab5ab5af Merge "fedora/RHEL: use env from /usr/bin" 2014-01-20 18:54:26 +00:00
Ralf Haferkamp
2e2b19fd4f Fix mysql package mappings for opensuse
Change-Id: I231df61a79e6c8686afbbd203fa6bba96497f228
2014-01-20 18:52:22 +01:00
Dirk Mueller
2def5b7727 fedora/RHEL: use env from /usr/bin
/usr/bin/env is the new canonical location of "env", use
it.

Change-Id: Ifc5fbd942d409b1f54057579c5f57ccac63d4b72
2014-01-20 16:16:41 +01:00
Jenkins
1831a2bfbd Merge "Allow use of mirrors when building Ubuntu images" 2014-01-20 11:35:31 +00:00
Steve Kowalik
48f5359efb Allow use of mirrors when building Ubuntu images
Add a script to ubuntu's pre-install.d that will change
archive.ubuntu.com to a mirror specified in $DIB_DISTRIBUTION_MIRROR.

Change-Id: Idfd5a40ec8a1912a5de5dcdcaf9795946b0b53b8
2014-01-20 15:07:04 +11:00
James Slagle
82668aacc0 Update to Fedora 20
Update the default version of Fedora to 20 for the cloud image. A few of
us have run through devtest using Fedora 20 and it's working.

To continue to use 19, just specify DIB_RELEASE=19 for your image
builds.

Change-Id: I3d48272dedcd424747db5485ae0d8e9953f04191
2014-01-17 11:12:58 -05:00
Angus Salkeld
a021b110ad Convert -dev to -devel in fedora/map-packages
If there is not a special case in package_map then
convert all *-dev packages into *-devel

Change-Id: I411935c1f3221016b56263ac9502221cd75d130d
2014-01-17 18:10:13 +10:00
Lucas Alvares Gomes
af8eeebeb8 Fix the curl command in the ironic-deploy element
We need to use the eval command together with curl so that the
$TOKEN_HEADER variable is seems by curl as two arguments: "-H" and
"'X-Auth-Token: ...". Without the eval bash would interpret the variable
as only one argument "-H 'X-Auth-Token: ...'" making the curl command
to not understand that parameter and fail to pass the auth_token to the
Ironic API.

Change-Id: I4dcfc323d6ab9b7fa207328386ef65a146a93617
2014-01-16 13:16:47 +00:00
Jenkins
016fc76d84 Merge "Symlink correct element install type" 2014-01-16 11:56:23 +00:00
Jenkins
5108838071 Merge "dib-run-parts should dereference symlinks" 2014-01-16 11:56:17 +00:00
Jenkins
71fb3e0f3c Merge "Wait for tgtd socket to be available" 2014-01-16 00:03:12 +00:00
Jenkins
e152e27937 Merge "Implement serial-console for systemd." 2014-01-16 00:03:11 +00:00
James Slagle
57ba300be4 Symlink correct element install type
The source-repositories element will now symlink the install scripts
corresponding to the set install type for an element into the install.d
directory.

Different install types are implemented by elements by writing scripts to do
the install type under install.d/<element-name>-<install-type>-install
directories.

For example, the nova element would provide:

nova/install.d/nova-package-install/74-nova
nova/install.d/nova-source-install/74-nova

source-repositories will create the following symlink for the package
install type:

$TMP_HOOKS_PATH/install.d/74-nova -> nova-package-install/74-nova

Or, for the source install type:

$TMP_HOOKS_PATH/install.d/74-nova -> nova-source-install/74-nova

Change-Id: I1bfaf39e5a98b2af904fbc6d674dbada30b27ed9
2014-01-15 14:57:08 -05:00
James Slagle
79f31d0476 dib-run-parts should dereference symlinks
dib-run-parts should dereference symlinks, and if they point to an
executable file, execute that file as a script. This can be accomplished
by using the xtype predicate in the find command instead of the type
predicate.

This change is needed so that we can dynamically symlink hook scripts
into hook directories such as install.d at runtime to support different
install types.

Change-Id: I933e7f4b4dcf16956841d8c14aa63b0f9a18fc5d
2014-01-15 14:57:08 -05:00
Clint Byrum
611a8b701a Add debian-upstart for experimenting with upstart
Upstart support in Debian is somewhat experimental, but having it would
be especially useful for TripleO which already generates upstart scripts
which will work fairly well on Debian based systems.

Change-Id: I267da5ae7bc4a484fac152a14963d96ca2ad2693
2014-01-14 22:41:25 -08:00
Clint Byrum
4ce573c582 Allow adding packages to debootstrap
Also allows setting the "distro name" so that things that change the
debootstrap signature will have a different tarball.

This is being done in advance of a debian-upstart element to allow
experimenting with debian and upstart.

Change-Id: I1a558fdaf5924c65f1cced9f9bf29d73643c30af
2014-01-14 22:41:25 -08:00
Clint Byrum
d0fd8915f6 Add support for Debian
Change-Id: Ibc6bc57dea1eab295fb23cccf40b38f3fcb285c3
2014-01-14 22:41:16 -08:00
James Slagle
8d910d10f8 Wait for tgtd socket to be available
tgtd returns execution control and backgrounds itself almost immediately
and before it has made it's listening socket available. This can cause a
race condition as the tgtd socket is not available when tgtadm is run,
resulting in an error:

failed to send request hdr to tgt daemon

Add a function to check if the socket is available before moving on to
calling tgtadm, and a wait_for helper function we can use.  We'll
check for the socket every 0.5 seconds, for up to 5 seconds.

I'm seeing this issue on almost every deploy using a ramdisk built from
Fedora 20. I'm not sure if something has changed in tgtd, but this
behavior is documented since Fedora 18 at least. In the systemd script
for tgtd, there is actually "sleep 5" to work around the problem.

See Also: https://bugzilla.redhat.com/show_bug.cgi?id=848942

Change-Id: Iffa9fc63393309ca653d592dff17316ecbea3e09
2014-01-14 12:53:03 -05:00
Jenkins
de7deb7ce9 Merge "Add new modprobe-blacklist element." 2014-01-14 13:05:55 +00:00
Jenkins
d9303ac9af Merge "Add package support to source-repositories" 2014-01-14 12:42:29 +00:00
Jenkins
85bb142641 Merge "Retry link check up to 10 times." 2014-01-14 11:12:25 +00:00
Jenkins
cccb870d92 Merge "Removing the config-applier element" 2014-01-14 11:09:14 +00:00
Dan Prince
ab9d5d3cc3 Add new modprobe-blacklist element.
This element can be used to provide a custom list
of modules via DIB_MODPROBE_BLACKLIST which
will be disabled via modprobe.d/blacklist.conf.

I'm using this to disable network adapters on the
Red Hat TripleO rack where the devices can't be
explicitly disabled via the BIOS.

Change-Id: I8a0a8ee05fa62628434d7f6422577dbf5cdd7a2e
2014-01-13 15:43:29 -05:00
Dan Prince
5ae9c5d24c Install traceroute on Fedora.
Useful for debugging, etc.

Change-Id: Ia06f8d7a76de2f082bea09c57fbb8150c9706afc
2014-01-13 15:12:39 -05:00
Dan Prince
ae28c99b5f Retry link check up to 10 times.
Updates the dhcp-all-interfaces element so that the link
check tries up to 10 times.

On some of my machines 3 times (seconds) doesn't appear to
be quite long enough...

Change-Id: Ibf7015162fc890d4de8a417b868b0301146944c0
2014-01-13 13:58:42 -05:00
Dan Prince
ae22d39472 Implement serial-console for systemd.
Update install.d/20-stty so that we try to start
a getty on ttyS0 and ttyS1 when using systemd.

Change-Id: Id5b605c74bf82855af0af4d0f95659cdd3a7b3a0
2014-01-13 13:02:54 -05:00
Derek Higgins
63a4c1e9d5 Removing the config-applier element
This is pretty much a duplicate of t-i-e/elements/os-apply-config, isn't used
in tripleo and the upstart script is out of date. We don't need to carry this.

Change-Id: I2b23d5930afae42cff28e2d5ce3d6c84224afa4a
2014-01-13 14:10:10 +00:00
James Slagle
d253ed16d8 Add package support to source-repositories
Adds a new "package" type to source-repositories. When the package type
is specified in an origin-repository-* file the package name and type
"package" need to be specified like so:

nova package

The existing map-packages mechanism can be used to map the specified
package name to the actual distribution package names.  The
pre-configured package repositories on the image will be used to install
the packages during the install.d phase of the image build.

If the repo type is package, an environment.d hook script is updated
with an environment variable which is sourced by dib-run-parts. This is
so that other hooks, particularly in install.d, will know what repo type
was used and can make the correct decision about doing a source install
or not.

Change-Id: Ief4e524664cd32d2087aeb3b1766f05e017de91e
2014-01-13 08:23:53 -05:00
James Slagle
380df31aac Remove old versions of grub2 from the yum cache
Before downloading and caching the latest version of grub2, we need to
first remove all old versions. Otherwise, the find command that writes
the grub2 version to /tmp/grub/install to install during the finalise
phase could pick one of the older versions of grub2 in the cache. This
will cause the install during the finalise phase to fail since the older
version of grub2 will conflict with the newer installed version of
grub2-tools.

Change-Id: I5b43e359db3ba81ed6283dfb41c7e89629516d38
2014-01-10 17:12:13 -05:00
Jenkins
843214246c Merge "Install tcpdump on Fedora" 2014-01-10 17:28:28 +00:00
Dan Prince
7511faa2b1 Implement dhcp-all-interfaces for systemd.
Includes:

 * Enhancements to the generate-interfaces-file.sh script
 so that it generates ifcfg network-scripts for
 distros like Fedora/RHEL.

 * Includes a new dhcp-all-interfaces.service systemd
 script which ensures network interfaces get generated
 before the network service starts on Fedora.

 * Add a new disable_interface function to
 generate-interfaces-file.sh which (on Fedora)
 will delete the ifcfg config for an interface which
 isn't plugged in.
 This is important because ifcfg-eth0 exists by default
 and we want to remove it if the NIC isn't connected.
 Previous behaviour on Ubuntu is unchanged and the
 generate-interfaces-file.sh just logs it as skipped.

 * General doc updates...

I tested this on a multi-nic machine w/ Fedora where the first NIC
exists but was not connected.

Change-Id: Ia99e312539da43caefd72aa60398d43dac5dcc8f
Closes-Bug: 1239880
2014-01-10 10:14:07 -05:00
Derek Higgins
864e209bfa Install tcpdump on Fedora
Tcpdump has a fairly small footprint and should be installed to all
images (it currently is on the ubuntu image by default). Sometimes the
reason you need it is also the reason its to late to get it.

Change-Id: I4849157515b389d534bdee755cdac6b0768fcb29
2014-01-10 14:16:30 +00:00
James Slagle
f5d4d8c711 Mark install-packages +x.
install-packages is an executable script, it should be +x.  The install
command that was installing it is setting permissions to 755, so we were
fine there.  However, I was trying to test the script outside of an
image build to test a change, and it's helpful to have it +x in the
repository for that context like all the other scripts under bin.

Change-Id: Ic300cc56d463640d119db4e310d375c66ed133ca
2014-01-07 07:54:47 -05:00
Jan Provaznik
c9cceeeb40 Make sure a loop device exists before kpartx is called
If "kpartx -av" is called and there is no unused /dev/loop* device,
then kpartx fails. Calling "losetup -f" finds first unused device
and creates it if it doesn't exist.

Fixes bug #1195264

Change-Id: I5d59b519fd8e2a7380b71a788f9d3f2331b6567f
2014-01-06 11:30:51 +01:00
Jenkins
e85bab58ec Merge "Fixed command dib-init-system not found error" 2014-01-03 10:11:25 +00:00
JUN JIE NAN
3968e5bd2c Fixed command dib-init-system not found error
dib-init-system is installed in 04-dib-init-system. Before it is
installed, 10-dib-init-system.bash may be sourced so command not found
issue found.

The patch invokes "/tmp/target.d/dib-init-system" directly in
10-dib-init-system.bash.

Change-Id: I9843224497c7dab8392261b26d62b6d79d1795ee
2014-01-03 17:42:21 +08:00
Tomas Sedovic
f176a78486 Add a package mapping for libvirt-dev
This is a dependency for the libvirt-python Python package that
ceilometer-agent-compute uses. libvirt-python is already installed globally in
the system via apt/yum, but that's not available to ceilometer-agent-compute's
virtualenv.

Change-Id: I07a12226889c6cda3bd04021240ee9714a396522
2014-01-02 12:01:03 +01:00
Chris Jones
4794df890a Log unsupported source repository types.
If we don't recognise git/tar/etc as source-repositories repotypes, we
now log the repotype, for debugging purposes.

Change-Id: I7b870807658441c928c7880291eb0284b6f97778
2013-12-20 14:53:53 +00:00
Chris Jones
4c7a3bf79f Improve source-repositories git caching.
We now unconditionally cache the repositories being fetched with
source-repositories.

Additionally, by improving the way we handle repository refs, we are now
able to build images with code taken directly from gerrit reviews.

Change-Id: Ifd4c5691f6761eb5551663e6d0aa1c0e42afced3
2013-12-20 14:48:30 +00:00
Jenkins
c1eb44c9b2 Merge "Add element to modify /etc/apt/sources.list in dib" 2013-12-19 11:01:13 +00:00
Kui Shi
5daa989fb8 Add element to modify /etc/apt/sources.list in dib
Add apt-sources element for Ubuntu OS to override the default
/etc/apt/sources.list in cloudimg.

Set DIB_APT_SOURCES with a proper sources.list file to replace
the default one.

If you want to use this element in tripleo project, set NODE_DIST or
EXTRA_ELEMENTS / UNDERCLOUD_DIB_EXTRA_ARGS / OVERCLOUD_DIB_EXTRA_ARGS
to make it take effect at build time and run time.

e.g. before running devtest.sh:
export DIB_APT_SOURCES=/etc/apt/sources.list
export NODE_DIST="ubuntu apt-sources"

Change-Id: I9ce0d03b506c2948b96382e4d6e85f0aff906450
2013-12-18 06:01:36 -08:00
Ralf Haferkamp
2da3235b82 Fix package-mappings for openSUSE
Change-Id: I6b59d62a9b3df89d8eef7aa2d7eb0b65308428d0
2013-12-18 14:23:33 +01:00
Ralf Haferkamp
641840b2c7 Update login.defs on openSUSE
Some of the tripleo image elements rely useradd a for creating a group
with the same name as the user that is added. On openSUSE is feature
is turned off by default. So enable it here.

Change-Id: I120067e5e522dc6e704e7f09b724d60dee0c7f22
2013-12-18 10:19:08 +01:00
Ralf Haferkamp
7da1425438 Merge force-text-mode-console into the bootloader script
52-force-text-mode-console, running after 51-bootloader, calls
grub2-mkconfig (on Ubuntu that will happen via the update-grub call).
This call will overwrite all the changes that 51-bootloader did to the
grub.cfg. By merging the 52-force-text-mode-console code into
51-bootloader we avoid that. This also means that 51-bootloader will now
call grub2-mkconfig regardless of whether there is already and existing
grub2 configuration or not (52-force-text-mode-console did that in the
past anyway).

Additionally this commit enables the force to textmode code for openSUSE
(was previously only working correctly for Ubuntu and Fedora).

Closes-Bug: #1260323
Change-Id: Ida8e1a69df22d41ad70eb7154707c50613b536a8
2013-12-16 13:58:42 +01:00
Jenkins
b89cf37104 Merge "Add zypper element" 2013-12-11 20:45:54 +00:00
Ralf Haferkamp
f8a0715c56 Add zypper element
This provides some customizations for zypper based distributions (e.g.
openSUSE and SLES). It is heavily inspired by the yum element and sets up
zypper to cache downloaded package outside of the chroot so that they can be
reused to speed up subsequent builds.

Change-Id: I775c921ee19cf7d1180fb68c1b7857ea6245a77d
2013-12-11 17:09:24 +01:00
Ralf Haferkamp
c719e5b4ee opensuse: Update README.md
Remove the comment about the rpm-distro element. The only tiny bit we copied
from it is 01-install-bin. 01-override-zypp-arch was unneeded and has been
removed from the opensuse element before the initial merge.

Change-Id: I9886bb6e7d723690f2d2ac476d6d3aca5a367607
2013-12-11 15:17:08 +01:00
Jenkins
ca667d0a33 Merge "local-config: Configure proxy for zypper repos" 2013-12-11 09:50:54 +00:00
Ralf Haferkamp
ba5aa8725e Fix $TARGET_ROOT usage in yum element
Fix the test for $TARGET_ROOT in root.d/50-yum-cache.

Don't use $TARGET_ROOT in pre-install, it's only set for "root" and "cleanup".

Change-Id: I048364ea08ef503a4466f3494f18f72ebf99b5c2
2013-12-10 17:56:42 +01:00
Ralf Haferkamp
f6c2901b0a local-config: Configure proxy for zypper repos
This allows using the local-config element with openSUSE based images.

Closes-Bug: #1256982
Change-Id: I670c938a44095ea7e7c31b81194acc1555212f06
2013-12-10 15:54:59 +01:00
Tim Serong
54c8508d45 Add support for building openSUSE images
This is currently experimental/incomplete.  See
elements/opensuse/README.md for details.

Change-Id: Ie5bcab15c9703cf15ceb642fb986a8afc34f96fb
Co-Authored-By: Ralf Haferkamp <rhafer@suse.de>
2013-12-05 15:18:46 +01:00
Jenkins
e8e7f45fb2 Merge "Log the repository fetches in source-repositories" 2013-12-05 13:06:17 +00:00
Mark McLoughlin
10eac21043 Log the repository fetches in source-repositories
Make the source-repositories element log which repo it's cloning from
so it's easier to tell what's going on.

Change-Id: I224f7bd98bc3d0ba94313dae9063d4ba4e25bdaa
2013-12-05 06:28:13 +00:00
Jenkins
0e536ba062 Merge "Default name for ramdisks to image." 2013-12-04 23:37:24 +00:00
James Slagle
6b220a044f Default name for ramdisks to image.
If you build a ramdisk with ramdisk-image-create and forget to specify
-o, you end up with a .kernel and .initramfs file as the image outputs.
This commit defaults the $IMAGE_NAME value to image so you'd end up
with image.kernel and image.initramfs

Change-Id: I10f3ac9ad178c32119523e4da930070e3ade4f73
2013-12-04 15:31:46 -05:00
Yuriy Zveryanskyy
bf222fe7bc Add deploy ramdisk element for Ironic
Ironic requires callback from deploy ramdisk via API with json data.
Auth token also required if Keystone auth enabled for Ironic API.
This patch adds separate element 'deploy-ironic' for Ironic deploy
ramdisk creation.

Change-Id: I5ef96711763e50307cfa17c406a9e16bffb937bc
2013-12-04 17:57:44 +02:00
JUN JIE NAN
d7e43c0861 Modify /etc/selinux/config if it exists
When `/etc/selinux/config' does not exist, although selinux has
already been disabled, the image creation will fail.

Change-Id: I9e4a9a006073fd3f708049407ef98f82c3f399d1
2013-12-03 15:35:49 +08:00
JUN JIE NAN
08797d44b5 Remove dot after TMP_HOOKS_PATH
"$TMP_HOOKS_PATH." is always be nonzero so we should remove the dot.

Change-Id: I0432bb3e21cc29806e1e2ae245dd4b6301ee9ba6
2013-12-02 12:44:11 +08:00
Jenkins
8a38b14ecf Merge "Switch Ubuntu element to installing saucy." 2013-11-27 13:30:24 +00:00
Chris Jones
a84b2763fc Switch Ubuntu element to installing saucy.
In tests, saucy has been able to match raring for the ability to bring
up clouds, so we can roll forwards and switch the default to saucy.

Change-Id: I4df35d40f902c0daa7b2761d7afea8db25526753
2013-11-27 10:10:39 +00:00
JUN JIE NAN
afc07e2b2d Enhance dib-run-parts usage message
and fix $1: unbound variable issue when run dib-run-parts --list.

Change-Id: Id18e21683f785f318bce11d401cb4f826bed1b01
2013-11-27 17:42:37 +08:00
James Slagle
476ec8feec Add mapping for tgt to RHEL element.
tgt maps to scsi-target-utils on RHEL.  This mapping already exists for
the Fedora element, so no need to add it there.

Change-Id: Ifc9a8976299458cdef50c2fbe00ba6a0c8d22f00
Closes-Bug: #1252975
2013-11-26 15:21:21 -05:00
James Slagle
f5d5667b32 Update openssl for Fedora.
We need to update openssl before proceeding as part of the image build
because the version on the Fedora cloud image is no longer compatible
with new python environments installed by virtualenv.

Change-Id: I3ed889f7382e7ca0de052ef5bdd6afeefc8bbdd8
Closes-Bug: #1254879
2013-11-26 09:09:47 -05:00
Mark McLoughlin
4f97b087d5 Fix typo in source-repositories README
Change-Id: Iaf5367099544fc5d4699140af642dfd392c7090f
2013-11-25 10:54:18 +00:00
Chris Jones
09eb230ea3 Detect udevd version and behave accordingly.
We can no longer rely on the name of the udevd binary as a hint for its
version, so instead we query the binary and behave accordingly.

Change-Id: Id4c869dfeeca5ed970169343b6b5f0f2c4499ca3
2013-11-20 11:13:06 +00:00
Jenkins
1373ed5374 Merge "Fix issue with Ubuntu grub pre-install step for ARM" 2013-11-20 09:24:45 +00:00
Ripal Nathuji
8f118af7d3 Fix issue with Ubuntu grub pre-install step for ARM
The Ubuntu build for ARM breaks when the pre-install step attempts to remove
grub components. This change gates those package removals based on whether
they are currently installed.

Change-Id: Ie169dbf12213e69b3713b3b1f9a68ae224f9fd31
2013-11-19 15:26:17 -06:00
iberezovskiy
41caf597a5 Check existence of rhel rpm key
The key 'RPM-GPG-KEY-redhat-release' may not exist in image
e.g. when you use non-default image (for example, if you build CentOS
image using rhel element).

Fixes bug #1252697

Change-Id: Iae642abb7a7f76c5528750dafedd4fd4bda56fe2
2013-11-19 20:02:12 +04:00
Chris Jones
f093d4d38d Fix troubleshooting override.
We can now actually override the $TROUBLESHOOT environment variable in
ramdisks, by not declaring it as readonly. Yay for insufficient
end-to-end testing.

Change-Id: I026c9dbdd411bf3eb945d5b18188eb1eedf3850a
2013-11-19 11:25:58 +00:00
Jenkins
9eb2719998 Merge "Add mapping for gearmand" 2013-11-18 22:06:59 +00:00
Chris Jones
a19d8ceee7 Document ramdisk troubleshooting.
We now document how to trigger a troubleshooting shell when booting a
di-b ramdisk.

Change-Id: Ief44347893fa81a7f2669143e1746a5ee99599af
2013-11-18 19:45:59 +00:00
Jenkins
45a0082fa7 Merge "Add troubleshooting override to deploy ramdisk." 2013-11-18 19:08:45 +00:00
Chris Jones
1f328087d9 Add troubleshooting override to deploy ramdisk.
We now allow operators to drop into a troubleshooting shell while
booting a deploy ramdisk, without having to have pre-configured their
desire to do so ahead of time, by means of a 10 second pause for them to
press a key.

Change-Id: I05ffa1ebaa95c83dee7bd2a2f52ba4c08928bb10
2013-11-18 17:22:26 +00:00
Derek Higgins
257a9dd139 Add mapping for gearmand
The ubuntu gearman-job-server package maps to the Red Hat gearmand
package.

Change-Id: I3ec90b2759498488fce2ecaed03effc18313d1e6
2013-11-18 16:44:14 +00:00
Jenkins
6f7eae51b2 Merge "Split network bringup out of base ramdisk init." 2013-11-18 11:38:43 +00:00
Jenkins
6fb7a22166 Merge "/bin/bash all the things." 2013-11-18 09:36:01 +00:00
Jenkins
670f1f7312 Merge "Don't block the upstart daemon if it doesn't exist" 2013-11-18 09:35:03 +00:00
Victor Lowther
dbacf3e8df Make pxe_mac accurate in two common cases.
If we are booting pxe booting using syslinux, and it has IPAPPEND 2 in
the boot stanza, then it will append the mac address of the device we
are booting from to the kernel parameters where we can get at it
pretty easily.

If we are booting physical hardware via UEFI over the network, we can
rely on the BootCurrent EFI variable to point at the boot entry for
the NIC we booted from, which will include the MAC address of that
nic.

If neither of those cases are in play, we can just fall back to the
all-physical-devices-with-links code.

This currently uses the Bash 4 support for associative arrays to handle
the netboot-in-UEFI case, if needed I can rewrite it to be Bash 3 compatible.

Change-Id: I5e50e30c60d6d732a09ab61251cbb9be08bb6113
2013-11-17 21:43:59 -06:00
Victor Lowther
43b70ce224 /bin/bash all the things.
Since we are using bash syntax in some of the element fragments,
we should make sure we use bash for all of them, so that things don't
break on systems where /bin/sh != /bin/bash.

Change-Id: If2f043c57aa4e1492b7f9839213ef6123f683612
2013-11-17 16:59:31 -06:00
Chris Jones
c9f432d45b Split network bringup out of base ramdisk init.
We now run the network bringup portion of the ramdisk init from an
init.d script, thus allowing image builders to inject code into the init
script before that point (e.g. configure network hardware).
The environment variables used by the network bringup code are retained
in the base init script so they can be overridden by init.d fragments.

Change-Id: I1ef0bb21e7f26c0ff3f02266f853ce5402bcb94d
Closes-Bug: #1252023
2013-11-17 20:37:26 +00:00
Russell Sim
b9348447b9 Don't block the upstart daemon if it doesn't exist
Check if Upstart exists before blocking the daemon.  Not all dpkg based
OS's have Upstart installed so this should help increase compatibility.

Closes-Bug: #1251949
Change-Id: I2dcb1ff3641778b5653ca5762a728398adb57da1
2013-11-17 14:46:14 +11:00
Chris Jones
5387b01af9 Make sorting of ramdisk init elements explicit.
We assemble the init script of ramdisks with the fragments from included
elements, alphabetically.
We now place leading numbers on all of the fragments we ship, to make it
more obvious to element authors and downstream users, how the process
works.

Closes-Bug: #1251706

Change-Id: I56b0d42971c8c462eddcfe1769f8124405e1233c
2013-11-15 17:18:34 +00:00
Roman Podoliaka
b01c73aadc Provide a way of determining init system used
dib-init-system script is installed into $PATH. Called
without arguments it will print the name of init system
used to stdout.

Additionally, set DIB_INIT_SYSTEM environment variable to
the init system used.

Tested on ubuntu+upstart, centos+upstart, fedora+systemd,
debian+sysv.

Closes-Bug: #1251610

Change-Id: I29668079091f6060dab66d8259890384d3bbd653
2013-11-15 17:10:38 +02:00
Derek Higgins
726a816cac Add mapping for libffi-dev
libffi header files are needed to pip install xattr, which is a
requirement of swift.

Change-Id: I69560425f7106c35ec432825eab94ca8424eb89e
2013-11-12 20:32:52 +00:00
Derek Higgins
64f4323ae7 Allow heat-admin to sudo without tty
This is needed to run scripts from devtest, in particular so that
swift ring files can be distributed.

Change-Id: Icbe8c5b24a0d494730357983f230a37a7f078de7
2013-11-12 20:26:11 +00:00
Jan Provaznik
f6f4e808d1 Remove BUSYBOX variable from ramdisk-defaults
ramdisk-defaults file is sourced before chroot to an image being created
is executed. Busybox path should be set inside chroot instead.

Change-Id: Ic0e1d3093ca9d489dd96b14157e93b336cf0baee
2013-11-11 14:36:15 +01:00
iberezovskiy
f23babe971 Check existence of directory 'lost+found'
The command `sudo rmdir $TARGET_ROOT/lost+found` will fail
if `$TARGET_ROOT/lost+found` directory doesn't exist,
e.g. when you use non-default image.

Fixes bug #1245856

Change-Id: I48c8f2f201b29912a726249023ca7d20893cc958
2013-10-29 18:51:49 +04:00
Derek Higgins
d91ececa06 Remove mapping of atftpd to tftpd-server
These are different tftp implementations, tftpd-hpa and tftpd-server are
equivalent, this mapping was added in a previous commit.

Change-Id: Idd1f554d4ee8e1d3c515d5f2f191e90abe0ff396
2013-10-25 12:27:39 +01:00
Jenkins
0e4ddaa586 Merge "Add redhat mapping from tftpd-hpa to tftp-server" 2013-10-25 11:24:20 +00:00
Derek Higgins
4508a685de Remove dependency on dracut-network
Now that we have moved onto neutron-dhcp-agent. dracut-network is no
longer needed, infact it was getting in the way. Dracut wasn't
requesting the classless-static-routes dhcp parameter but was
configuring networking. Removing this causes the network service to
redo dhcp and set the appropriate static route (specifically so
overcloud nodes are routed to the correct metadata server). This element
is still required for undercloud nodes so needs to be included there.

Change-Id: Ic23560164aff7791dd87ed2c3ad008efd248ae93
2013-10-25 09:15:40 +01:00
Derek Higgins
66b293af1c Add redhat mapping from tftpd-hpa to tftp-server
This will be needed by a patch I'll be submitting to t-e-i to
change the tftp implementation used by ubuntu, so that both
Ubuntu and Fedora can use the same implementation.

Change-Id: Ie9581265dfa2c37342e01162fe580b1948dd3e03
2013-10-24 14:55:22 +01:00
Jenkins
3144cc63be Merge "Allow for bad Fedora image mirrors" 2013-10-23 23:10:29 +00:00
Tomas Sedovic
1c132f6aa5 Add apache and mod_wsgi to Fedora's package map
We need this for the Horizon element (which should work both on debian and
fedora-based distros).

Change-Id: I3a1dacbd1823abe051ba977b92b67ed27423f405
Signed-off-by: Tomas Sedovic <tsedovic@redhat.com>
2013-10-23 15:23:42 +02:00
Derek Higgins
9947cdebd9 Allow for bad Fedora image mirrors
I am getting a lot of http 404's from Fedora mirrors, in particular in
the US. Alternatively we could get cache-url to use previously existing
files if they exist and there is an error but this may not be the
correct thing todo in all cases.

So for now we do a retry if HTTP 404 is returned to the request for a
Fedora image.

Change-Id: I7634aeac3bdfe6930fafe6d276186c09fc8587b5
2013-10-23 10:49:29 +01:00
Derek Higgins
1adb4072b8 exit 44 if http 404 is returned
This will allow the fedora element to know that there was a http 404 so
it can retry.

Change-Id: I42100a175e42af1c53909508245695ac82204fc0
2013-10-23 10:44:47 +01:00
Jenkins
194df26096 Merge "Remove old-kernels for fedora" 2013-10-17 21:01:34 +00:00
Lucas Alvares Gomes
f707a6bd82 Dracut regenerate initrd w/ the right kernel
When picking the latest version of the kernel on i386 we should prioritize
PAE kernels. Debug kernels will be ignored.

Change-Id: Ic0fc5907074ee2a5ddfbbb1db2f1c8a6060cae9f
Related-Bug: #1240873
2013-10-17 13:50:54 +01:00
Lucas Alvares Gomes
0181b177c2 Remove old-kernels for fedora
Remove the old kernels on fedora to save some space in the image.

Change-Id: Ia7ace63c286052713ac1ef3092373cc2712db11a
2013-10-17 11:51:44 +01:00
Derek Higgins
d23ba2a8a8 Conditionally add dhcp-all-interfaces
If the /etc/init drectory gets created on Fedora, it causes
os-svc-daemon to add upstart init scripts instead of systemd.
As a result none of the openstack services can start.

Partial-Bug: #1239880

Change-Id: Iec317baa3eb9ff651fa66c582d2f614993cde45e
2013-10-15 13:43:50 +13:00
Jenkins
a019fcf470 Merge "Specify distro release in fedora image name" 2013-10-14 11:51:22 +00:00
Jenkins
52956a2c3a Merge "Allow for redirects to ftp servers" 2013-10-12 15:38:39 +00:00
Clint Byrum
f96f705559 Remove \r chars from dhcp-all-interfaces
For some reason we are adding CR's to/etc/network/interfaces, but it
does not require them. They are a bit annoying to see in an editor and
serve no known purpose there.

Change-Id: I9aeeff5533f418f09fcf33edd42e5d85cd486d23
2013-10-11 11:06:16 -07:00
Clint Byrum
28190d5954 Make dhcp-all-interfaces block all interfaces
Previously dhcp-all-interfaces was only blocking the first network
interface to come up. We add an instance stanza to make it run one
instance of generate-interfaces-file per interface. We then use flock to
serialize runs of generate-interfaces-file.

Fixes bug #1233577

Change-Id: Ib16bed6b37ce0789e315ef57e05ac561470a6f2a
2013-10-11 11:06:16 -07:00
Clint Byrum
2129d78124 Fix ifquery call in dhcp-all-interfaces
Ifquery does not print anything for interfaces which only have a single
iface line. It does, however, return non-zero if the interface is not
configured at all, so we can use that to indicate whether or not there
is a configuration.

Fixes bug #1233579

Change-Id: Ia2fdafbea57e806eba99ae8ddaf395ebdcc306e1
2013-10-11 11:06:16 -07:00
Derek Higgins
b67b850a7e Allow for redirects to ftp servers
The Fedora image can redirect to a ftp server, in which case outputs a
213 if not modified.

Change-Id: I714fc3562bf188994357e1918af309d8edeecde5
2013-10-11 10:16:51 +01:00
Derek Higgins
11b116977c Specify distro release in fedora image name
The fedora element downloads the latest available image so presumably
will jump to F20 once available. Probably causing several days (weeks?)
of busted stuff. Of course it will be impossible to know when all users
of the elements are ready to switch but the least we can do is allow a
little time as a buffer.

This commit ties it down to a specific version which can then be updated
when the consumers of this element are ready. This allso follows the
same pattern as the ubuntu element.

Change-Id: I15c8e15a66e8af1bd152c27144acbc55af9da88e
2013-10-10 08:21:52 +01:00
Tim Serong
43c0082aa7 Move /tmp/ccache setup to base element
Commit c7d80dd (Cleanup mount points automatically) removed the unmount
of $TMP_MOUNT_PATH/tmp/ccache in run_d_in_target() and moved the
"rm /tmp/ccache" to elements/base/finalise.d/02-remove-ccache.  There
are two problems with this:

1) Not unmounting at the end of run_d_in_target() results in tmp/ccache
   being bind mounted muliple times on top of itself (three times, if you
   just run `disk-image-create base`).  It is eventually unmounted, but
   somehow the auto unmount code is confused, and tries to unmount it
   one more time than it was mounted, which results in an error like
   "umount: /tmp/image.THQkZxQa/mnt/tmp/ccache: not mounted".
   This doesn't actually break anything, but it's a little messy.

2) "rm /tmp/ccache" in elements/base/finalise.d/02-remove-ccache never
   succeeds in removing /tmp/ccache, because that hook is invoked by
   run_d_in_target(), *while* /tmp/ccache is mounted.

This present commit solves the above by moving the ccache setup glue out
of img-functions and into the base element's root.d.  This has the
following implications:

1) lib/img-functions is a little cleaner.

2) /tmp/ccache is available in the chroot during the root, extra-data,
   pre-install, install and post-install stages.  It is not available
   during block-device, finalise and cleanup stages as it will have been
   automatically unmounted by then.

3) /tmp/ccache won't be setup if you're building an image that doesn't
   include the base element.

Change-Id: Ief4c0a6f4ec622db6c6f652776215684178d8943
2013-10-10 16:27:09 +11:00
Jenkins
126099cd6d Merge "Use --numeric-owner when extracting base image" 2013-10-08 22:54:08 +00:00
Tim Serong
6da49c6d49 Use --numeric-owner when extracting base image
When extracting the base image without --numeric-owner, user and group
names in the tarball are mapped to uid/gid by the host.  This can cause
problems when building an image for some other distro than you're
running yourself.  For example, building an Ubuntu image on openSUSE
ends up with /var/cache/man in the image owned by 'proxy' (uid 13)
instead of 'man' (uid 6), because the host (openSUSE) uses uid 13 for
the 'man' user.  This particular man/proxy discrepancy results in
"fopen: Permission denied" errors when apt-get does its "Processing
triggers for man-db" thing in the Ubuntu system.  I wouldn't be
surprised if there were other kinks caused by this uid/gid mapping
discrepancy too, but that's the one I found so far.

The same thing can also happen with Fedora, but seems to be less likely,
or at least less obvious to me when building Fedora images on openSUSE.
But, IMO, it's better to be safe and just use --numeric-owner on all
base image untarring outside the chroot.

Change-Id: I9da5ac66dd182e7278fe4fee932093f61d35673a
2013-10-08 22:45:51 +11:00
Matthew Farrellee
ddef437cd2 Make RHEL subscription optional
The code to handle unregister of RHEL subscriptions was buggy and
broke if no subscription credentials were supplied.

Change-Id: Iac29c45f207725e31eac6487a87367fcd3d34d49
2013-10-07 14:46:57 -04:00
Jenkins
21a81d8b79 Merge "Fix mellanox module loading." 2013-10-07 03:49:46 +00:00
Jenkins
59db52aa80 Merge "Ignore empty files returned by curl" 2013-10-07 00:36:55 +00:00
Chris Jones
8c17a5d040 Fix mellanox module loading.
My previous attempt at forcing the mellanox module to load was
completely bogus. This should not be (although I lack hardware to be
100% sure).

Change-Id: I22ff88181c9c9f0c024e021eeb7f16d79715241a
Closes-Bug: #1233949
2013-10-04 09:53:03 +01:00
Derek Higgins
50ea375154 Ignore empty files returned by curl
In cases where servers ignore the Modified time, curl cancels the
download, outputs a http 200 and leaves the output file untouched, we
don't want this empty file.

Fixes bug #1234926

Change-Id: I05b0dd95dcd53ca50d88ec07f2f1ee9958b6adb7
2013-10-04 09:26:43 +01:00
Clint Byrum
4a751ba162 Generate interfaces file before cloud-init runs
Cloud-init needs to query the metadata server after the network
interfaces are configured. The upstart job "cloud-init-nonet" is
specifically in place to provide a hook to block cloud-init from
running while we rearrange network interface configurations.

Fixes bug #1233577

Change-Id: Ib5cf75d858fdb670b2abcc082e912c4644d6b169
2013-10-03 16:00:19 -07:00
Robert Collins
2684cddd3d Fix typo in dhcp-all-interfaces.
It is meant to write to /etc/network/interfaces, not
/etc/network/interfaces].

Change-Id: I15ea677edf7129ffd222c39627f7f76356f68e11
2013-10-01 20:32:40 +13:00
Tomas Sedovic
d5afe2b1fa Set locale for the profiling printf command
When LC_NUMERIC is set to a format that doesn't use a decimal point,
`printf` will fail.

Change-Id: Ie6c4d075928f47b17cc413d537fc31c9d0734bdb
Signed-off-by: Tomas Sedovic <tsedovic@redhat.com>
2013-09-30 12:10:07 +02:00
Clint Byrum
f120e122bc Add mapping for atftpd in fedora/rhel
Upcoming tripleo-image-elements changes will need this.

Change-Id: I775dda78aaee892c8b47999707574054c3d9a95b
2013-09-26 10:38:44 -07:00
Jenkins
d744f7697c Merge "Make RHEL subscriptions optional" 2013-09-25 09:47:30 +00:00
Matthew Farrellee
1d4893b8bb Make RHEL subscriptions optional
The code to handle RHEL subscriptions was buggy and broke if no
subscription was supplied.

Change-Id: Ie4f30171f716312b69d760d45f25659606dadc1f
2013-09-24 17:05:40 -04:00
Matthew Farrellee
0add227af3 Add RHEL mapping for openssh-client
Change-Id: Ic0b5355c186dea403a9535497d28330f693e6843
2013-09-24 07:46:39 -04:00
Matthew Farrellee
18acacc26a Add RHEL mapping for augeas-tools
Change-Id: I296425504f5302e21bd4ba89e772c20c2be13838
2013-09-24 07:46:39 -04:00
Matthew Farrellee
50cb156369 Add RHEL mapping for default-jre
Change-Id: Icb474af27670d1266ce02cfd8179443496b3948f
2013-09-24 07:46:39 -04:00
Jenkins
6ede4c2016 Merge "Add DIB_IMAGE_CACHE" 2013-09-19 20:47:17 +00:00
Cody A.W. Somerville
a8d8e7a30d Do not prompt on removal of apt-xapian-index from ubuntu cloud images
Change-Id: I49582a0c51aafcda6c0ad7911c291d58bb6c52a6
2013-09-19 16:35:31 -04:00
Matthew Farrellee
1d2a85d5e4 Add DIB_IMAGE_CACHE
DIB_IMAGE_CACHE will be a user override for the location where images
are cached. Default location is ~/.cache/image-create

Change-Id: I3e9b9f970864d555c9ec9436344b53f6d3d66dfa
2013-09-19 14:34:05 -04:00
Clint Byrum
2d70812e81 Remove apt-xapian-index from ubuntu cloud images
This package recently caused us some very large headaches when it
was updated for a security issue. It is completely unnecessary and
should be removed.

Note that we have recommended that it be removed from the cloud images
in launchpad bug #1227425.

fixes bug #1227420

Change-Id: Ic0d4efa7b44c46271d19576f5191c9421d07c015
2013-09-18 17:22:59 -07:00
Monty Taylor
5e3b5320e4 Remove github references
OpenStack runs git.openstack.org which is more reliable and responsive
when projects operate within OpenStack Infra. Replace all of the
references to github with referneces to git.openstack.org.

Change-Id: Ib3ece85aba6451801487b0bdbd83147e39d9e155
2013-09-18 08:17:10 -05:00
Chris Jones
b50dbb0571 Move textmode forcer to vm element.
Changing the grub config makes no sense in a build not heading for a vm
and may fail because grub is removed from images not including the vm
element. Forcing textmode for those images would be better done in nova.

Change-Id: I1c5b89e551e62df2463200b1889cb2342498c7dd
2013-09-18 01:57:35 +01:00
Jenkins
bd632a92fd Merge "Fix grub/linux text mode override." 2013-09-18 00:05:12 +00:00
Chris Jones
d799ad5beb Fix grub/linux text mode override.
Apply a wider range of hammers to the recurring issue of images booting
with graphical consoles.

Change-Id: I96345bf8f2f13d615131557b572201b9d41b6c75
2013-09-17 22:03:02 +01:00
Ghe Rivero
6d1f03bda3 Add deploy-kexec element
Boots into the new image kernel once baremetal-deploy-helper signals
it is finished using kexec utilities.

Change-Id: I705787cc394ef14200d80404ee497762ab79b452
2013-09-17 14:01:20 -07:00
Dan Prince
fb33adc5ba Consolidate the checks for /etc/grub.
Create a single check for grub2 so we skip these steps on F19.

Change-Id: I5f05adcf409be5c2532478439bcb55ab397da20e
2013-09-17 11:08:04 -07:00
Jenkins
f5e191dc8e Merge "Check cached file size when downloading an url" 2013-09-16 21:24:53 +00:00
Jan Provaznik
9a5e14cbc5 Check cached file size when downloading an url
In some cases cache-url fails when downloading an image and leaves
an empty cached file. qemu-img then fails with "Wrong medium type"
error on next run.

Change-Id: I23e91c52094f27248cf8452f192ad63646051190
2013-09-16 13:32:36 -07:00
Dan Prince
15007cc184 Fedora 19 has no grub2 conf file.
This fixes ramdisk creation on Fedora 19.

Change-Id: Ica486473cecb93feac740ae905fba5f2a3723bc2
2013-09-16 12:21:59 -07:00
Steve Baker
9ba761ef27 Install fedora grub from cached rpm during finalise
When uninstalling grub2, leave all its dependencies
including grub2-tools installed to minimise the number of packages
which need to be installed in the finalise stage.

Since the yum cache is unmounted during finalise, installing
grub2 in finalise is slowed by re-populating the yum cache.

This change copies the grub2 rpm out of the yum cache so it can be installed
from file during finalise.

This should prevent disk becoming full during finalise on Fedora.

Closes-Bug: #1217185

Change-Id: If095adc4abb52a19a3aa0b1caebfb3e4d8f605ef
2013-09-15 09:11:58 +10:00
Jeff Peeler
ccd7b86b60 Remove -r option from kpartx for successful build
This option does not exist on RHEL hosts and matches what is
currently present in elements/rhel/root.d/10-rhel-cloud-image.

Change-Id: I578233c1f37d035c67600fc60e7c4eb4ff75cbb3
2013-09-12 17:56:30 -04:00
Matthew Farrellee
43b96d91c2 Add package mapping for default-jre
Change-Id: Idffbc84ea2afaf2584c1bb79729c3e0cf52b851e
2013-09-10 20:01:21 -04:00
Jenkins
d339f9b25e Merge "Add --list support to dib-run-parts." 2013-09-04 10:59:46 +00:00
Chris Jones
6311388892 Add --list support to dib-run-parts.
Our custom dib-run-parts now supports --list for better compatibility
with some OS-provided run-parts tools.

Change-Id: I2feb615b2094d9d6522bfe9b422362223bb8e652
Closes-Bug: #1190521
2013-09-04 09:44:48 +01:00
Steve Baker
c182fef45c On Fedora, use Linux Foundation bzr lsb_release
Installing redhat-lsb-core adds 87M to the install due
to bugzilla #1002342

lsb_release is a platform agnostic script, so this change fetches
a single file via source-repositories from linuxfoundation bzr http
and installs it to /usr/local/bin

Change-Id: I4c979d4eb5e34a753d9143d9dcfe4e5d08f74a0d
2013-09-04 10:16:49 +12:00
Steve Baker
9e5bc37265 Add support for file to source-repositories
Adds a file type to source-repositories to allow individual files
to make use of source-repositories caching.

REPO_DIRECTORY has been rename to REPO_DEST since it is now sometimes
a file.

This change also fixes the incorrect path to the cache-url in the
tar type.

Change-Id: Ib79e41969c982a02f0235318d9f254b39c3c6d93
2013-09-04 10:16:49 +12:00
Steve Baker
cc681eb830 Install patch for dracut patching
patch is installed indirectly via redhat-lsb-core, so it
should really be installed directly.

Meanwhile, Daddy Shadowman waited for these patches to be
submitted to bugzilla.

Change-Id: I2107cf7be99f396c5dd489a2693b949022a512e7
2013-08-29 15:10:11 +12:00
Jenkins
9211a7fecb Merge "Add rhel installation element" 2013-08-28 16:58:37 +00:00
Chris Alfonso
44c3ff7ece Add rhel installation element
If you want to have the installation update packages, you'll
need to register the system log in to rhn and subscribe to an
available subscription.
export DIB_RHSM_USER to your rhn username
export DIB_RHSM_PASSWORD to your rhn password

To get the qcow2 image, log into rhn.redhat.com and download the
image from
https://rhn.redhat.com/rhn/software/channel/downloads/Download.do?cid=16952
Then export DIB_CLOUD_IMAGES to whereever you're hosting the qcow2.

Change-Id: Idb547f4ffe75514b1e3f6b34f5f347493b132925
2013-08-28 11:35:53 -04:00
IvanBerezovskiy
d4a9b44e73 Add package mappings for MySQL-MariaDB
Partial-Bug: #1213891

Change-Id: I1ac1d11ff1716eccee4c734395282be25dd5b541
2013-08-28 15:04:12 +04:00
Lucas Alvares Gomes
5dd6c4a834 Extlinux fallback
If GRUB2 is not available in the system a fallback to install Extlinux
will happen. This patch also allows you to enforce the use of Extlinux
exporting the DIB_EXTLINUX variable.

Change-Id: I88fa792a0bd19cacca167134f2054b11c807a451
2013-08-27 11:14:44 +01:00
Lucas Alvares Gomes
b01f412c1e Rename 51-grub to 51-bootloader
This patch is part of a another big patch that will implement a fallback
to Extlinux in case GRUB2 is not available in the system. The reason why
it's being splitted in two patches is because git is not being able to
recognize the file changes when the rename happens in the same commit.

Change-Id: Ic2027dfa057fd6d62b532203b7ff50c3c739bccb
2013-08-27 10:43:26 +01:00
James Slagle
0702ce9843 Add a pip-cache element.
The element bind mounts a pip cache inside the image build chroot so
that pip downloads can be reused across image builds.  While similar
in purpose to the PyPi element that sets up a mirror, this element
just allows for a reusable download cache and doesn't require anything
to be setup beforehand.

The pip-cache element is not concurrency safe, and that is indicated
in the README for the element.  An upstream bug was file as well:
https://github.com/pypa/pip/issues/1141

Change-Id: Ibd1d4ea17c24923ed939357ada95b781e3179cfd
2013-08-26 15:17:41 -04:00
Derek Higgins
11df3a45f4 Fix URL to os-apply-config
Just incase anybody is actually using this element(which may not be the
case).

Change-Id: I663f8e4db86a38d8c9063d02da3dfaa6715e465a
2013-08-26 11:19:32 +01:00
Jenkins
6687bdd267 Merge "Fixed a network setup issue for F19" 2013-08-25 22:35:10 +00:00
Richard Su
1fb5d232be qpid package mappings
Change-Id: Iad2c21f3c2cd771f58d5444f61186b3aaf84e89e
2013-08-22 23:52:05 -07:00
Chris Alfonso
301c3c4475 Extracting common functionality for rpm based distros
Rather than dublicating code to implement rhel or any
other derivitive, this patch introduces an rpm-distro
element that should be used as a dependency.

Change-Id: I8a92bb041764d03f430b438f0013704f79a8674c
2013-08-20 16:44:19 -04:00
Chris Alfonso
e7a8fefa0b Use kpartx if partition device doesn't exist
If the loop partition isn't created as a device by the kernel, go ahead and use
kpartx to create the device.

Change-Id: I53290b7724d0cb45a1fc9225ec096025db8978dd
2013-08-20 13:27:49 -04:00
sjing
6c26cec656 Set file permission to be executable.
When ramdisk-image-create is called with the hwdiscovery element,
it failed to install hwinfo package.

The root reason is 10-hwinfo is not executable, the fix is to
set the file permission of 10-hwinfo to be executable.

Change-Id: I3a57147103f607b3975eb6ad23a44475f26c2f1e
2013-08-20 08:44:43 +00:00
Jan Provaznik
c93dfeec7f Fixed a network setup issue for F19
IP address is passed as a kernel boot param when booting undercloud
machine. For Fedora 19, this IP is not set, instead IP is fetched
from DHCP.

The problem is caused by a bug in dracut-network version 029, which
causes that dracut network setup script is not executed properly. This
patch applies a fix for the bug when building undercloud image.

Fixes: bug #1213833
Change-Id: Ie94a86ed716044343791628973143c1a7d0778dc
2013-08-19 10:37:46 +02:00
Monty Taylor
d18891d952 Update stackforge references to openstack
Change-Id: I17e896eddf6fe84f4784cac8231552ee3d21aa39
2013-08-17 22:58:26 -04:00
James Slagle
c9a6aef9f4 Add yum element.
The yum element provides for specific customizations
for yum based distros.  The included customization is
a yum cache mounted outside of the chroot so that yum
downloaded packages are reused on subsequent image builds.

Change-Id: I6833c9fdbc83cb09debec6a789082e105c917800
2013-08-14 18:26:51 -04:00
Steven Dake
2c73a001b2 dkms is unavailable on RHEL and derivatives
The dynamic kernel module system is not available on RHEL, CentOS,
Scientific Linux, or SUSE.  Make it part of the distro post-install
rather then base post-install.

Change-Id: Ic2c345bf9f0738dadae611194e263d3a5d424a3e
2013-08-12 19:07:28 -07:00
Clint Byrum
89fe51b83d Fix pypi element README.md errors
- pip install to a github URL without a git+ prefix fails.
- Need to close EOF heredoc when creating mirror.yaml

Change-Id: Ic555174881e9c06fede1cf731141cbe6eb0e0b02
2013-08-12 14:53:18 -07:00
Robert Collins
97bc5d7853 Allow using a pypi mirror to install via pip.
Using a custom pypi mirror can be very convenient, making image builds
substantially faster - because we create multiple virtual
environments we benefit more than single-virtualenv users would.

Change-Id: I997daf1f9477c447e1fb30818aea9e80a49b31a6
2013-08-10 22:07:30 +12:00
Robert Collins
c7d80ddeed Cleanup mount points automatically.
Manually listing the mount points that need cleanup is not
maintainable and makes it hard to write elements that use bind
mounting to inject resources into the build environment.

Change-Id: I7c9ade444f4ebe42552f8e321f257a7ec0a674ef
2013-08-09 22:53:32 +12:00
Derek Higgins
a5806cd482 Change the rootfs label in F18 and F19
The images produced by the diskimage-builder have their filesystem with
a label of "cloudimg-rootfs", The regex changing fstab on fedora didn't
match the F19 case.

This regex matches both
F18 :
LABEL=_/   /         ext4    defaults        1 1
F19 :
UUID=f2da0fc8-2e12-4a6e-8e95-a31a474aa1a8 / ext4 defaults,noatime 0 0

Change-Id: Iafcf5e069199f1df76874309b2d5064f343fb5e5
2013-08-07 09:57:21 +01:00
Robert Collins
c7c4ec19a3 Capture the repositories from source-repositories.
We try to expose metadata to users about what dib included in the
image and source-repositories was not doing that, so this fixes it. It
also permits easy introspection from other elements (such as an
upcoming PyPI cache layer).

Change-Id: I1e48e65f69ebe1f4556cc3bf35ccf195e99bcc76
2013-08-05 09:52:28 +12:00
Jenkins
9233ca4056 Merge "Enable running disk-image-create on SUSE Linux" 2013-08-01 00:36:36 +00:00
Clint Byrum
5a69fa879e Make Ubuntu 13.04 (raring) the default release
Ubuntu 13.04 has been released now for 3 months. The updated libvirt,
openvswitch and kernel are all beneficial to various OpenStack components,
and many other software is updated beyond the versions in Ubuntu 12.10.

Change-Id: I358aed8bf906c3ff5103f19b1f9e6ac689b5d5ee
2013-07-29 17:02:23 -07:00
Jenkins
45f32b4b82 Merge "Ignore emacs autosave files in source-repositories" 2013-07-29 05:04:12 +00:00
Jenkins
1146fe63fc Merge "Add needed symlink for Fedora deployment ramdisk." 2013-07-28 19:58:59 +00:00
Monty Taylor
4535c5323a Ignore emacs autosave files in source-repositories
source-repositories uses find to discover things it should import. If
someone happens to use emacs, and happens to have had an autosave file
in a directory with a wildly broken version of a source-repositories-
file, then dib will attempt to execute the contents of the autosave file
and the user will be sad. Adding a filter to the find command for files
ending in ~ will prevent pain and suffering.

Change-Id: I27c8a4b53d3c9b913ca926db16b4271941f6fb5d
2013-07-27 14:53:29 -04:00
Monty Taylor
c179d9b04d Add downloadonly flag to fedora
devstack-gate needs to be able to make an image that has pre-downloaded
but not installed a sequence of packages. To support that, add a -d flag
to install-packages to allow downloading but not installing. -d passes
through on a dpkg install, and with this patch should be transformed
to --downloadonly which is the yum version.

Change-Id: Ia4e61fa4304df86afbee7dfc8067ea1bdf528f3e
2013-07-27 12:17:02 -04:00
James Slagle
92fa96b2f6 Add needed symlink for Fedora deployment ramdisk.
Symlink /usr/lib -> ../lib for Fedora deployment ramdisk.  This
symlink is needed for systemd-udevd.  Otherwise, the network device is
not loaded and initialized when the deployment image is booted.  This
logic was previously contained in
elements/deploy/root.d/50-redhat-prepare-fs, but now that ramdisk
images are built in a chroot, the change needs to be moved so that it
is applied to the ramdisk build.

Change-Id: Icea43230126956ccf5fb8a6a96ca706b75d5c32f
2013-07-26 11:09:29 -04:00
Derek Higgins
d8ba46f8e3 Install which on Fedora
Create an install hook where core utilities can be installed
on Fedora. In this case we are installing which.
Which is being used by
  ramdisk/post-install.d/01-ensure-binaries
  vm/finalise.d/51-grub
The absence to which caused the ramdisk build to fail on F19.

On Ubuntu which is installed with the package debianutils, this pretty
much has to exist on Ubuntu images, so a corresponding package install for
ubuntu isn't needed.

Fixes Bug 1202612

Change-Id: I847c1619cdab021c9348caed3fa32db64e663e8e
2013-07-26 11:04:58 +01:00
Tim Serong
ee5ae03d58 Enable running disk-image-create on SUSE Linux
- Ensures /sbin and friends are in $PATH when invoked (without this,
  various sudo invocations fail in exciting ways).
- Use dib-run-parts in lib/common-functions instead of run-parts
  (neither SLES nor openSUSE ship run-parts).
- Ensure dib-run-parts doesn't descend into subdirectories (same
  behaviour as run-parts).
- Move dib-run-parts from root.d to bin (cleaner, consistent with
  other elements with separate bin scripts).
- Tested by building Ubuntu image on openSUSE 12.3.
- Note: this doesn't add support for creating SUSE images, it just
  lets you run disk-image-create on SUSE-based distros.

Change-Id: I906c6bc3cf51cdf2c4415adeae1ca250faac25e1
2013-07-24 20:16:31 +10:00
Jenkins
5f52562570 Merge "Support repo names with multiple '-'s" 2013-07-19 19:45:42 +00:00
Derek Higgins
a4a0523f2b Support repo names with multiple '-'s
The search and replace that converted dashes in REPONAME only converted
the first dash. They should all be converted.

Change-Id: I76a48cf5bdfa8664f0a65d0238fa4da7d80305db
2013-07-19 17:45:01 +01:00