Commit Graph

59 Commits

Author SHA1 Message Date
Ian Wienand
36b59c001c Standarise tracing for scripts
There is a wide variety of tracing options through the various shell
scripts.  Some use "set -eux", others explicity set xtrace and others
do nothing.  There is a "-x" option to bin/disk-image-create but it
doesn't flow down to the many scripts it calls.

This adds a global integer variable set by disk-image-create
DIB_DEBUG_TRACE.  All scripts have a stanza added to detect this and
turn on tracing.  Any other tracing methods are rolled into this.  So
the standard header is

---
if [ "${DIB_DEBUG_TRACE:-0}" -gt 0 ]; then
    set -x
fi
set -eu
set -o pipefail
---

Multiple -x options can be specified to dib-create-image, which
increases the value of DIB_DEBUG_TRACE.  If script authors feel their
script should only trace at higher levels, they should modify the
"-gt" value.  If they feel it should trace by default, they can modify
the default value also.

Changes to pachset 16 : scripts which currently trace themselves by
default have retained this behaviour with DIB_DEBUG_TRACE defaulting
to "1".  This was done by running [1] on patch set 15.  See the thread
beginning at [2]

dib-lint is also updated to look for the variable being matched.

[1] https://gist.github.com/ianw/71bbda9e6acc74ccd0fd
[2] http://lists.openstack.org/pipermail/openstack-dev/2014-November/051575.html

Change-Id: I6c5a962260741dcf6f89da9a33b96372a719b7b0
2015-02-12 10:41:32 +11:00
Gregory Haynes
c4bbb6f3bc Create docs site containing element READMEs
We currently do not have the ability to create a docs site which
outlines all the elements.

Change-Id: I77ccf61e0c4b1509b3e7ce9b8f15ea5ccfd50d9b
2015-02-10 11:45:35 -08:00
Gregory Haynes
fd0708286c Check python with flake8 instead of dib-lint
We check python files with dib-lint rather than flake8 which have
conflicting opinions. This means weve been (forcibly) writing non pep8
python.

Also fixing pep8 issues so tests pass.

Change-Id: Idc9db40334f6e15738a7802c06697270df68741c
2014-11-21 09:04:52 -08:00
Jenkins
d506c42ec4 Merge "Deprecate map-packages, replaced by pkg-map" 2014-11-20 01:41:45 +00:00
Ryan Brady
4178c0c69d Deprecate map-packages, replaced by pkg-map
This patch writes a warning out to stderr to notify element authors
that may be using map-packages to migrate to pkg-map.  This patch
also prints out a warning during image-create that lists the specific
elements that are not using pkg-map.

Change-Id: I7e2a7611dd5650815736ce998aa94a7833193d06
2014-11-19 12:04:12 -05:00
Victor Lowther
38b14df72c Make diskimage-builder work in Docker.
When running inside a Docker container, we cannot rely on devices in
/dev/mapper to be automagically created by udev, because we probably
don't have a udev at all.  To work around this, run dmsetup mknodes
after every kpartx run.

Change-Id: If7e30579224ce54c5ed26d08974d8293c144719a
2014-09-15 06:43:53 -05:00
Pino Toscano
267e96f52c Centralize handling of /lost+found
Just remove globally /lost+found after the root and install phases, so
distribution elements don't need to do that by themselves.

Change-Id: Ic783e613bd8794aefd3f40c9a7c308d14cd04b8d
2014-08-04 11:48:21 +02:00
Ryan Brady
7f78be9318 Adds RHEL common element
This element contains the installation steps common to RHEL
operating system versions.

Change-Id: I37cce0bd5b5d31525f1335ecd416be570d36ad61
2014-07-25 14:17:03 -04:00
Om Kumar
aff9b29c4d Update RHEL 6.5 Image name.
The default value points to old image name
which results in 404 error. So updating the
value to point to correct name.

Change-Id: Ia66f0cf4f4167926892a0786467412d277ebffc3
2014-07-15 14:17:06 +00:00
Jenkins
4e97def3d5 Merge "Correction: if then statement code style." 2014-07-10 07:38:49 +00:00
AzherKhan
d06a1caafb Correction: if then statement code style.
Corrected the if; then statement to follow bash convention.

Change-Id: Ia3f80520bb9446c4feb5ce36dccffa2b1c1d72c4
2014-07-07 03:43:15 -07:00
Ghe Rivero
4fd84db547 Rename rhel element yum blacklist
Rename rhel 10-yum-blacklist to 10-rhel-blacklist to
avoid future name conflict with a yum element blacklist.

Change-Id: Ic2136a06e1ec8f19908ada978782733d5fa7cacc
2014-07-02 10:53:35 +00:00
Jenkins
7489540b22 Merge "Correct the wrong rename in rhel element" 2014-06-12 13:24:55 +00:00
tanlin
ebd54b1010 Correct the wrong rename in rhel element
There is a substitution of suffix in rhel element, like
rhel-guest-image-6.5-20140121.0.x86_64.qcow2 to
rhel-guest-image-6.5-20140121.0.x86_64.raw

The convert is correct but the substitution of the suffix is wrong.

${PARAMETER#PATTERN} will match the string from the beginning and generate rhel-guest-image-6.5-20140121.0.x86_64.qcow2.raw

We should use ${PARAMETER%PATTERN} instead, which is from the end.

Change-Id: Ic45a3f800058ea9e5d746ff543c068a54632407d
2014-06-03 06:42:35 +00:00
Dan Prince
50cb019a25 Set DISTRO_NAME in OS environment.d
Updates the various operating system elements so that we
set the DISTRO_NAME variable for each distro.

This env var is used by bin/pkg-map to set the default
distribution name for package name mappings.

Change-Id: Ib4b05eb7191dd50d25799a0bac51fd15c01b74cb
2014-05-28 12:22:02 -04:00
Jenkins
ab8a0f4bdd Merge "add some missing \n at end of file" 2014-05-28 00:40:57 +00:00
Jenkins
0f6f533993 Merge "Export unbound variable DIB_RHSM_USER" 2014-05-22 17:02:19 +00:00
iberezovskiy
8995d6402a Export unbound variable DIB_RHSM_USER
If 'DIB_RHSM_USER' isn't set, the image build with rhel
element fails because of check 'set -u'. So, we should export
'DIB_RHSM_USER' in script to avoid error during build

Change-Id: Ic63a20e3d91a25bbe804440a5919b035489c60a3
Closes-bug: #1318717
2014-05-13 12:03:27 +04:00
Gonéri Le Bouder
ea3f4dd459 4 spaces indent
ensure 4 spaces indentation is used everywhere.

Change-Id: Ieb48faacb4c96b7b358771d70c17f2f22d0354f4
2014-05-10 11:17:29 +02:00
Ben Nemec
16be6d7ce0 set -u and -o pipefail everywhere
As with the previous similar changes, this is intended to catch
problems as they happen instead of ignoring them and continuing on
to potentially fail later.  Setting this on all existing scripts
will allow us to enforce use via Jenkins.

Change-Id: Iad2d490c86dceab148ea9ab08f457c49a5d5352e
2014-05-06 15:51:07 -05:00
Gonéri Le Bouder
3ae4edc276 add some missing \n at end of file
This to avoid the “No newline at end of file” error from patch/git.

Change-Id: Ice82d4a36a0893e8f71dbffbdab154d1fbab8a09
2014-05-06 13:43:53 +02:00
Jenkins
301dc5cc0e Merge "set -e all the things" 2014-04-29 11:29:43 +00:00
Ben Nemec
f6ba2aeaf4 set -e all the things
Using set -e in all of our scripts will prevent some subtle bugs
from slipping in, and will allow us to enforce use of set -e with
tooling.

This change also adds -u and set -o pipefail in the less complex
scripts where it is unlikely to cause problems.  A follow-up change
will enable those options in the complex scripts so that if it
breaks something it can be reverted easily.

Change-Id: I0ad358ccb98da7277a0ee2e9ce8fda98438675eb
2014-04-25 17:38:51 -05:00
Alexis Lee
21f01e937c Map openjdk-7-jre-headless to RHEL+SUSE
Provide package name mappings for openjdk-7-jre-headless:
        RHEL/Fedora: java-1.7.0-openjdk-headless
        SUSE: java-1_7_0-openjdk-headless

See https://review.openstack.org/#/c/86314

Change-Id: I3a351d0f9854f6f320ba145a71ffc66fd02c231c
2014-04-24 07:49:12 +00:00
Alexis Lee
6b2a78f3ab Sort rhel/bin/map-packages
Unlike the redhat-common and opensuse map-packages maps, the RHEL one
appears to be unsorted. Fix that.

Change-Id: I75c5d2694525731660c2a3b5b58ee696c61923ba
2014-04-23 08:53:03 +01:00
Jon-Paul Sullivan
793dae8fe6 Use provides to note an element provides an OS
Use "operating-system" to denote that an element provides that to the
image being created.  This can then be used to specify an abstract
dependency in other elements to assist in build-time image verification.

Change-Id: Ic125803da65e0f8f8c20ef90b08815f4f7d09762
2014-04-10 14:42:04 +01:00
Ben Nemec
63230414d2 Alphabetize all element-deps
Change-Id: Idc31d60daeea61d6df0b70dac4a5671fa5a804ee
2014-03-26 15:23:18 -05:00
Om Kumar
dd7b7765e6 RHEL Package maps for build-essential, python-dev and libz-dev.
Some of the tripleo-image-elements were failing for RHEL due to lack of
build essential, python-dev and libz-dev.

This commit should fix those failure.

Change-Id: I080f9601faedb3259deada5cfbda7d00d3f13eb9
2014-03-14 08:54:48 +05:30
Perry Myers
b9e77477bb Update the image name for RHEL Guest Image
The RHEL 6 Guest Image has been updated with a new release that
addresses a number of bugs.  The diskimage-builder element that
references this file needed to be updated to reference the correct
image name.  In addition, some notes to help users download the
image from RHN have been added to the README.

Change-Id: I3c9e64a1887afd9c0b370d70eff4b7f9598cebd1
Closes-bug: launchpad-bug-1286568
2014-03-01 12:23:02 -05:00
Ben Nemec
004474e93b Generalize install-packages for yum
All of the yum-based install-packages scripts are doing essentially
the same thing, so let's use one script for all of the elements
that depend on the yum element.

Change-Id: I49e8c9b44e41bcf4cb9fa820e8a9179754694a97
2014-02-21 12:00:24 -06:00
Dan Prince
4f795b91ed Add libvirt-bin -> libvirtd to map-services.
We'll need this soon for the Nova TripleO image element.

Change-Id: I9f4284aa503f422828e2ccb56597852b5a0c9591
2014-02-20 14:38:56 -05:00
Dirk Mueller
0ab2a1300f Pep8/Pyflakes fixing
Just avoiding the error popups in the editor that way.

Change-Id: Ic0cb903dfb0e9eff6b12d004da720919f6f1a009
2014-02-17 22:37:34 +01:00
Shane Wang
4aeee175d8 Fix misspellings in diskimage-builder
Fix misspellings detected by:
* pip install misspellings
* git ls-files | grep -v locale | misspellings -f -

Change-Id: I42acbcce7611ff3e2e120168acccc518299c2c42
Closes-Bug: #1257295
2014-02-07 14:42:45 +08:00
James Slagle
dfde5c434b Add package map for stunnel4
stunnel4 is called stunnel on Fedora/RHEL.

Change-Id: Ic27692e485792c87a9e54c5baa6b7a75410e8cf2
2014-01-28 19:47:32 -05:00
aweiteka
bb04ff1ef2 Add support for Red Hat Satellite
A Red Hat Satellite URL may be provided as the source for
Red Hat packages. A Satellite key may be provided for a pre-
defined set of RHN channels to be automatically applied. Support
for installation of a Satellite certificate also provided.

Change-Id: Iae5b97d6f4732a28efa08ab778a56c2f9b6b6bd9
2014-01-21 14:26:35 -05:00
aweiteka
ce14214a7a Support list of Red Hat channels and repos
When registering to RHN or RHSM a list of space-separated
channels or repositories may be provided. This allows control
over which packages are available to the system during build.
Note, the optional channel or repository is added by default
since diskimage-builder requires access to its packages. This
list is for additional channels or repos.

Change-Id: I9cf90d47ab1d09f5491f574132410438f9e294e3
2014-01-21 14:26:35 -05:00
aweiteka
78dbbe68fe Add support Red Hat Network (RHN)
This provides basic support to register to the Red Hat Network
(RHN Classic) for packages. The image is unregistered from RHN
before completing image build. The rhnsd service is prevented
from starting since it is not needed for packagage install.

Change-Id: I2dc124cdc6ae59a63fd94d82d77108acccc3f668
2014-01-21 14:26:35 -05:00
aweiteka
3dc40c925d Update default RHEL guest image
Red Hat periodically updates the qcow2 guest image available
for download. This sets the default image name to the latest.

Change-Id: Iba3075bbee3b41918d5cd3da9721fcbf98ff3bcd
2014-01-21 14:26:35 -05:00
Dirk Mueller
2def5b7727 fedora/RHEL: use env from /usr/bin
/usr/bin/env is the new canonical location of "env", use
it.

Change-Id: Ifc5fbd942d409b1f54057579c5f57ccac63d4b72
2014-01-20 16:16:41 +01:00
James Slagle
f5d4d8c711 Mark install-packages +x.
install-packages is an executable script, it should be +x.  The install
command that was installing it is setting permissions to 755, so we were
fine there.  However, I was trying to test the script outside of an
image build to test a change, and it's helpful to have it +x in the
repository for that context like all the other scripts under bin.

Change-Id: Ic300cc56d463640d119db4e310d375c66ed133ca
2014-01-07 07:54:47 -05:00
James Slagle
476ec8feec Add mapping for tgt to RHEL element.
tgt maps to scsi-target-utils on RHEL.  This mapping already exists for
the Fedora element, so no need to add it there.

Change-Id: Ifc9a8976299458cdef50c2fbe00ba6a0c8d22f00
Closes-Bug: #1252975
2013-11-26 15:21:21 -05:00
iberezovskiy
41caf597a5 Check existence of rhel rpm key
The key 'RPM-GPG-KEY-redhat-release' may not exist in image
e.g. when you use non-default image (for example, if you build CentOS
image using rhel element).

Fixes bug #1252697

Change-Id: Iae642abb7a7f76c5528750dafedd4fd4bda56fe2
2013-11-19 20:02:12 +04:00
Derek Higgins
257a9dd139 Add mapping for gearmand
The ubuntu gearman-job-server package maps to the Red Hat gearmand
package.

Change-Id: I3ec90b2759498488fce2ecaed03effc18313d1e6
2013-11-18 16:44:14 +00:00
Victor Lowther
43b70ce224 /bin/bash all the things.
Since we are using bash syntax in some of the element fragments,
we should make sure we use bash for all of them, so that things don't
break on systems where /bin/sh != /bin/bash.

Change-Id: If2f043c57aa4e1492b7f9839213ef6123f683612
2013-11-17 16:59:31 -06:00
Derek Higgins
726a816cac Add mapping for libffi-dev
libffi header files are needed to pip install xattr, which is a
requirement of swift.

Change-Id: I69560425f7106c35ec432825eab94ca8424eb89e
2013-11-12 20:32:52 +00:00
iberezovskiy
f23babe971 Check existence of directory 'lost+found'
The command `sudo rmdir $TARGET_ROOT/lost+found` will fail
if `$TARGET_ROOT/lost+found` directory doesn't exist,
e.g. when you use non-default image.

Fixes bug #1245856

Change-Id: I48c8f2f201b29912a726249023ca7d20893cc958
2013-10-29 18:51:49 +04:00
Derek Higgins
d91ececa06 Remove mapping of atftpd to tftpd-server
These are different tftp implementations, tftpd-hpa and tftpd-server are
equivalent, this mapping was added in a previous commit.

Change-Id: Idd1f554d4ee8e1d3c515d5f2f191e90abe0ff396
2013-10-25 12:27:39 +01:00
Derek Higgins
66b293af1c Add redhat mapping from tftpd-hpa to tftp-server
This will be needed by a patch I'll be submitting to t-e-i to
change the tftp implementation used by ubuntu, so that both
Ubuntu and Fedora can use the same implementation.

Change-Id: Ie9581265dfa2c37342e01162fe580b1948dd3e03
2013-10-24 14:55:22 +01:00
Jenkins
126099cd6d Merge "Use --numeric-owner when extracting base image" 2013-10-08 22:54:08 +00:00
Tim Serong
6da49c6d49 Use --numeric-owner when extracting base image
When extracting the base image without --numeric-owner, user and group
names in the tarball are mapped to uid/gid by the host.  This can cause
problems when building an image for some other distro than you're
running yourself.  For example, building an Ubuntu image on openSUSE
ends up with /var/cache/man in the image owned by 'proxy' (uid 13)
instead of 'man' (uid 6), because the host (openSUSE) uses uid 13 for
the 'man' user.  This particular man/proxy discrepancy results in
"fopen: Permission denied" errors when apt-get does its "Processing
triggers for man-db" thing in the Ubuntu system.  I wouldn't be
surprised if there were other kinks caused by this uid/gid mapping
discrepancy too, but that's the one I found so far.

The same thing can also happen with Fedora, but seems to be less likely,
or at least less obvious to me when building Fedora images on openSUSE.
But, IMO, it's better to be safe and just use --numeric-owner on all
base image untarring outside the chroot.

Change-Id: I9da5ac66dd182e7278fe4fee932093f61d35673a
2013-10-08 22:45:51 +11:00