Commit Graph

56 Commits

Author SHA1 Message Date
Ian Wienand
36b59c001c Standarise tracing for scripts
There is a wide variety of tracing options through the various shell
scripts.  Some use "set -eux", others explicity set xtrace and others
do nothing.  There is a "-x" option to bin/disk-image-create but it
doesn't flow down to the many scripts it calls.

This adds a global integer variable set by disk-image-create
DIB_DEBUG_TRACE.  All scripts have a stanza added to detect this and
turn on tracing.  Any other tracing methods are rolled into this.  So
the standard header is

---
if [ "${DIB_DEBUG_TRACE:-0}" -gt 0 ]; then
    set -x
fi
set -eu
set -o pipefail
---

Multiple -x options can be specified to dib-create-image, which
increases the value of DIB_DEBUG_TRACE.  If script authors feel their
script should only trace at higher levels, they should modify the
"-gt" value.  If they feel it should trace by default, they can modify
the default value also.

Changes to pachset 16 : scripts which currently trace themselves by
default have retained this behaviour with DIB_DEBUG_TRACE defaulting
to "1".  This was done by running [1] on patch set 15.  See the thread
beginning at [2]

dib-lint is also updated to look for the variable being matched.

[1] https://gist.github.com/ianw/71bbda9e6acc74ccd0fd
[2] http://lists.openstack.org/pipermail/openstack-dev/2014-November/051575.html

Change-Id: I6c5a962260741dcf6f89da9a33b96372a719b7b0
2015-02-12 10:41:32 +11:00
Gregory Haynes
c4bbb6f3bc Create docs site containing element READMEs
We currently do not have the ability to create a docs site which
outlines all the elements.

Change-Id: I77ccf61e0c4b1509b3e7ce9b8f15ea5ccfd50d9b
2015-02-10 11:45:35 -08:00
yogananth subramanian
15ccf776be Add support for using local PowerPC VM image
Patch adds support for PowerPC Big-endian and Little-endian local
fedora VM images and using VM images with two partitions,  PReP
boot and root partition. Since PowerPC requires PReP boot
partition to be present.

Change-Id: Iba03226e187609df898732c13b1aa5f895b156e3
Closes-Bug: 1413487
2015-01-23 05:11:08 +05:30
Gregory Haynes
2626606980 Migrate to new package-installs
We have a new package-installs file format. Migrating existing
package-installs files to the new format.

Change-Id: I57d4a007efee9624e60c41357cefa627d8c7373f
2014-12-01 21:32:13 -08:00
Monty Taylor
1b51e22f7d Remove the grub2 install from redhat-common
We immediately remove this, but the -grub2 causes a later attempt to
remove it again which fails. Don't install it in the first place.

Change-Id: I31d64abc9596707bdb8a0505d0dcbf7b82e1b82d
2014-11-28 11:01:40 -05:00
Jenkins
019196ef7f Merge "Add deprecation warning when using map-services" 2014-11-22 14:48:33 +00:00
Jenkins
d506c42ec4 Merge "Deprecate map-packages, replaced by pkg-map" 2014-11-20 01:41:45 +00:00
Ryan Brady
4178c0c69d Deprecate map-packages, replaced by pkg-map
This patch writes a warning out to stderr to notify element authors
that may be using map-packages to migrate to pkg-map.  This patch
also prints out a warning during image-create that lists the specific
elements that are not using pkg-map.

Change-Id: I7e2a7611dd5650815736ce998aa94a7833193d06
2014-11-19 12:04:12 -05:00
Ryan Brady
1aecb41b7e Add deprecation warning when using map-services
This patch writes a warning out to stderr to notify element authors
that may be using map-services to migrate to svc-map.

Change-Id: Ic80db16c607958d025e89b3a4058a9cbb568938e
2014-11-12 17:03:49 +00:00
Derek Higgins
c26a9ee4b1 Install lsb_release from package
The URL we were using to download lsb_release from no longer works,
install from package so this is not affected by disappearing URL's
in future.

This was originally installed from a URL because the package dependencies
caused a 87M increase in the size of the base image. For fedora the
increase is now 27M which is a little more tolerable.

Change-Id: I6e56a4a81786b33c5c6b92df2bd8236cd3f19670
2014-11-10 11:28:49 +00:00
AzherKhan
bb38d0e325 Optimize speed of deletion in find command
I would like to recommend to use + instead of \; in the find
command. As this will ensure the removal of all selected
directories in a single invocation.

Hence improve the speed of deletion.

Change-Id: I409fe11aae217afb6f790491591005c679264ed4
2014-10-27 21:04:09 -07:00
Jenkins
34c273d8df Merge "Make diskimage-builder work in Docker." 2014-10-24 20:13:45 +00:00
Jenkins
8ba6a26e0a Merge "Allow source-repositories to be disabled completely" 2014-10-20 11:35:40 +00:00
Jenkins
e289b31d12 Merge "Consolidate lsb_release source-repositories" 2014-10-16 21:14:05 +00:00
Ghe Rivero
873de01491 Remove first-boot.d support
After being deprecated two releases ago, finally remove any reference
for the support of first-boot.d

Change-Id: I08d67404ef48cad61db3b18fb86e970abfa5d2b6
2014-10-16 09:48:14 +00:00
Ben Nemec
0eccd2808c Allow source-repositories to be disabled completely
In our official image builds we are only allowed to use resources
that are "blessed" by the build system.  This means that external
things like git repos and tar files are not allowed.  Currently,
even in offline mode source-repositories expects those things to
be available in the cache, so we need a way to disable it entirely.

This change adds an environment variable NO_SOURCE_REPOSITORIES
that does so.  It can be set in an environment.d script so elements
that might rely on a source repository will know it's not available.
The 02-lsb script in redhat-common is one such example and is
updated to handle this case.

Change-Id: I0de63bee6ad79733d6711478c707a9b41593e85f
2014-10-15 12:24:50 -04:00
Ben Nemec
0d08925892 Consolidate lsb_release source-repositories
Each distro had its own copy of the file, but they were all the
same and it's actually installed in redhat-common so it belongs
there.

Change-Id: Iad15f39fd28d1c4e20d30dcb2eb0ae6f2fa2b015
2014-10-14 15:09:44 -05:00
Ben Nemec
edd7477891 Save extended attributes when creating tar
The way redhat-common's extract-image script was creating the base
tarball caused file capabilities to get dropped, which meant that
things like ping in RHEL 7 images was unusable for regular users.
This change adds the necessary options to the tar call to maintain
as many extended attributes as possible.

--acls and --selinux are intentionally omitted, and the selinux
xattrs are filtered out because all of those items cause issues
in our chroot environment.  We restore selinux attributes at the
end of the build anyway so that shouldn't be a problem.

bz reference: https://bugzilla.redhat.com/show_bug.cgi?id=1144149

Change-Id: Ibff99ce9bde01bc5ecf95dc3a5d3e2cebe5015b9
2014-10-03 23:05:17 -05:00
Jenkins
0ddd704e08 Merge "Update to newer Centos7 images" 2014-09-19 22:05:10 +00:00
Ian Wienand
45afb6d86c Update to newer Centos7 images
Update to newer Centos7 images as announced at [1].

Changes included:
 * update to GA image
 * respect TMP_DIR for image working directory
 * use suffix-match rather than prefix-match to build RAW_IMAGE name
   to be more robust against weird names
 * partition layout work-arounds not needed any-more, all one
   partition.

Resulting image booted locally with kvm and virtualbox (after
conversion to vdi)

[1] https://www.redhat.com/archives/rdo-list/2014-August/msg00045.html

Closes-Bug: #1363146
Change-Id: Ie24033468b78587ea87188ee1b843b26895798ff
2014-09-18 10:48:40 +10:00
Victor Lowther
38b14df72c Make diskimage-builder work in Docker.
When running inside a Docker container, we cannot rely on devices in
/dev/mapper to be automagically created by udev, because we probably
don't have a udev at all.  To work around this, run dmsetup mknodes
after every kpartx run.

Change-Id: If7e30579224ce54c5ed26d08974d8293c144719a
2014-09-15 06:43:53 -05:00
James Slagle
9fa28a31f1 Add ceilometer service mappings
Add the service mappings needed for ceilometer elements.

Change-Id: Id139a7c208659ead6330842bec01f7466d7ba417
2014-09-12 09:00:18 -04:00
Jenkins
f4011996ee Merge "Lock around extract-image downloads and extracts" 2014-09-11 11:56:40 +00:00
James Slagle
07c22185bd Use package-installs in more elements
Updates additional elements in diskimage-builder to use the declarative
package install support provided by the package-installs element.
package-installs does not yet support the pkg-map functionality, so
elements using pkg-map are not migrated. That support will come in the
near future.

Change-Id: I3d36adad317ba44326eabd95243d45807e2a8a16
2014-09-08 15:16:14 -04:00
Jon-Paul Sullivan
ed833e628f Lock around extract-image downloads and extracts
This wil be needed for parallel image builds.

Change-Id: I6ba3a10b44fe37f8d8903ad06302a672c3e972c8
2014-08-28 17:42:25 +01:00
Jenkins
7d0e7fe5ad Merge "Centralize handling of /lost+found" 2014-08-28 16:33:12 +00:00
Jenkins
597acccbd7 Merge "map-services: add openstack-nova-novncproxy" 2014-08-26 08:28:13 +00:00
James Slagle
0c92466f5b Add service mappings for ironic
Adds the needed service mappings for ironic-api and ironic-conductor.

Change-Id: I8b218f926561d8fba1644fe883bf8aae4bb6473f
2014-08-20 17:31:45 -04:00
Derek Higgins
bd83c2e699 map-services: add openstack-nova-novncproxy
Add a mapping from nova-novncproxy to openstack-nova-novncproxy, it was missing.

Change-Id: Ia163a967925087b9d20f7b8581f0f636e53328d8
2014-08-18 13:15:10 +01:00
Pino Toscano
267e96f52c Centralize handling of /lost+found
Just remove globally /lost+found after the root and install phases, so
distribution elements don't need to do that by themselves.

Change-Id: Ic783e613bd8794aefd3f40c9a7c308d14cd04b8d
2014-08-04 11:48:21 +02:00
Jan Provaznik
517c858275 Remove package mapping for mariadb-rdo-galera-server
This mapping is not needed anymore because mariadb-rpm element
(renamed from mariadb-rdo) uses pkg-map.

Relies on: I90e7ca4803516b86179b0f6639e307a60297960b
Change-Id: I19fb44ee6008c6c57dd331a780eec3234d46b15c
2014-07-28 10:36:24 +02:00
Ian Wienand
5abb4a4f12 Initial centos7 support
Initial support for a centos7 image.

This is separate to rhel7 because the major differences are things
like repo and image locations, which are always going to be different.
We should merge any real changes into the redhat-common layers.

Apart from the added support files in centos7/*, the other change is
mostly modifications to redhat-common's extract-image to handle
different partition layouts of the centos7 image.

Change-Id: I943abe5ff0a803f36eda266a79af0d9220edcae7
2014-07-16 10:43:05 +10:00
AzherKhan
d06a1caafb Correction: if then statement code style.
Corrected the if; then statement to follow bash convention.

Change-Id: Ia3f80520bb9446c4feb5ce36dccffa2b1c1d72c4
2014-07-07 03:43:15 -07:00
Jenkins
83e35dfa49 Merge "map-services: add openvswitch" 2014-06-23 16:00:54 +00:00
Dan Prince
93d8d022d7 Delete redhat-common 00-usr-local-bin-secure-path
Both the rpm-distro and redhat-common elements had the
same exact 00-usr-local-bin-secure-path. This commit
deletes the one in redhat-common and leaves the one
in rpm-distro (which is also required by the same
elements).

Change-Id: Ie9ef29d268b3737a6741fce016ef335db6651b59
2014-06-11 08:16:41 -04:00
Dan Prince
50374fe4e7 Name 99-setup-first-boot uniquely
While an end user would never use both redhat-common and ubuntu
elements at the same time we should always use unique names
for things per DIB conventions.

Change-Id: I2cabc583a1fd41d4fe7fbf8cf93127d49e26aff4
2014-06-11 08:16:41 -04:00
Gonéri Le Bouder
1de9612dad map-services: add openvswitch
OVS service is called:
 - openvswitch-switch on Debian/Ubuntu
 - openvswitch on RHEL/Fedora

This changes is needed to get the neutron element to work
on Debian/Ubuntu with systemd:
  I6e3df30dc3a6918f3a949a7dac47289ede5c3d1f

Change-Id: I2748a9db7f8ee456698187a9646bc7cda2256fe7
2014-06-09 09:18:02 +00:00
Jenkins
ab8a0f4bdd Merge "add some missing \n at end of file" 2014-05-28 00:40:57 +00:00
Gonéri Le Bouder
ea3f4dd459 4 spaces indent
ensure 4 spaces indentation is used everywhere.

Change-Id: Ieb48faacb4c96b7b358771d70c17f2f22d0354f4
2014-05-10 11:17:29 +02:00
Ben Nemec
16be6d7ce0 set -u and -o pipefail everywhere
As with the previous similar changes, this is intended to catch
problems as they happen instead of ignoring them and continuing on
to potentially fail later.  Setting this on all existing scripts
will allow us to enforce use via Jenkins.

Change-Id: Iad2d490c86dceab148ea9ab08f457c49a5d5352e
2014-05-06 15:51:07 -05:00
Gonéri Le Bouder
3ae4edc276 add some missing \n at end of file
This to avoid the “No newline at end of file” error from patch/git.

Change-Id: Ice82d4a36a0893e8f71dbffbdab154d1fbab8a09
2014-05-06 13:43:53 +02:00
Jenkins
301dc5cc0e Merge "set -e all the things" 2014-04-29 11:29:43 +00:00
Jenkins
1384f8aa16 Merge "Map openjdk-7-jre-headless to RHEL+SUSE" 2014-04-29 08:10:37 +00:00
Jenkins
b1e8c082c3 Merge "Add a mapping for kernel headers" 2014-04-26 00:38:12 +00:00
Ben Nemec
f6ba2aeaf4 set -e all the things
Using set -e in all of our scripts will prevent some subtle bugs
from slipping in, and will allow us to enforce use of set -e with
tooling.

This change also adds -u and set -o pipefail in the less complex
scripts where it is unlikely to cause problems.  A follow-up change
will enable those options in the complex scripts so that if it
breaks something it can be reverted easily.

Change-Id: I0ad358ccb98da7277a0ee2e9ce8fda98438675eb
2014-04-25 17:38:51 -05:00
Monty Taylor
1c07d4ba1a Remove All Rights Reserved
The phrase is no longer needed as of August 23, 2000 with Nicaragua's
joining of the Berne Convention.

Additionally, in at least one instance,
elements/cache-url/bin/cache-url, its existence in the file between
Copyright lines is just weird and feels misleading, even though it is
not.

Remove all of the lines, because sanity.

Change-Id: I24fd76c2b4f66b8036010b5079db39ead729abee
2014-04-25 11:25:10 -07:00
Monty Taylor
ace82f16e9 Add a mapping for kernel headers
Nodepool needs to be able to install kernel headers, which means we
need a mapping for redhat systems here.

Change-Id: Ic83d642b5ebe85b042cf7b23b295be74bbe185fa
2014-04-24 10:44:40 -07:00
Alexis Lee
21f01e937c Map openjdk-7-jre-headless to RHEL+SUSE
Provide package name mappings for openjdk-7-jre-headless:
        RHEL/Fedora: java-1.7.0-openjdk-headless
        SUSE: java-1_7_0-openjdk-headless

See https://review.openstack.org/#/c/86314

Change-Id: I3a351d0f9854f6f320ba145a71ffc66fd02c231c
2014-04-24 07:49:12 +00:00
Jenkins
ed5a872adf Merge "Add mapping for mariadb rdo package" 2014-04-16 14:30:19 +00:00
Jan Provaznik
40adbff681 Add mapping for mariadb rdo package
Mariadb package in RDO repository is mariadb-galera-server, but
this name is mapped to MariaDB-Galera-server because of
mariadb installed from mariadb.org.

To avoid mapping conflicts mariadb-rdo uses virtual package name
mariadb-rdo-galera-server which is mapped to mariadb-galera-server.

Change-Id: Idf77d91876baeb81d89eb688808b892ece2bf19a
2014-04-15 12:02:52 +02:00