Commit Graph

34 Commits

Author SHA1 Message Date
Ian Wienand
bc6be85424 Trace package install in package-installs-v2
When running the package install, trace the output so we can see what
packages were installed.

Change-Id: I5442f544ff0ef3ddffdbe6b898d178548d699a41
2016-11-23 19:58:45 +11:00
Ian Wienand
f15550f9fe Special case dib-python in dib-lint
It seems that on Xenial, it does not take much to confuse "file" and
it's mime guessing such that it thinks some files are not python.

"package-installs-v2" is a good example, since it has an interpreter
"dib-python" that "file" doesn't know about, and no extension.  While
looking at this, I've added emacs vars here so it opens in python
mode.

Change-Id: I01994b08c5ad8987925f1eec4062f5b6ee72eb8f
2016-11-23 19:58:43 +11:00
Ian Wienand
8a1c8370a1 package-installs: add list to arch and "not-arch" list
Icf8a075224833fcfbbe2128e8802ff41c39f3c09 looked rather ugly, and it's
easy for us to expand the processing done in the arch list.

Change "arch" to a comma-separated list of architectures that should
match for install.

Add a "not-arch" list which will exclude the package from installation
on those architectures.  (An aside -- I considered making it just he
one list with foo,!bar,moo but ! has special meaning in YAML, so it's
easier to have two lists).

$ ARCH=ppc64 package-installs-squash --elements ironic-agent --path=./elements/ /dev/stdout | grep dmidecode
$ ARCH=ppc64 package-installs-squash --elements ironic-agent --path=./elements/ /dev/stdout | grep lshw
    "lshw",
$ ARCH=amd64 package-installs-squash --elements ironic-agent --path=./elements/ /dev/stdout | grep lshw
$ ARCH=amd64 package-installs-squash --elements ironic-agent --path=./elements/ /dev/stdout | grep dmidecode
    "dmidecode",

Change-Id: Ic69dd02a09e6f3ba9078a2377d8df29871a20db2
2016-07-01 21:31:59 +02:00
Gregory Haynes
642f906476 Add install-bin element
The various distro elements include a copy of this script which installs
all files in the bin/ dir of the copied element hooks in to
/usr/local/bin. Lets share code rather than repeating ourselves.

Change-Id: I354382f8c42ede2e9b8c548b7df8367c03e6836e
2016-05-13 11:52:23 +10:00
Jenkins
1fabb01a4f Merge "Prioritize venv python on host" 2016-04-18 23:17:28 +00:00
Matthew Thode
175e292ec1 Add Gentoo to the list of supported distributions
We also document any additional package needed.

Change-Id: I7b9193fa5711138287f587935fc864fee4f95856
2016-02-29 12:20:50 +00:00
Gregory Haynes
cb15e7ef33 Prioritize venv python on host
If we are being run from a python 3 virtualenv where python2 is also
available on the host we will use the hosts' python2 over the virtualenv
python3 (even if the virtualenv is higher precedence in PATH).

Change-Id: I9a25b9e45845a4121aab8250fd53c6f006825742
2016-02-28 20:50:02 -08:00
Matthew Thode
22ef95c1a1
Reorder the package-uninstall action
This action has been reordered so cleanup can occur before copy to blockdev
occurs.  Documentation has been added about the ordering of this element in
relation to cleanup actions as well.

Change-Id: I3f9334a3669ee588d7fa7129202c97fa22fdb050
2016-02-11 18:30:05 -06:00
Matthew Thode
c6ef183975 Fix package-installs for python3
subprocess.check_call() returns a byte-string which needs to be turned
into a unicode string for python3 compatability.

Also some minor refactoring while we're here.

Closes-Bug: 1536462
Change-Id: Icd957bc4d93ccad94b1246ad62e6e02ee14d9ca5
2016-02-01 22:22:15 +00:00
Jenkins
af85ea066a Merge "Run package-intalls with py3k if we must" 2016-01-19 05:58:58 +00:00
Gregory Haynes
d1e32f80a6 Run package-intalls with py3k if we must
Use dib-python to run package-installs using the provided python
version. Automatically detect the python version for our
package-installs-squash since that runs outside the chroot.

Change-Id: I926022bcf8cbcd81b051026ffd5d6477650045ad
2016-01-04 23:37:30 +00:00
Gregory Haynes
40799839e0 Package installs defaults to tracing off
There isn't a good reason we should be defaulting to trace on for these
scripts.

Change-Id: I1b9ac9388f51a49e6912d800914a0e0f2faec604
2015-12-20 00:03:51 +00:00
Jenkins
cb4ce88048 Merge "package-installs: fix error case for Python 2.6" 2015-05-07 17:25:54 +00:00
Pino Toscano
655648f99b package-installs: fix error case for Python 2.6
subprocess.CalledProcessError in Python 2.6 does not have the 'out'
parameter for __init__, so pass only two of them and manually set
'output' in that case.

Fixes/improves commit 7f410aaff2.

Change-Id: I279bdf433b1272a9c3af4d66a2a52c78a7ac5de2
2015-05-06 16:18:30 +02:00
Gregory Haynes
28c42fafdc Support arch-specific package-installs
In some cases, like linux-image-* on debian, we need to only install
packages for a specific target architecture.

Change-Id: Ic0009d0c1e121d6f3f1f21345c544e2d98f080f9
2015-04-24 21:58:05 +00:00
Dan Prince
208da5f605 Update packages earlier
Turns out that updating packages last causes some pretty
non-intuitive behaviour if you are trying to pin a package
to a specific version. Lets just update the base RPMs first...
subsequent installations should install the most updated version
anyways (unless they are pinned).

Also moves the package-installs script from the 00 step to 01 so
we can do the update first.

Co-Authored-By: Ben Nemec <bnemec@redhat.com>

Change-Id: I962046cc6048e852e6582fbc579f88bb73e23fdd
2015-04-13 21:50:10 +00:00
James Slagle
ba19541d47 Fix check for installtype
Previously, this code was not checking for the proper environment
variable for an element's installtype. There was a line replacing '-'
with '_' as is required, but that value was not actually used when
searching for the environment variable.

Change-Id: I0bbd56969188389db81844d9276269464870f776
2015-03-10 21:59:03 -04:00
Jenkins
cec46c56a4 Merge "Standarise tracing for scripts" 2015-03-04 00:38:44 +00:00
Jenkins
226eaa3e3f Merge "Dont fail if we have no old-style package-installs" 2015-02-28 18:05:27 +00:00
Pino Toscano
7f410aaff2 package-installs: work with Python < 2.7
subprocess.check_output() has been introduced in Python 2.7, so the
script will fail when trying to install stuff in guests with Python 2.6
and older (like RHEL 6 / CentOS 6, for example).

Thus gracefully fallback to subprocess.Popen() when
subprocess.check_output() is not available.

Change-Id: I335148397932177810f095a942b993b249991107
Closes-Bug: #1415240
2015-02-25 17:46:08 +01:00
Ian Wienand
36b59c001c Standarise tracing for scripts
There is a wide variety of tracing options through the various shell
scripts.  Some use "set -eux", others explicity set xtrace and others
do nothing.  There is a "-x" option to bin/disk-image-create but it
doesn't flow down to the many scripts it calls.

This adds a global integer variable set by disk-image-create
DIB_DEBUG_TRACE.  All scripts have a stanza added to detect this and
turn on tracing.  Any other tracing methods are rolled into this.  So
the standard header is

---
if [ "${DIB_DEBUG_TRACE:-0}" -gt 0 ]; then
    set -x
fi
set -eu
set -o pipefail
---

Multiple -x options can be specified to dib-create-image, which
increases the value of DIB_DEBUG_TRACE.  If script authors feel their
script should only trace at higher levels, they should modify the
"-gt" value.  If they feel it should trace by default, they can modify
the default value also.

Changes to pachset 16 : scripts which currently trace themselves by
default have retained this behaviour with DIB_DEBUG_TRACE defaulting
to "1".  This was done by running [1] on patch set 15.  See the thread
beginning at [2]

dib-lint is also updated to look for the variable being matched.

[1] https://gist.github.com/ianw/71bbda9e6acc74ccd0fd
[2] http://lists.openstack.org/pipermail/openstack-dev/2014-November/051575.html

Change-Id: I6c5a962260741dcf6f89da9a33b96372a719b7b0
2015-02-12 10:41:32 +11:00
Gregory Haynes
c4bbb6f3bc Create docs site containing element READMEs
We currently do not have the ability to create a docs site which
outlines all the elements.

Change-Id: I77ccf61e0c4b1509b3e7ce9b8f15ea5ccfd50d9b
2015-02-10 11:45:35 -08:00
Gregory Haynes
fc1b5d8b6d Dont fail if we have no old-style package-installs
If there are no old-style package-installs and the package-installs
element is included we fail because we call install-packages with no
arguments.

Change-Id: I3c78b51d0a2673ecc581f9faded078974dbc27b7
2015-02-04 18:23:36 -08:00
James Slagle
92b0125da1 Use sudo -E when squashing package installs files
Since the package-installs-squash script  relies on the defined
environment variables for figuring out what the values are for an
element's install type and the default install types, sudo -E must be
used to preserve the environment.

Change-Id: Ice5cf0bca65d8078f84daedbef123fc73d5a7b63
2015-02-04 13:25:48 -05:00
Gregory Haynes
3ff92589e1 Add installtype support to package-installs
Some packages should only be installed if a certain installtype is
specified.

Change-Id: Ia1a5af9ab4653e2646870ebd2d2db7e00a59305b
2015-01-28 20:11:51 -08:00
Jenkins
c597e99f4e Merge "Ignore stderr from pkg-map" 2015-01-06 21:20:18 +00:00
Abel Lopez
da7787069d Ignore stderr from pkg-map
The latest update to package-install captures both stderr and stdout
from pkg-map, unfortunately, pkg-map has a 'missing-ok' option
which causes it to print an error message on stderr.
The result is that package-install tries to look for packages named
"Missing", "package", "name", etc.

Change-Id: I86b3b71a64b29d533b42fd0cae020e8ecf22cac2
Closes-bug: 1402085
2014-12-17 10:08:08 -08:00
Jenkins
9a53631e1c Merge "Fix rst rendering" 2014-12-16 13:51:59 +00:00
Monty Taylor
98d008c6af Rework package-installs to collapse on build host
Instead of doing the work in the image of parsing through the element's
package-install declarations, we can squash it on hostside, where we
have both YAML and JSON available to us, and then emit a single
pre-processed file into the target to be used later.

Change-Id: I3f182aa3aae0a79b2f3ea4e66c1878ad12878b0a
2014-12-11 00:05:32 -08:00
Abel Lopez
563d334105 Fix rst rendering
The file extention claims rst format, but it's plain text
as such, the online documentation doesn't render properly

Change-Id: I24accb45ab8c7803a25f2642ce1b2d479d5a6e9c
2014-12-03 12:30:28 -08:00
Gregory Haynes
e5b8656141 Add new package-installs system
We currently support package-installs definitions which has some
limitations and oddities. This new format requires only one definition
which does not reside in our run-parts directories and follows a
consistent naming scheme (package-installs.yaml).

Change-Id: Ie51a7c4fdc15634ae8e069728e5e07cc1dc36095
2014-12-01 21:29:47 -08:00
Gregory Haynes
c27e32f769 package-installs assumes packages have a pkg-map
Our package-installs script fails when installing a package which does
not have a pkg-map but a pkg-map file exist for the element.

Change-Id: I3dab802e23bccfc916efcc1c70c6ce6c4a9ccf67
2014-10-27 09:42:58 -07:00
James Slagle
17f8bda93f package-installs for pre-install.d/post-install.d
Packages are often also installed in both the pre-install.d and
post-install.d phases. This patch expands the package-installs element
to support declarative package support for these phases in addition to
the existing support for install.d. The actual install/uninstall logic
is moved to common scripts under bin/ so that it can be reused across
the different phases.

Change-Id: Id51d0bbad232737fc8b5ffaf016dec50cd5b66c9
2014-08-14 21:18:19 -04:00
James Slagle
7f9ebf2632 Support declarative package installs/uninstalls
Adds a new element, package-installs, that provides an interface for
declarative package installs and uninstalls. Packages to install can be
added to an install.d/package-installs-<element-name> file. The set of
packages listed across such files are installed in a single transaction
at the beginning of install.d.

Prefacing the package name with a "-" indicates that the package should
be uninstalled at the end of the install.d phase. Again, the full set of
uninstalls are done in a single transaction.

An element providing a package-installs file should add package-installs
to its element-deps file.

Change-Id: I5b540388eff1079c8dee933b869463371481152b
2014-05-01 08:47:14 -04:00