It seems that on Xenial, it does not take much to confuse "file" and
it's mime guessing such that it thinks some files are not python.
"package-installs-v2" is a good example, since it has an interpreter
"dib-python" that "file" doesn't know about, and no extension. While
looking at this, I've added emacs vars here so it opens in python
mode.
Change-Id: I01994b08c5ad8987925f1eec4062f5b6ee72eb8f
Icf8a075224833fcfbbe2128e8802ff41c39f3c09 looked rather ugly, and it's
easy for us to expand the processing done in the arch list.
Change "arch" to a comma-separated list of architectures that should
match for install.
Add a "not-arch" list which will exclude the package from installation
on those architectures. (An aside -- I considered making it just he
one list with foo,!bar,moo but ! has special meaning in YAML, so it's
easier to have two lists).
$ ARCH=ppc64 package-installs-squash --elements ironic-agent --path=./elements/ /dev/stdout | grep dmidecode
$ ARCH=ppc64 package-installs-squash --elements ironic-agent --path=./elements/ /dev/stdout | grep lshw
"lshw",
$ ARCH=amd64 package-installs-squash --elements ironic-agent --path=./elements/ /dev/stdout | grep lshw
$ ARCH=amd64 package-installs-squash --elements ironic-agent --path=./elements/ /dev/stdout | grep dmidecode
"dmidecode",
Change-Id: Ic69dd02a09e6f3ba9078a2377d8df29871a20db2
The various distro elements include a copy of this script which installs
all files in the bin/ dir of the copied element hooks in to
/usr/local/bin. Lets share code rather than repeating ourselves.
Change-Id: I354382f8c42ede2e9b8c548b7df8367c03e6836e
If we are being run from a python 3 virtualenv where python2 is also
available on the host we will use the hosts' python2 over the virtualenv
python3 (even if the virtualenv is higher precedence in PATH).
Change-Id: I9a25b9e45845a4121aab8250fd53c6f006825742
This action has been reordered so cleanup can occur before copy to blockdev
occurs. Documentation has been added about the ordering of this element in
relation to cleanup actions as well.
Change-Id: I3f9334a3669ee588d7fa7129202c97fa22fdb050
subprocess.check_call() returns a byte-string which needs to be turned
into a unicode string for python3 compatability.
Also some minor refactoring while we're here.
Closes-Bug: 1536462
Change-Id: Icd957bc4d93ccad94b1246ad62e6e02ee14d9ca5
Use dib-python to run package-installs using the provided python
version. Automatically detect the python version for our
package-installs-squash since that runs outside the chroot.
Change-Id: I926022bcf8cbcd81b051026ffd5d6477650045ad
subprocess.CalledProcessError in Python 2.6 does not have the 'out'
parameter for __init__, so pass only two of them and manually set
'output' in that case.
Fixes/improves commit 7f410aaff2.
Change-Id: I279bdf433b1272a9c3af4d66a2a52c78a7ac5de2
In some cases, like linux-image-* on debian, we need to only install
packages for a specific target architecture.
Change-Id: Ic0009d0c1e121d6f3f1f21345c544e2d98f080f9
Turns out that updating packages last causes some pretty
non-intuitive behaviour if you are trying to pin a package
to a specific version. Lets just update the base RPMs first...
subsequent installations should install the most updated version
anyways (unless they are pinned).
Also moves the package-installs script from the 00 step to 01 so
we can do the update first.
Co-Authored-By: Ben Nemec <bnemec@redhat.com>
Change-Id: I962046cc6048e852e6582fbc579f88bb73e23fdd
Previously, this code was not checking for the proper environment
variable for an element's installtype. There was a line replacing '-'
with '_' as is required, but that value was not actually used when
searching for the environment variable.
Change-Id: I0bbd56969188389db81844d9276269464870f776
subprocess.check_output() has been introduced in Python 2.7, so the
script will fail when trying to install stuff in guests with Python 2.6
and older (like RHEL 6 / CentOS 6, for example).
Thus gracefully fallback to subprocess.Popen() when
subprocess.check_output() is not available.
Change-Id: I335148397932177810f095a942b993b249991107
Closes-Bug: #1415240
There is a wide variety of tracing options through the various shell
scripts. Some use "set -eux", others explicity set xtrace and others
do nothing. There is a "-x" option to bin/disk-image-create but it
doesn't flow down to the many scripts it calls.
This adds a global integer variable set by disk-image-create
DIB_DEBUG_TRACE. All scripts have a stanza added to detect this and
turn on tracing. Any other tracing methods are rolled into this. So
the standard header is
---
if [ "${DIB_DEBUG_TRACE:-0}" -gt 0 ]; then
set -x
fi
set -eu
set -o pipefail
---
Multiple -x options can be specified to dib-create-image, which
increases the value of DIB_DEBUG_TRACE. If script authors feel their
script should only trace at higher levels, they should modify the
"-gt" value. If they feel it should trace by default, they can modify
the default value also.
Changes to pachset 16 : scripts which currently trace themselves by
default have retained this behaviour with DIB_DEBUG_TRACE defaulting
to "1". This was done by running [1] on patch set 15. See the thread
beginning at [2]
dib-lint is also updated to look for the variable being matched.
[1] https://gist.github.com/ianw/71bbda9e6acc74ccd0fd
[2] http://lists.openstack.org/pipermail/openstack-dev/2014-November/051575.html
Change-Id: I6c5a962260741dcf6f89da9a33b96372a719b7b0
If there are no old-style package-installs and the package-installs
element is included we fail because we call install-packages with no
arguments.
Change-Id: I3c78b51d0a2673ecc581f9faded078974dbc27b7
Since the package-installs-squash script relies on the defined
environment variables for figuring out what the values are for an
element's install type and the default install types, sudo -E must be
used to preserve the environment.
Change-Id: Ice5cf0bca65d8078f84daedbef123fc73d5a7b63
The latest update to package-install captures both stderr and stdout
from pkg-map, unfortunately, pkg-map has a 'missing-ok' option
which causes it to print an error message on stderr.
The result is that package-install tries to look for packages named
"Missing", "package", "name", etc.
Change-Id: I86b3b71a64b29d533b42fd0cae020e8ecf22cac2
Closes-bug: 1402085
Instead of doing the work in the image of parsing through the element's
package-install declarations, we can squash it on hostside, where we
have both YAML and JSON available to us, and then emit a single
pre-processed file into the target to be used later.
Change-Id: I3f182aa3aae0a79b2f3ea4e66c1878ad12878b0a
The file extention claims rst format, but it's plain text
as such, the online documentation doesn't render properly
Change-Id: I24accb45ab8c7803a25f2642ce1b2d479d5a6e9c
We currently support package-installs definitions which has some
limitations and oddities. This new format requires only one definition
which does not reside in our run-parts directories and follows a
consistent naming scheme (package-installs.yaml).
Change-Id: Ie51a7c4fdc15634ae8e069728e5e07cc1dc36095
Our package-installs script fails when installing a package which does
not have a pkg-map but a pkg-map file exist for the element.
Change-Id: I3dab802e23bccfc916efcc1c70c6ce6c4a9ccf67
Packages are often also installed in both the pre-install.d and
post-install.d phases. This patch expands the package-installs element
to support declarative package support for these phases in addition to
the existing support for install.d. The actual install/uninstall logic
is moved to common scripts under bin/ so that it can be reused across
the different phases.
Change-Id: Id51d0bbad232737fc8b5ffaf016dec50cd5b66c9
Adds a new element, package-installs, that provides an interface for
declarative package installs and uninstalls. Packages to install can be
added to an install.d/package-installs-<element-name> file. The set of
packages listed across such files are installed in a single transaction
at the beginning of install.d.
Prefacing the package name with a "-" indicates that the package should
be uninstalled at the end of the install.d phase. Again, the full set of
uninstalls are done in a single transaction.
An element providing a package-installs file should add package-installs
to its element-deps file.
Change-Id: I5b540388eff1079c8dee933b869463371481152b
