There are a couple of loops identified here that output a lot of
tracing each iteration for little value. Make them only trace at
higher levels (-x -x and above).
Points of actual interest within the loops are replaced with an
explicit "echo" statement.
Additionally, export DIB_DEBUG_TRACE explicitly and in all cases, not
just when tracing is turned on.
Change-Id: Id710c0b111fc1f5e1ae87fc35f6db28b24867bad
dib_[environment|args] manifest files are currently generated by the
base element and then moved by the manifest element.
This creates too many corner cases -- if you don't include the base
element (we are trying to empty it ATM) you don't get the env/args
saved at all; if you include base but don't include the manifest
element they're saved to /etc, but if you do have the manifest element
they're moved to the manifest dir.
Move generation of these into the manifest element directly and update
the documentation to reflect this. In practice this doesn't change
things, because the "manifests" element gets pulled in via deps for
most builds.
Change-Id: I3f23037058137d166b29f0b70fd1a02c22c07fc8
Signed-off-by: Andreas Florath <andreas@florath.net>
In error scenarios there might be no loadable state available.
This patch adds some robustness handling this scenarios
by checking the result and possible bailing out.
Change-Id: I02e2731b14bec22c22db08d60d8d15db1911a84e
Signed-off-by: Andreas Florath <andreas@florath.net>
With the old configuration structure it was only possible
to use one image and one partition layout. The new
block-device configuration uses a list at top level;
therefore it is possible to use multiple instances
of each element type.
Change-Id: I9db4327486b676887d6ce09609994116dbebfc89
Signed-off-by: Andreas Florath <andreas@florath.net>
When merging feature/v2 we have re-enabled the centos-minimal test
which has a blank line in the element-deps. The modified get_elements
functions added in If97bcd21e45de1b5ed91023fdc441a4617051a6b were not
handling this.
Fixing this, we can extract this into some common code for
element-deps and element-provides.
Also, I noticed while debugging that the element list is an iterator
from reverse(), so printing it doesn't help. Fix up the debug output.
Change-Id: I8e9b079f586e8a36dba3797a0fc03ca1b2041a04
Change ec7f56c1b2 add added unit tests
in diskimage_builder/tests/functional (this is probably misnamed).
These are found and run by testr just as part of the normal "setup.py
test" run. Don't run them as part of the functional tests (this
breaks "-h"/"-l" because it installs a virtualenv and runs tests, and
also is incorrect in the gate where it's creating a nested virtualenv
underneath the testing virtualenv).
Change-Id: I9908e080042d3026a198ba89eb653c6eff376d22
During the creation of a disk image (e.g. for a VM), there is the need
to create, setup, configure and afterwards detach some kind of storage
where the newly installed OS can be copied to or directly installed
in.
This patch implements partitioning handling.
Change-Id: I0ca6a4ae3a2684d473b44e5f332ee4225ee30f8c
Signed-off-by: Andreas Florath <andreas@florath.net>
This option allows to specifiy command for compress built initramfs image
for ironic-agent element. This command can be specified with arguments and
should read raw data from stdin and write compressed data to stdout.
Default if "gzip".
Change-Id: I0fdd2ab91d7bf7aaaa6cdd5278e3902d44c5b883
On redhat/fedora /sbin is a symlink to /usr/sbin, but not on all
platforms. This was put in with
Ibf74dd1b2678ea76e0676711a7aa5ba6b88d5421
Change-Id: I7847b29503c3c07503430a7d85a5364911894c6c
Closes-bug: #1658297
It looks like I installed the cleanup file in the wrong location. Moving it
to the correct location and cleaning some more stuff up should allow for a
much smaller image, currently tested at 300M vs 490M.
Change-Id: I9d0a24b0fe59d4f2d38aa88ba47e4400e0476b46
Now that fedora-25 is released, bump fedora-minimal / fedora elements
to use it.
Change-Id: Ifdb05255e7a138b99099252fe17041fe4621b6f4
Depends-On: I3cc467a2d08486458b3f625f94ba969532f6cd04
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
The start script of dhcp-all-interfaces currently requires ifup command.
ifup command provided by package ifupdown, which is not installed in
ubuntu-minimal Xenial.
This change adds ifupdown package for debian family, as it required for
dhcp-all-interfaces.
Closes-bug: #1647853
Change-Id: I6dfc0108ec067f350b22e9fb933b9e8d47b09fde
I noticed in debugging that with python3 all the pkg-installs output
is preceeded by a b'foo', which suggests coding issues lurking.
The universal_newlines argument makes subprocess readline() returns a
decoded str (via locale) instead of a byte-string. This clears up a
couple of points where we explicitly decode and cleans up the print
output.
Minor formatting cleanup of command and exit display. Don't strip the
leading spaces so indents retain in the output
Change-Id: I2894f10a0c2fc618563641b9d106b716f4a544aa
DIB automatically includes it's base elements in the ELEMENT_PATH;
this double-set means the base elements are in the path twice, leading
to some tools that walk the element list (package-installs-squash, for
example) to do things twice.
Change-Id: Icf631427601cfd2abed436711cedb9e002fff8f9
There's a few things going on here
Firstly, we need to install coverage tool in test-requirements
Secondly, .testr.conf has to use PYTHON because the coverage report
works by resetting PYTHON='coverage run ...'
Thirdly, because we call ourselves diskimage-builder but the python
module is diskimage_builder that seems to confuse things. We need to
use "setup.py test" (note, that is different to "setup.py testr"!) to
use the PBR testr wrapper. That exposes a --coverage-package-name
argument that calls the coverage tool with the right argument.
With this I got a coverage report for our unit tests
Change-Id: I9012e18eb7d01bee035140e70afa76c47c27eb01
Per the bug report, %I results in -'s in the interface name being
replaced with /'s, which means when we try to look up the interface
by path in dhcp-all-interfaces.sh we end up at an invalid path.
Using %i instead should fix the problem.
See https://www.freedesktop.org/software/systemd/man/systemd.unit.html
Essentially what is happening is that we start with a name like
br-ex that wasn't escaped in the first place. However, because of
the - it looks like it could have been escaped to systemd. When
we use %I, which tries to unescape escaped values, it unescapes a
value that was already not escaped.
Change-Id: I434ed2e084d4477dc7a2b7827164586f8ea0c1e3
Closes-Bug: 1649409
We can use ${DIB_DEBIAN_COMPONENTS} to get this information. We also
already set the mirror.
Change-Id: Idd11c2b7df1d247c6d32a5f936b8601b4741b519
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Currently we run grub-mkconfig then go and edit the generated output
in /boot/grub/grub.cfg to override the "root=" arguments to our label.
If another element like project-config's
finalise.d/99-fix-grub-timeout then goes and re-runs grub2-install, it
overwrites these changes.
Grub has GRUB_DEVICE which should override the root device. Let's try
using this. Maybe it wasn't around at the time, this code is
more-or-less unchanged from I7d83bb2b359e7a8c3858eca04c96e35cf4e1fe9e
Change-Id: Ibaaa81124098f3c6febe48e455d3e1cd0a5f1761
Avoid dangerous file parsing and object serialization libraries.
yaml.load is the obvious function to use but it is dangerous[1]
Because yaml.load return Python object may be dangerous if you
receive a YAML document from an untrusted source such as the
Internet. The function yaml.safe_load limits this ability to
simple Python objects like integers or lists.
In addition, Bandit flags yaml.load() as security risk so replace
all occurrences with yaml.safe_load(). Thus I replace yaml.load()
with yaml.safe_load()
[1]https://security.openstack.org/guidelines/dg_avoid-dangerous-input-parsing-libraries.html
Change-Id: I84640973fd9f45a69d2b21f6d594cd5bf10660a6
Closes-Bug: #1634265
Cleaning logs was split, some was done in the
img-functions.finalise_base, some was done in the base element.
The version unifies tidy up logs in the lib/img-functions.
Especially when building docker container images the base element
cannot be used. This patch removes about some hundreds KB of
useless logs in cases when the base element is not used.
Change-Id: I165bafb73daf9144c2f3a83930e85e8d8cf5fae3
Signed-off-by: Andreas Florath <andreas@florath.net>
