Commit graph

3444 commits

Author SHA1 Message Date
Ian Wienand
f1af76f228 Don't run unit tests from run_functests.sh
Change ec7f56c1b2 add added unit tests
in diskimage_builder/tests/functional (this is probably misnamed).

These are found and run by testr just as part of the normal "setup.py
test" run.  Don't run them as part of the functional tests (this
breaks "-h"/"-l" because it installs a virtualenv and runs tests, and
also is incorrect in the gate where it's creating a nested virtualenv
underneath the testing virtualenv).

Change-Id: I9908e080042d3026a198ba89eb653c6eff376d22
2017-02-02 15:20:24 +11:00
Ian Wienand
bfca36c772 Release 1.25.2
-----BEGIN PGP SIGNATURE-----
 
 iQEcBAABAgAGBQJYV1yqAAoJEBty/58O8cX8hLwIAKP66w6MdPN8PDgUOteui/Sx
 N0UFKJ9yR4GQOAP0NffPLjch5/g0iJLs3eFKOhtGC1LjbDjpVgjX8vW18ib8wBZK
 GemOZPF3uxg8FROrZF1vpoDy/cHgL1YV10hCnwdjN/r9rb8zOuSabqjW+Dennj2n
 fZ0SJfa8Owfudn3YxGuOymVb/wMtEloDmVGBEI1Y+h7osELCCDi3OXmwsA8qMsdl
 cTwbeugBs4PlOVbZUK/JKGuwIHKgPnDYzYu5KpXw77/MdjGT0fo5Tlq5AOBDI2sC
 9JOFEBDli4Ro05VwvI58ADMpvvOax+9EvOhLbB1dRPdZl21Iyb6gOdy2PUbFO0c=
 =aKxq
 -----END PGP SIGNATURE-----

Merge tag '1.25.2' into merge-branch

Release 1.25.2

Change-Id: I698bcf2e82117bd81649cd065a7af5cac85990c7
2017-02-02 11:20:00 +11:00
Jenkins
b4266001d0 Merge "Spec for changing the block device handling: partitioning" 2017-01-31 17:45:19 +00:00
Jenkins
47e1ab5c9e Merge "Create ubuntu/fedora test for pip-and-virtualenv" 2017-01-31 17:45:14 +00:00
Jenkins
7f08bd6234 Merge "Speed up chroot checking loop" 2017-01-31 17:42:04 +00:00
Jenkins
f054e5f24a Merge "debootstrap: avoid duplicate network configuration" 2017-01-31 17:39:28 +00:00
Ian Wienand
83bdfe179a Git ignore coverage stuff
One less piece of clutter when you're trying to merge ...

Change-Id: I794027280359f5d1e02d4f3b99222d35783075d5
2017-01-31 14:21:21 +11:00
Ian Wienand
b6e631360f Release 1.24.0
-----BEGIN PGP SIGNATURE-----
 
 iQEcBAABAgAGBQJYP+r7AAoJEBty/58O8cX8ZcMIAI7Yy2BfGqY/jfyLYNryATSu
 g78McMxAeOiPYII+KKyVkeWsChKJUafLZr/ppO95pm7ZLXeHQeC4MHDI68RmghIX
 HVXn4hPYzwlZ+Yr4hqaEf11H+vw0TDxo2a5bxlQZw+FCx9+3lkR60QxWf8a6QCxv
 nLqKIhtpwGi9mIyvAya+MaOmHMHqbNYgIBHKP0yXhTdxoHZoJWZeXN1rkelwAF9N
 TS1+KwIk+uemn1ZlqzYDWj8euyFyAv8L0WlF0ztCKe7TAFvNeMo26fLykeKHVDEo
 GC5R8MshiOtihPC972rNIponIzd62AetTlDxfK+gcUZO21pNTrlgdLX5Jn04fo0=
 =BpQc
 -----END PGP SIGNATURE-----

Merge tag '1.24.0' into merge-branch

Release 1.24.0

Change-Id: I23cfa6f8dc66bb6a1d957a49a5b468cfd47c005b
2017-01-31 14:14:19 +11:00
Jenkins
cb7173609d Merge "Remove hardcoded components" 2017-01-31 00:10:38 +00:00
Jenkins
619a0d736a Merge "Don't set base element path in run_functests.sh" 2017-01-27 16:40:48 +00:00
Jenkins
7421f61366 Merge "move post-install.d to finalize.d" 2017-01-27 16:40:15 +00:00
Jenkins
7f218fe10e Merge "Use %i instead of %I in dhcp-interface@.service" 2017-01-26 23:02:15 +00:00
Jenkins
7b8b673d15 Merge "Update hpssacli to ssacli in proliant-tools element" 2017-01-25 07:48:35 +00:00
Andreas Florath
ec7f56c1b2 Refactor: block-device handling (partitioning)
During the creation of a disk image (e.g. for a VM), there is the need
to create, setup, configure and afterwards detach some kind of storage
where the newly installed OS can be copied to or directly installed
in.

This patch implements partitioning handling.

Change-Id: I0ca6a4ae3a2684d473b44e5f332ee4225ee30f8c
Signed-off-by: Andreas Florath <andreas@florath.net>
2017-01-24 19:59:10 +00:00
Jenkins
29ac096e49 Merge "Bump fedora release to 25" 2017-01-24 16:36:19 +00:00
Jenkins
1a94a72965 Merge "Set grub device in /etc/default/grub" 2017-01-24 06:16:21 +00:00
Jenkins
fbe5f5a11a Merge "ironic-agent: use /sbin for modprobe" 2017-01-23 23:19:55 +00:00
Andrey Shestakov
11d384e2fa Add DIB_IPA_COMPRESS_CMD option
This option allows to specifiy command for compress built initramfs image
for ironic-agent element. This command can be specified with arguments and
should read raw data from stdin and write compressed data to stdout.
Default if "gzip".

Change-Id: I0fdd2ab91d7bf7aaaa6cdd5278e3902d44c5b883
2017-01-23 13:28:32 +00:00
Ian Wienand
ca60b2cf7d ironic-agent: use /sbin for modprobe
On redhat/fedora /sbin is a symlink to /usr/sbin, but not on all
platforms.  This was put in with
Ibf74dd1b2678ea76e0676711a7aa5ba6b88d5421

Change-Id: I7847b29503c3c07503430a7d85a5364911894c6c
Closes-bug: #1658297
2017-01-23 14:35:25 +11:00
Matthew Thode
dc8449bf02
move post-install.d to finalize.d
It looks like I installed the cleanup file in the wrong location.  Moving it
to the correct location and cleaning some more stuff up should allow for a
much smaller image, currently tested at 300M vs 490M.

Change-Id: I9d0a24b0fe59d4f2d38aa88ba47e4400e0476b46
2017-01-22 15:53:02 -06:00
Paul Belanger
90a347707c Bump fedora release to 25
Now that fedora-25 is released, bump fedora-minimal / fedora elements
to use it.

Change-Id: Ifdb05255e7a138b99099252fe17041fe4621b6f4
Depends-On: I3cc467a2d08486458b3f625f94ba969532f6cd04
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-01-21 16:00:54 -05:00
Jenkins
6c0eb3d905 Merge "Add get_elements; preserve backwards compat for expand_dependencies" into feature/v2 2017-01-20 10:25:54 +00:00
Jenkins
48bbbe6268 Merge "Fix coverage report" 2017-01-20 00:13:31 +00:00
Jenkins
75e5ab5393 Merge "Fix dhcp-all-interfaces for ubuntu-minimal xenial" 2017-01-19 21:11:40 +00:00
Jenkins
074a10fd74 Merge "Allow disto-specific mirror settings" 2017-01-19 21:00:56 +00:00
Andrey Shestakov
a45ad8eaf5 Fix dhcp-all-interfaces for ubuntu-minimal xenial
The start script of dhcp-all-interfaces currently requires ifup command.
ifup command provided by package ifupdown, which is not installed in
ubuntu-minimal Xenial.

This change adds ifupdown package for debian family, as it required for
dhcp-all-interfaces.

Closes-bug: #1647853

Change-Id: I6dfc0108ec067f350b22e9fb933b9e8d47b09fde
2017-01-19 17:29:43 +00:00
Ian Wienand
fb942b752d Use strings in package-installs follow output
I noticed in debugging that with python3 all the pkg-installs output
is preceeded by a b'foo', which suggests coding issues lurking.

The universal_newlines argument makes subprocess readline() returns a
decoded str (via locale) instead of a byte-string.  This clears up a
couple of points where we explicitly decode and cleans up the print
output.

Minor formatting cleanup of command and exit display.  Don't strip the
leading spaces so indents retain in the output

Change-Id: I2894f10a0c2fc618563641b9d106b716f4a544aa
2017-01-18 19:26:40 +11:00
Ian Wienand
0ceb138d87 Don't set base element path in run_functests.sh
DIB automatically includes it's base elements in the ELEMENT_PATH;
this double-set means the base elements are in the path twice, leading
to some tools that walk the element list (package-installs-squash, for
example) to do things twice.

Change-Id: Icf631427601cfd2abed436711cedb9e002fff8f9
2017-01-18 19:26:28 +11:00
Ian Wienand
baf3ced0c3 Fix coverage report
There's a few things going on here

Firstly, we need to install coverage tool in test-requirements

Secondly, .testr.conf has to use PYTHON because the coverage report
works by resetting PYTHON='coverage run ...'

Thirdly, because we call ourselves diskimage-builder but the python
module is diskimage_builder that seems to confuse things.  We need to
use "setup.py test" (note, that is different to "setup.py testr"!) to
use the PBR testr wrapper.  That exposes a --coverage-package-name
argument that calls the coverage tool with the right argument.

With this I got a coverage report for our unit tests

Change-Id: I9012e18eb7d01bee035140e70afa76c47c27eb01
2017-01-18 16:14:01 +11:00
Jenkins
f735864b78 Merge "Replace yaml.load() with yaml.safe_load()" 2017-01-18 04:08:12 +00:00
Ben Nemec
d20754f6ed Use %i instead of %I in dhcp-interface@.service
Per the bug report, %I results in -'s in the interface name being
replaced with /'s, which means when we try to look up the interface
by path in dhcp-all-interfaces.sh we end up at an invalid path.
Using %i instead should fix the problem.

See https://www.freedesktop.org/software/systemd/man/systemd.unit.html

Essentially what is happening is that we start with a name like
br-ex that wasn't escaped in the first place.  However, because of
the - it looks like it could have been escaped to systemd.  When
we use %I, which tries to unescape escaped values, it unescapes a
value that was already not escaped.

Change-Id: I434ed2e084d4477dc7a2b7827164586f8ea0c1e3
Closes-Bug: 1649409
2017-01-17 17:24:07 -06:00
Jenkins
359892aa4b Merge "update pkg-map entries for python3" 2017-01-17 02:33:00 +00:00
Jenkins
f3d0d6e8ec Merge "Remove yum chroot caching" 2017-01-17 01:10:41 +00:00
Paul Belanger
1f43432313 Remove hardcoded components
We can use ${DIB_DEBIAN_COMPONENTS} to get this information. We also
already set the mirror.

Change-Id: Idd11c2b7df1d247c6d32a5f936b8601b4741b519
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-01-16 18:03:00 -05:00
Jenkins
084937617f Merge "dib-lint: python3 compatibility fixes" 2017-01-16 19:36:32 +00:00
Ian Wienand
6c8d7432a6 Set grub device in /etc/default/grub
Currently we run grub-mkconfig then go and edit the generated output
in /boot/grub/grub.cfg to override the "root=" arguments to our label.

If another element like project-config's
finalise.d/99-fix-grub-timeout then goes and re-runs grub2-install, it
overwrites these changes.

Grub has GRUB_DEVICE which should override the root device.  Let's try
using this.  Maybe it wasn't around at the time, this code is
more-or-less unchanged from I7d83bb2b359e7a8c3858eca04c96e35cf4e1fe9e

Change-Id: Ibaaa81124098f3c6febe48e455d3e1cd0a5f1761
2017-01-16 09:53:56 +00:00
Luong Anh Tuan
ff8ae43265 Replace yaml.load() with yaml.safe_load()
Avoid dangerous file parsing and object serialization libraries.
yaml.load is the obvious function to use but it is dangerous[1]
Because yaml.load return Python object may be dangerous if you
receive a YAML document from an untrusted source such as the
Internet. The function yaml.safe_load limits this ability to
simple Python objects like integers or lists.

In addition, Bandit flags yaml.load() as security risk so replace
all occurrences with yaml.safe_load(). Thus I replace yaml.load()
with yaml.safe_load()

[1]https://security.openstack.org/guidelines/dg_avoid-dangerous-input-parsing-libraries.html

Change-Id: I84640973fd9f45a69d2b21f6d594cd5bf10660a6
Closes-Bug: #1634265
2017-01-16 15:07:05 +07:00
Jenkins
50941b13bc Merge "Update documented default Ubuntu version" 2017-01-15 23:31:57 +00:00
Andreas Florath
022d93ee82 Unify tidy up logs in lib/img-functions
Cleaning logs was split, some was done in the
img-functions.finalise_base, some was done in the base element.
The version unifies tidy up logs in the lib/img-functions.
Especially when building docker container images the base element
cannot be used. This patch removes about some hundreds KB of
useless logs in cases when the base element is not used.

Change-Id: I165bafb73daf9144c2f3a83930e85e8d8cf5fae3
Signed-off-by: Andreas Florath <andreas@florath.net>
2017-01-14 09:40:44 +00:00
Jenkins
1f75aea634 Merge "Handle failure of carrier check in dhcp-all-interfaces.sh" 2017-01-13 20:11:20 +00:00
Jenkins
56d0dfc27b Merge "Also check bin/ for tabs" 2017-01-13 18:33:32 +00:00
Jenkins
753ab9a019 Merge "Make DHCP timeout configurable" 2017-01-13 06:19:54 +00:00
Ian Wienand
812ded362d Also check bin/ for tabs
Add bin/ to dib-lint basic script checks

Change-Id: I92849ab608a6c792d790f041dc7614278c274d30
2017-01-13 15:45:56 +11:00
Jenkins
14957664d4 Merge "Fix Gentoo builds on Ubuntu 16.04 Xenial hosts" 2017-01-12 23:07:27 +00:00
Jenkins
b8a985fc02 Merge "Run dhcp-interface@.service after network.target" 2017-01-12 15:47:03 +00:00
Bob Fournier
f8eba14d99 Handle failure of carrier check in dhcp-all-interfaces.sh
As described in the bug, there are conditions with certain switches
in which the interface is 'admin down'ed during initialization.
Doing a 'cat' on /sys/class/net/<interface>/carrier when it is
'admin down'ed produces an 'Invalid Argument' error and the script
terminates.  What this fix does is ignore failures of the 'cat'
operation (by '|| echo 0') and place the link up inside the retry
loop.

Change-Id: I4f098aa5078b8482681394a3e9a6b17ed4bd4451
Closes-Bug: 1654046
2017-01-12 10:36:43 -05:00
Matthew Thode
6c5234e162
Fix Gentoo builds on Ubuntu 16.04 Xenial hosts
Xenial's bind of /dev into the chroot includes /dev/shm which is in
use by the host.  An alternitive fix for this would be to use rbind
to recursivly bind mount /dev instead of just the base bind of /dev

Change-Id: I2c0f70afd1e82dd52a522f0dd2b3ea618b30b6c6
2017-01-10 10:34:12 -06:00
Jenkins
7fc4856c6a Merge "Update component docs to refer to ironic-agent" 2017-01-10 04:27:44 +00:00
Jenkins
4e99a9c9e2 Merge "Change "Openstack" to "OpenStack"" 2017-01-10 02:34:25 +00:00
Ben Nemec
ccd00b10b2 Make DHCP timeout configurable
As noted in the bug, there may be circumstances where a longer
timeout than the current default is needed.  This patch allows users
to tune this timeout for their environment if need be.

Change-Id: I173f3dad684894fbc3c27dece5ae15b5f63bae5a
Closes-Bug: 1654027
2017-01-04 15:41:04 -06:00