Commit Graph

2786 Commits

Author SHA1 Message Date
Pierre Riteau
0576d20d49 Update documented default Ubuntu version
Since commit fd5fbdd4b5 xenial is the
version used by default.

Change-Id: I18e01d806635539b2d6c8a4e6b2d25460647c910
2016-12-21 11:49:27 +00:00
Ian Wienand
2978ff885b Revert Xenial to Python 2
There are issues with pip packages and a python3 only Xenial systems.
This is occuring after Ie609de51cc5fcde701296c9474e315981d9778a2.

We believe the issue is with VIRTUAL_ENV being set within the chroot
and messing up pip installs
(Icdb769541eee9793f261b4b8ec563be76ee13fe2) but a full solution is not
yet clear.

For now, set Xenial to ensure we use python2.  Install the package for
the ubuntu element (75-debian-minimal-baseinstall will install python2
for the minimal elements).

Change-Id: Id403919b0af93b375a900186c01a0d3a3bdfafea
2016-12-21 20:46:13 +11:00
Antoine Musso
2209d34372 debootstrap: avoid duplicate network configuration
On Debian network configuration can be done via /etc/network/interfaces.
It can accept a statement to load additional files, which varied in
history:

Wheezy only supports 'source' (see b822581)
Jessie supports 'source-directory' and comes with the statement by
default.

However since 754dd05 we inconditionally inject 'source', thus on Jessie
the configuration ends up with:

 source-directory /etc/network/interfaces.d/*
 source /etc/network/interfaces.d/*

When networking is started, 'ifup -a' parses the list of interfaces
twice. When configured with dhcp, that causes two dhclient to spawn
which might conflict with each other.

Inject the source statement only if there is neither a source or
source-directory with the same path.

Change-Id: Iefa9c9584f676e50481c621b4111eded3125a50b
2016-12-21 10:23:32 +01:00
Jenkins
e32688e71e Merge "Start func testing on centos-minimal again" 2016-12-21 04:13:42 +00:00
Jenkins
9ee7acc5ee Merge "Increase func testing for ubuntu-minimal element" 2016-12-21 04:03:57 +00:00
Paul Belanger
9d9a1e3000 Start func testing on centos-minimal again
Because we run this image in openstack-infra, we want to increase our
test coverage to help avoid potential breaks to our CI systems.

Change-Id: I26405e3f7465654075278ec35b5e0da1338bb45e
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2016-12-21 11:52:14 +11:00
Paul Belanger
d9dcb3fe99 Increase func testing for ubuntu-minimal element
Since we still run these 3 version of ubuntu-minimal elements in
openstack-infra, also run functional testing for them.

Trusty and xenial will be in voting gate, precise added as skipped for
non-voting.

Add the default skip/run status to the "-l" output just to confirm
this too.

Change-Id: Icfbfd0cb7d9acae824972474b77e2fe0486c4f69
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2016-12-21 11:50:47 +11:00
Jenkins
c73e6b5ab1 Merge "Set grub timeout default" 2016-12-20 22:45:07 +00:00
Jenkins
a6f7e30055 Merge "Add squashfs output image format" 2016-12-20 03:50:29 +00:00
Jenkins
9351618e70 Merge "Add output image format tgz support" 2016-12-20 03:48:24 +00:00
Jenkins
0a34d135fb Merge "Switch py34 tests to py35" 2016-12-20 03:41:25 +00:00
Jenkins
1b7c415b4a Merge "Activate virtualenv in disk-image-create" into feature/v2 2016-12-20 03:41:19 +00:00
Jenkins
f9055b938d Merge "Support sysv init system used by Debian Wheezy" 2016-12-20 03:31:31 +00:00
Jenkins
05de769154 Merge "Update our package classifiers" 2016-12-20 03:03:52 +00:00
Ian Wienand
4585955a8b Remove yum chroot caching
Every run we are doing a full tar.gz of the chroot environment that
never gets used.

It's not suitable for CI since we use fresh images each time there.

The cache in general isn't really isn't a very safe thing to have
around, because there's no invalidation procedure and no real way to
make one -- we've no guarantee that a new chroot build even moments
after a previous one wouldn't bring in or different packages, etc (of
course this is *unlikely*, but the longer you go between builds the
worse the problem becomes.  Also, tons of packages get installed after
this not from any cache, so potential speed-up is rather marginal.

Debian turned this off with I58fc485aacacaa17243bf9ce760ed91256d1f182.
However, given the reasons above and it's complete lack of testing, I
don't see this as useful.

If we really want this type of thing, I think we should come up with a
way to use a persistent external yum/dnf cache that yum/dnf keeps in
sync with it's usual invalidation rules.

Change-Id: I66789c35db75c41bc45ea1ad2e26f87456de4e4d
2016-12-20 13:56:46 +11:00
Ian Wienand
61087d33e9 Set grub timeout default
Set the grub timeout to 5 seconds by default, and add notes on how to
update this.  This will stop infra having to carry an element that
goes and rewrites the grub configuration.

Change-Id: I556b3f48eff1b67ee8c4b9b64f749af95100fb99
2016-12-20 11:46:22 +11:00
Gregory Haynes
54dd54d81d Switch py34 tests to py35
Py35 is now commonly available on things like -infra test nodes, lets
use that instead.

Change-Id: I554ec84d709a9937df5ec3c2fa7eab8f158cf64e
2016-12-19 23:22:24 +00:00
Jenkins
46af2452f7 Merge "Generate ssh-hostkeys on boot for ironic agent" 2016-12-19 22:55:30 +00:00
Gregory Haynes
5da7574aee Add get_elements; preserve backwards compat for expand_dependencies
expand_dependencies() was a public interface so we should try and
preserve backwards compat. However, since the interface is really
broken, add a new exported function "get_elements" that instack can
switch to.  This returns the canonical list of elements without
duplicates, and gives the path to each element too.

This highlighted that the unit tests were really a bit wrong.  They're
testing inner functions when we have an "API" in the get_elements()
function.  Convert all unit-tests to use this function instead.  Since
this is a library call, convert the sys.exit() calls to raised
exceptions.

Refactor the variable output into a separate function so we can do a
sanity check on it.

The added flake8 ignores are for the "over-indented for ... indent"
which happens a lot with these new longer lines.  Most other projects
ignore them.

This is an alternative proposal to
I15609389c18adf3017220fc94552514d195b323a

Change-Id: If97bcd21e45de1b5ed91023fdc441a4617051a6b
2016-12-20 08:18:59 +11:00
Gregory Haynes
7c4c304f8a Update our package classifiers
Our setuptools action classifiers are woefully out of date, notably: we
are no longer alpha and we support python3.

Change-Id: I2425152129406e22073936275761bd5d850903fb
2016-12-19 18:55:44 +00:00
Jenkins
029b6dd3d5 Merge "set default DIB_PYTHON_VERSION=2 for rhel7" 2016-12-19 18:07:31 +00:00
Jenkins
bbe81c30be Merge "Switch to openSUSE Leap 42.2 release by default" 2016-12-19 08:40:32 +00:00
Noam Angel
4789aa317e set default DIB_PYTHON_VERSION=2 for rhel7
python 3 not exist on minimal/KVM guest image. set default python version
2 for rhel7 also.

Change-Id: Icbc10e742da8dded25625a1eed0a79065702837d
2016-12-19 07:31:34 +00:00
Tristan Cacqueray
9d13084c41 Add squashfs output image format
The squashfs format brings a couple of advantages over the other
formats. Image is often an order of magnitude smaller and it can
be used natively, either as an initrd, either with loop mount.

Change-Id: If72940b0c4dafb2504c52dd0429a8eb3f8305751
2016-12-19 07:21:39 +00:00
Jenkins
e0a2163157 Merge "Fix bootloader element on ppc" 2016-12-19 04:55:05 +00:00
Jenkins
b8e60d52ea Merge "FIx the DIB_CLOUD_INIT_ALLOW_SSH_PWAUTH variable name in README file" 2016-12-19 04:10:17 +00:00
Jenkins
9e9425ba3a Merge "Install dracut-generic-config package" 2016-12-19 02:58:55 +00:00
Paul Belanger
da41ee6012 Add output image format tgz support
We now support tgz (tar.gz) as an output format.

Change-Id: Iadec92f2f96c3f904f28bd49f87ffc7d48ef7bd7
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2016-12-17 16:41:42 -05:00
Ian Wienand
f6a02fbdb9 Install dracut-generic-config package
dracut has a "hostonly" mode where it builds an initramfs that is
suitable for booting the system it is building on.  This is on by
default, but obviously in our nested multi-platform chroot situation
this is fraught with danger.

As highlighted by [1] our builds were inadvertently turning off
"hostonly" mode when the mountpoints in the chroot were not found.
The CentOS 7.3 behaviour change broke this and we ended up with an
initramfs with no file-system modules.

Iaf2a1e8470f642bfaaaad3f9b7f26cfc8cc445c9 introduced a regeneration of
the initramfs, which I think does work as described because it runs in
the loopback device.

However, dracut includes a package that installs configuration
overrides to build a generic initramfs.  This is really what we want,
and should solve the problem no matter where the initramfs is created.

Add this package into yum-minimal and remove the extra re-create call
which should not be necessary.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1405238

Change-Id: I5d203f2abe743cb23a44d449850e692a948e7871
2016-12-17 16:37:55 +00:00
Dirk Mueller
54f4e12765 Switch to openSUSE Leap 42.2 release by default
openSUSE 13.1 was discontinued on Feb 3rd, 2016, so defaulting
to it doesn't make sense (see https://en.opensuse.org/Lifetime).

Leap 42.2 is the most current release that is supported by
disk-image-builder and being tested in a 3rd party ci.

Enable functests for it to ensure we're not regressing again.
Moved to non-voting gate first.

Depends-On: Iff495b3cd0b6c3558c44cf4883651eca67b572d6
Change-Id: Iae6cd34a5853f1e309861c554d94d8595cbd9993
2016-12-17 10:46:17 +01:00
Jeffrey Zhang
ba93e63145 FIx the DIB_CLOUD_INIT_ALLOW_SSH_PWAUTH variable name in README file
DIB_CLOUD_INIT_ALLOW_SSH_PWAUTH is the correct one.

Change-Id: I3813cadf21327fcc8d960deb43df2309d812a05a
2016-12-17 11:51:32 +08:00
Paul Belanger
6d82543682 Add ubuntu-precise support to dib-python
Change-Id: I2796da88d839ed49ec28ae7b139ede04af51f068
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2016-12-16 11:27:45 -05:00
Mikhail S Medvedev
9d7725b475 Fix bootloader element on ppc
For some reason [1] introduced -m option without ever checking that the
mapping exists. Because there is no grub-ieee1275 mapping anywhere (not
in base, not in bootloader), pkg-map fails. So stop using the mapping in
package-install of grub-ieee1275 on ppc.

There is another patch that tries to solve the same bug by adding the
mapping [2]. I think it is better to undo the breakage introduced in [1]
first, and then, if various distributions have differing names for the
package, introduce various mappings. My reasoning is that at the moment
this element is broken for all ppc64 distributions. This patch would
fix it for some (namely, Ubuntu). Then we can add mappings as tests
are done for other distributions.

[1] Ibca43173c30c2a74a73a2e2d9dd6d6d832c62694
[2] Id2b0f63a7015f883070fd59b79fd96a1c024858a

Change-Id: I8425876c26e9e416c8ce2f53a4e38d26b4208633
Closes-Bug: #1624021
2016-12-15 18:10:29 -06:00
Gregory Haynes
6a40b60717 Create ubuntu/fedora test for pip-and-virtualenv
This element does some funky stuff WRT python2 vs python3 so lets get
some multi-distro testing in place.

Change-Id: I1e3c3bfa0a109419d4cbee7fa32a18392b7e1a93
2016-12-15 18:00:41 +00:00
Ian Wienand
a72645f431 Recreate initramfs within loopback image
dracut has a loop [1] where it probes top-level directories, tries to
find what block device they are on, then determines the file-system of
that block device.  It then puts those file-system modules into the
initramfs for boot.

Since we install the kernel package during the chroot phase, / there
is not a block device and thus this loop matches nothing and we end up
with no file-system modules in the initramfs.  This results in a very
annoying silent boot hang.

By moving re-generation of dracut into finalise.d phase, we run inside
the final image where / is the loop-device; the root file-system gets
detected correctly and the ext4 module is included correctly.

[1] http://git.kernel.org/cgit/boot/dracut/dracut.git/tree/dracut.sh?h=RHEL-7#n1041

Change-Id: Iaf2a1e8470f642bfaaaad3f9b7f26cfc8cc445c9
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2016-12-15 12:37:00 -05:00
Peter Stachowski
280896759a Pip install as 10- incompatible with 05-heat-cfntools
Tripleo-image-elements have an install.d file '05-heat-cfntools' that runs
the following command:

virtualenv --setuptools $VENV

With the recent change to diskimage-builder (moving the install of pip
and virtualenv to the 10- range) virtualenv is no longer available for
this elementr; as a side-effect, the trove kick-start command is now
broken and gate jobs are failing.

The solutions is to move the (now) 10-install-pip to 04-install-pip.
This should still alleviate the race condition that
https://review.openstack.org/#/c/408277/ attempted to fix, as all
*-package-installs files are 00-, 01- or 02-.

Change-Id: Ia4e01f00c4c5e9a2087df1e2a91d9154480a0422
Closes-Bug: #1650008
2016-12-14 20:50:00 +00:00
Saverio Proto
3417bd6298 Support sysv init system used by Debian Wheezy
Change-Id: Ia6ca11ab78f16a51aba7b627c72c615c184d338d
2016-12-14 15:53:04 +01:00
Markos Chandras
339ecee2b2 elements: dib-python: Add python2 as the default version for openSUSE
Commit 6278371eaa13("Make dib-python use the default python for distro")
added default python version for various distros but it missed openSUSE
which leads to build failures since the openSUSE elements are pulling
python2 packages. Add openSUSE to the list of python2 distributions
until python3 support for the openSUSE elements is in place.

Change-Id: I95f1fa849a22607c430387a2a915f9d19c9c209f
2016-12-14 09:38:45 +00:00
Jenkins
5a64c9e9cf Merge "Fix pip-and-virtualenv to work with python3" 2016-12-14 07:18:32 +00:00
Jenkins
d1ca1b1957 Merge "Catch errors in DIB_INIT_SYSTEM export" 2016-12-14 07:14:10 +00:00
Gregory Haynes
3e777cd8f4 Fix pip-and-virtualenv to work with python3
We are explicitly calling python in this element which does not work on
systems which only have python3.

Change-Id: Ia730850a48e2478fd5461710a9d2619408725cd8
2016-12-14 17:14:02 +11:00
Jenkins
8565867734 Merge "Allow package-installs to parse DIB_PYTHON_VERSION" 2016-12-14 02:23:47 +00:00
Jenkins
1c0ad1d4b0 Merge "Disable centos6 testing" 2016-12-14 01:14:03 +00:00
Gregory Haynes
ecae8dcbd5 Allow package-installs to parse DIB_PYTHON_VERSION
Now that we are explicit about what python version we intend to use
for dib we can have package installs optionally install packages
depending on this.  Add a new dib_python_version that matches on the
DIB_PYTHON_VERSION string set by dib-python.

Co-Authored-By: Adam Harwell <flux.adam@gmail.com>
Change-Id: I70659aab7d12924bdb9bc0489a7f02d5fd0dbb39
2016-12-14 12:13:40 +11:00
Jenkins
d981d6b0cc Merge "Add install-types as pip-and-virtualenv dep" 2016-12-14 00:57:45 +00:00
Jenkins
95b6c1b00b Merge "Move pip-and-virtualenv source install to 10-" 2016-12-14 00:36:19 +00:00
Gregory Haynes
a1dfe505ea Add install-types as pip-and-virtualenv dep
This element supports install-types so we need to depend on it.

Change-Id: Ib1193673ca1c1a1cafe0006eabef981a01c87781
2016-12-14 11:21:11 +11:00
Gregory Haynes
c3293b3f7e Disable centos6 testing
Centos6 is no longer reasonable to expect to function with openstack,
and infra does not host a mirror for it.

Change-Id: I95ccf5840807fee73d6e78d596c82709e476bb3a
2016-12-14 11:10:54 +11:00
Gregory Haynes
6ddbb457d6 Move pip-and-virtualenv source install to 10-
We currently have this as a 01- script which causes it to race with
package-installs (the deps are installed after the script runs).

Change-Id: I7b04b4c186eaae783b8e2bda1aa724c0d7823eab
2016-12-14 11:07:12 +11:00
Jenkins
b34c5db441 Merge "Update sysctl-write-value to do conflict checking" 2016-12-14 00:04:55 +00:00