Commit Graph

1941 Commits

Author SHA1 Message Date
Matthew Thode
6c5234e162
Fix Gentoo builds on Ubuntu 16.04 Xenial hosts
Xenial's bind of /dev into the chroot includes /dev/shm which is in
use by the host.  An alternitive fix for this would be to use rbind
to recursivly bind mount /dev instead of just the base bind of /dev

Change-Id: I2c0f70afd1e82dd52a522f0dd2b3ea618b30b6c6
2017-01-10 10:34:12 -06:00
Ben Nemec
ccd00b10b2 Make DHCP timeout configurable
As noted in the bug, there may be circumstances where a longer
timeout than the current default is needed.  This patch allows users
to tune this timeout for their environment if need be.

Change-Id: I173f3dad684894fbc3c27dece5ae15b5f63bae5a
Closes-Bug: 1654027
2017-01-04 15:41:04 -06:00
Ben Nemec
5bed4a6d5e Run dhcp-interface@.service after network.target
When we configure dhcp interfaces before network.target has run,
network.target will try to bring up those interfaces a second time
after our service does so.  This causes two issues - first, the
network target will always fail because it can't bring up an
interface that is already up, and second, when configuring interfaces
that don't actually have an available DHCP server it will result in
a five minute delay waiting for DHCP on those interfaces.  This will
also cause the network target to fail and is an unnecessary delay.

By moving the dhcp-interface service to run after the network
target we avoid both of these problems.  network.target will still
bring up the interfaces on subsequent boots.  This could result in
the five minute delay happening on reboots, but the expected use
case for interfaces without DHCP is that they would be configured
statically on initial deployment so this should be a minor issue.

The dhcp-interface service is also configured to run before the
network-online target so that services which depend on the network
actually being available will not race the DHCP process.

A snippet from /var/log/messages on a node with this patch applied
is included in the bug to demonstrate the behavior described above.

Change-Id: I5cfabf20f920beea52abf4c42362b6f6ac0b37c4
Closes-Bug: 1653812
2017-01-04 10:49:59 -06:00
Cady_Chen
1d4bb04853 Change "Openstack" to "OpenStack"
According to the word choice convention in
http://docs.openstack.org/contributor-guide/writing-style/word-choice.html
We should use OpenStack instead of Openstack.

Change-Id: I66f0bf9eb81593220eb0000fe8192c478e8d075d
2016-12-30 01:15:27 +00:00
Matthew Thode
d6fe34cc59
update pkg-map entries for python3
Map python3 packages properly, or else we get invalid package names.

Change-Id: I7af8621c856c689840a11d3cb9d094f02a0b9593
2016-12-23 20:17:34 -06:00
Gregory Haynes
7603f97cad Revert "Revert Xenial to Python 2"
We landed the fix for this in
Icdb769541eee9793f261b4b8ec563be76ee13fe2.

This reverts commit 2978ff885b.

Change-Id: Iecfc41ab2aad57bc4f6f86a13810b534d19a8fd5
2016-12-22 14:26:35 +11:00
Andreas Florath
7337c9e706 dib-lint: python3 compatibility fixes
When using up to date distributions for dib development, pep8
installs using python3.  This patch fixes the problem, that
not the complete dib-lint (which is called) is compatible
with python3.

Change-Id: I417d03746edb4d34011b997edf8b5b9662ea6f09
Signed-off-by: Andreas Florath <andreas@florath.net>
2016-12-21 20:29:26 +00:00
Pierre Riteau
0576d20d49 Update documented default Ubuntu version
Since commit fd5fbdd4b5 xenial is the
version used by default.

Change-Id: I18e01d806635539b2d6c8a4e6b2d25460647c910
2016-12-21 11:49:27 +00:00
Ian Wienand
2978ff885b Revert Xenial to Python 2
There are issues with pip packages and a python3 only Xenial systems.
This is occuring after Ie609de51cc5fcde701296c9474e315981d9778a2.

We believe the issue is with VIRTUAL_ENV being set within the chroot
and messing up pip installs
(Icdb769541eee9793f261b4b8ec563be76ee13fe2) but a full solution is not
yet clear.

For now, set Xenial to ensure we use python2.  Install the package for
the ubuntu element (75-debian-minimal-baseinstall will install python2
for the minimal elements).

Change-Id: Id403919b0af93b375a900186c01a0d3a3bdfafea
2016-12-21 20:46:13 +11:00
Antoine Musso
2209d34372 debootstrap: avoid duplicate network configuration
On Debian network configuration can be done via /etc/network/interfaces.
It can accept a statement to load additional files, which varied in
history:

Wheezy only supports 'source' (see b822581)
Jessie supports 'source-directory' and comes with the statement by
default.

However since 754dd05 we inconditionally inject 'source', thus on Jessie
the configuration ends up with:

 source-directory /etc/network/interfaces.d/*
 source /etc/network/interfaces.d/*

When networking is started, 'ifup -a' parses the list of interfaces
twice. When configured with dhcp, that causes two dhclient to spawn
which might conflict with each other.

Inject the source statement only if there is neither a source or
source-directory with the same path.

Change-Id: Iefa9c9584f676e50481c621b4111eded3125a50b
2016-12-21 10:23:32 +01:00
Jenkins
9ee7acc5ee Merge "Increase func testing for ubuntu-minimal element" 2016-12-21 04:03:57 +00:00
Paul Belanger
d9dcb3fe99 Increase func testing for ubuntu-minimal element
Since we still run these 3 version of ubuntu-minimal elements in
openstack-infra, also run functional testing for them.

Trusty and xenial will be in voting gate, precise added as skipped for
non-voting.

Add the default skip/run status to the "-l" output just to confirm
this too.

Change-Id: Icfbfd0cb7d9acae824972474b77e2fe0486c4f69
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2016-12-21 11:50:47 +11:00
Jenkins
c73e6b5ab1 Merge "Set grub timeout default" 2016-12-20 22:45:07 +00:00
Jenkins
f9055b938d Merge "Support sysv init system used by Debian Wheezy" 2016-12-20 03:31:31 +00:00
Ian Wienand
4585955a8b Remove yum chroot caching
Every run we are doing a full tar.gz of the chroot environment that
never gets used.

It's not suitable for CI since we use fresh images each time there.

The cache in general isn't really isn't a very safe thing to have
around, because there's no invalidation procedure and no real way to
make one -- we've no guarantee that a new chroot build even moments
after a previous one wouldn't bring in or different packages, etc (of
course this is *unlikely*, but the longer you go between builds the
worse the problem becomes.  Also, tons of packages get installed after
this not from any cache, so potential speed-up is rather marginal.

Debian turned this off with I58fc485aacacaa17243bf9ce760ed91256d1f182.
However, given the reasons above and it's complete lack of testing, I
don't see this as useful.

If we really want this type of thing, I think we should come up with a
way to use a persistent external yum/dnf cache that yum/dnf keeps in
sync with it's usual invalidation rules.

Change-Id: I66789c35db75c41bc45ea1ad2e26f87456de4e4d
2016-12-20 13:56:46 +11:00
Ian Wienand
61087d33e9 Set grub timeout default
Set the grub timeout to 5 seconds by default, and add notes on how to
update this.  This will stop infra having to carry an element that
goes and rewrites the grub configuration.

Change-Id: I556b3f48eff1b67ee8c4b9b64f749af95100fb99
2016-12-20 11:46:22 +11:00
Jenkins
46af2452f7 Merge "Generate ssh-hostkeys on boot for ironic agent" 2016-12-19 22:55:30 +00:00
Jenkins
029b6dd3d5 Merge "set default DIB_PYTHON_VERSION=2 for rhel7" 2016-12-19 18:07:31 +00:00
Jenkins
bbe81c30be Merge "Switch to openSUSE Leap 42.2 release by default" 2016-12-19 08:40:32 +00:00
Noam Angel
4789aa317e set default DIB_PYTHON_VERSION=2 for rhel7
python 3 not exist on minimal/KVM guest image. set default python version
2 for rhel7 also.

Change-Id: Icbc10e742da8dded25625a1eed0a79065702837d
2016-12-19 07:31:34 +00:00
Jenkins
e0a2163157 Merge "Fix bootloader element on ppc" 2016-12-19 04:55:05 +00:00
Jenkins
b8e60d52ea Merge "FIx the DIB_CLOUD_INIT_ALLOW_SSH_PWAUTH variable name in README file" 2016-12-19 04:10:17 +00:00
Jenkins
9e9425ba3a Merge "Install dracut-generic-config package" 2016-12-19 02:58:55 +00:00
Ian Wienand
f6a02fbdb9 Install dracut-generic-config package
dracut has a "hostonly" mode where it builds an initramfs that is
suitable for booting the system it is building on.  This is on by
default, but obviously in our nested multi-platform chroot situation
this is fraught with danger.

As highlighted by [1] our builds were inadvertently turning off
"hostonly" mode when the mountpoints in the chroot were not found.
The CentOS 7.3 behaviour change broke this and we ended up with an
initramfs with no file-system modules.

Iaf2a1e8470f642bfaaaad3f9b7f26cfc8cc445c9 introduced a regeneration of
the initramfs, which I think does work as described because it runs in
the loopback device.

However, dracut includes a package that installs configuration
overrides to build a generic initramfs.  This is really what we want,
and should solve the problem no matter where the initramfs is created.

Add this package into yum-minimal and remove the extra re-create call
which should not be necessary.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1405238

Change-Id: I5d203f2abe743cb23a44d449850e692a948e7871
2016-12-17 16:37:55 +00:00
Dirk Mueller
54f4e12765 Switch to openSUSE Leap 42.2 release by default
openSUSE 13.1 was discontinued on Feb 3rd, 2016, so defaulting
to it doesn't make sense (see https://en.opensuse.org/Lifetime).

Leap 42.2 is the most current release that is supported by
disk-image-builder and being tested in a 3rd party ci.

Enable functests for it to ensure we're not regressing again.
Moved to non-voting gate first.

Depends-On: Iff495b3cd0b6c3558c44cf4883651eca67b572d6
Change-Id: Iae6cd34a5853f1e309861c554d94d8595cbd9993
2016-12-17 10:46:17 +01:00
Jeffrey Zhang
ba93e63145 FIx the DIB_CLOUD_INIT_ALLOW_SSH_PWAUTH variable name in README file
DIB_CLOUD_INIT_ALLOW_SSH_PWAUTH is the correct one.

Change-Id: I3813cadf21327fcc8d960deb43df2309d812a05a
2016-12-17 11:51:32 +08:00
Paul Belanger
6d82543682 Add ubuntu-precise support to dib-python
Change-Id: I2796da88d839ed49ec28ae7b139ede04af51f068
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2016-12-16 11:27:45 -05:00
Mikhail S Medvedev
9d7725b475 Fix bootloader element on ppc
For some reason [1] introduced -m option without ever checking that the
mapping exists. Because there is no grub-ieee1275 mapping anywhere (not
in base, not in bootloader), pkg-map fails. So stop using the mapping in
package-install of grub-ieee1275 on ppc.

There is another patch that tries to solve the same bug by adding the
mapping [2]. I think it is better to undo the breakage introduced in [1]
first, and then, if various distributions have differing names for the
package, introduce various mappings. My reasoning is that at the moment
this element is broken for all ppc64 distributions. This patch would
fix it for some (namely, Ubuntu). Then we can add mappings as tests
are done for other distributions.

[1] Ibca43173c30c2a74a73a2e2d9dd6d6d832c62694
[2] Id2b0f63a7015f883070fd59b79fd96a1c024858a

Change-Id: I8425876c26e9e416c8ce2f53a4e38d26b4208633
Closes-Bug: #1624021
2016-12-15 18:10:29 -06:00
Gregory Haynes
6a40b60717 Create ubuntu/fedora test for pip-and-virtualenv
This element does some funky stuff WRT python2 vs python3 so lets get
some multi-distro testing in place.

Change-Id: I1e3c3bfa0a109419d4cbee7fa32a18392b7e1a93
2016-12-15 18:00:41 +00:00
Ian Wienand
a72645f431 Recreate initramfs within loopback image
dracut has a loop [1] where it probes top-level directories, tries to
find what block device they are on, then determines the file-system of
that block device.  It then puts those file-system modules into the
initramfs for boot.

Since we install the kernel package during the chroot phase, / there
is not a block device and thus this loop matches nothing and we end up
with no file-system modules in the initramfs.  This results in a very
annoying silent boot hang.

By moving re-generation of dracut into finalise.d phase, we run inside
the final image where / is the loop-device; the root file-system gets
detected correctly and the ext4 module is included correctly.

[1] http://git.kernel.org/cgit/boot/dracut/dracut.git/tree/dracut.sh?h=RHEL-7#n1041

Change-Id: Iaf2a1e8470f642bfaaaad3f9b7f26cfc8cc445c9
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2016-12-15 12:37:00 -05:00
Peter Stachowski
280896759a Pip install as 10- incompatible with 05-heat-cfntools
Tripleo-image-elements have an install.d file '05-heat-cfntools' that runs
the following command:

virtualenv --setuptools $VENV

With the recent change to diskimage-builder (moving the install of pip
and virtualenv to the 10- range) virtualenv is no longer available for
this elementr; as a side-effect, the trove kick-start command is now
broken and gate jobs are failing.

The solutions is to move the (now) 10-install-pip to 04-install-pip.
This should still alleviate the race condition that
https://review.openstack.org/#/c/408277/ attempted to fix, as all
*-package-installs files are 00-, 01- or 02-.

Change-Id: Ia4e01f00c4c5e9a2087df1e2a91d9154480a0422
Closes-Bug: #1650008
2016-12-14 20:50:00 +00:00
Saverio Proto
3417bd6298 Support sysv init system used by Debian Wheezy
Change-Id: Ia6ca11ab78f16a51aba7b627c72c615c184d338d
2016-12-14 15:53:04 +01:00
Markos Chandras
339ecee2b2 elements: dib-python: Add python2 as the default version for openSUSE
Commit 6278371eaa13("Make dib-python use the default python for distro")
added default python version for various distros but it missed openSUSE
which leads to build failures since the openSUSE elements are pulling
python2 packages. Add openSUSE to the list of python2 distributions
until python3 support for the openSUSE elements is in place.

Change-Id: I95f1fa849a22607c430387a2a915f9d19c9c209f
2016-12-14 09:38:45 +00:00
Jenkins
5a64c9e9cf Merge "Fix pip-and-virtualenv to work with python3" 2016-12-14 07:18:32 +00:00
Jenkins
d1ca1b1957 Merge "Catch errors in DIB_INIT_SYSTEM export" 2016-12-14 07:14:10 +00:00
Gregory Haynes
3e777cd8f4 Fix pip-and-virtualenv to work with python3
We are explicitly calling python in this element which does not work on
systems which only have python3.

Change-Id: Ia730850a48e2478fd5461710a9d2619408725cd8
2016-12-14 17:14:02 +11:00
Jenkins
8565867734 Merge "Allow package-installs to parse DIB_PYTHON_VERSION" 2016-12-14 02:23:47 +00:00
Gregory Haynes
ecae8dcbd5 Allow package-installs to parse DIB_PYTHON_VERSION
Now that we are explicit about what python version we intend to use
for dib we can have package installs optionally install packages
depending on this.  Add a new dib_python_version that matches on the
DIB_PYTHON_VERSION string set by dib-python.

Co-Authored-By: Adam Harwell <flux.adam@gmail.com>
Change-Id: I70659aab7d12924bdb9bc0489a7f02d5fd0dbb39
2016-12-14 12:13:40 +11:00
Jenkins
d981d6b0cc Merge "Add install-types as pip-and-virtualenv dep" 2016-12-14 00:57:45 +00:00
Jenkins
95b6c1b00b Merge "Move pip-and-virtualenv source install to 10-" 2016-12-14 00:36:19 +00:00
Gregory Haynes
a1dfe505ea Add install-types as pip-and-virtualenv dep
This element supports install-types so we need to depend on it.

Change-Id: Ib1193673ca1c1a1cafe0006eabef981a01c87781
2016-12-14 11:21:11 +11:00
Gregory Haynes
6ddbb457d6 Move pip-and-virtualenv source install to 10-
We currently have this as a 01- script which causes it to race with
package-installs (the deps are installed after the script runs).

Change-Id: I7b04b4c186eaae783b8e2bda1aa724c0d7823eab
2016-12-14 11:07:12 +11:00
Jenkins
b34c5db441 Merge "Update sysctl-write-value to do conflict checking" 2016-12-14 00:04:55 +00:00
Jenkins
a52a82036e Merge "DIB element to support cinder local attach/detach functionality" 2016-12-13 23:58:36 +00:00
Jenkins
8ddbc6425d Merge "Make dib-python use the default python for distro" 2016-12-13 23:28:50 +00:00
Ben Nemec
bf5af6155c Don't set the executable bit on dhcp-interface@.service
systemd doesn't like it when service files have the executable bit
so this causes it to spam the journal with messages like:

Configuration file /usr/lib/systemd/system/dhcp-interface@.service is
marked executable. Please remove executable permission bits.
Proceeding anyway.

Removing the executable bit from the install permissions should
eliminate those messages.

Change-Id: Ie1bc39465b3fcb55dcda5cee9e46a128a6ccffcb
2016-12-12 10:55:03 -06:00
Gregory Haynes
6278371eaa Make dib-python use the default python for distro
Right now dib-python works by trying to find any python on a system in
an order of precedence. A much better way is if we are explicit about
the python we intend to be there which will allow us to make better
decisions in other elements (such as allowing for package-installs to
take into account DIB_PYTHON_VERSION) as well as allow for users to
specify a preferred python version.

Co-Authored-By: Adam Harwell <flux.adam@gmail.com>
Change-Id: Ie609de51cc5fcde701296c9474e315981d9778a2
2016-12-09 09:25:37 -08:00
Gregory Haynes
cd1965f7b7 Allow disto-specific mirror settings
Most of our distros support DIB_DISTRIBUTION_MIRROR for setting the
mirror to be used when building. This has a major downside where if
someone would like to create a static configuration for building various
distro images it cannot set DIB_DISTRIBUTION_MIRROR (think of the case
where we want to run our tests).

By adding support for DIB_$DISTRO_DISTRIBUTION_MIRROR this problem is
resolved.

Change-Id: I92964b17ec3e47cf97e3a3091f054b2a205ac768
2016-12-08 16:02:15 -08:00
Aparna
cb3e1076e3 Update hpssacli to ssacli in proliant-tools element
This commit updates the CLI utility name hpssacli to
ssacli  as part of company name change from HP to HPE.
ssacli will work seamlessly for all the cases where
hpssacli is used.

It also updates the custom URL location for ssacli
utility with new location.

Change-Id: I2de4d3430e7308d25e36dc6960783699aa17dfae
2016-12-07 15:56:41 +00:00
Michael Johnson
2e82d7f214 Update sysctl-write-value to do conflict checking
Adds conflict checking to the sysctl-write-value script
to detect settings from multiple elements conflicting.

Change-Id: If312d199388036d6f4103e94dca99249cb3bcbaf
2016-12-06 22:58:20 +00:00
Jenkins
b25044fe11 Merge "elements: Drop executable bits from environment files" 2016-12-06 03:18:33 +00:00
Jenkins
e19a3d823a Merge "add option to configure cloud-init to allow password authentication" 2016-12-05 06:54:50 +00:00
Jenkins
753ac91ec8 Merge "Replace six.iteritems() with .items()" 2016-12-02 16:38:22 +00:00
Markos Chandras
e22faa0f77 elements: Drop executable bits from environment files
Files in $element/environment.d are meant to be sourced, so drop
the executable bit. Moreover, drop the executable bit from a couple
of other scripts that are either meant to be sourced or simply because
they are configuration files.

Change-Id: I7f724dd9d409f4a835a136f12f48a84aa9acc41e
2016-12-01 23:06:56 +00:00
Jenkins
ef1effd202 Merge "elements: Drop unneeded DIB_INIT_SYSTEM usage" 2016-12-01 19:44:47 +00:00
Jenkins
f7e095e880 Merge "elements: pip-and-virtualenv: Add python-xml dependency" 2016-12-01 07:34:55 +00:00
Jenkins
ff688c67c5 Merge "elements: zypper-minimal: Add ca-certificates-mozilla package" 2016-12-01 05:28:32 +00:00
Jenkins
c867c8cdb5 Merge "Trace package install in package-installs-v2" 2016-12-01 01:52:03 +00:00
Anshul Jain
34bdc7df90 DIB element to support cinder local attach/detach functionality
This element adds python-brick-cinderclient-ext to the make customized image
to support cinder local attach/detach functionality. Currently it has the
dependency on known bug<https://launchpad.net/bugs/1623549>, which would be
resolved with next release of python-brick-cinderclient-ext.

Change-Id: Idfe83bafa2843c781c18b83f1a3aece3ae852f78
2016-11-30 08:46:13 +00:00
Jenkins
fffe15e763 Merge "Perform package install outside of debootstrap" 2016-11-30 05:32:09 +00:00
Jenkins
3e68c5aaa5 Merge "Fix a typo" 2016-11-30 04:20:56 +00:00
Gregory Haynes
45df304d48 Perform package install outside of debootstrap
Debootstrap only supports one apt repository to install packages from.
As a result, we do not consider the updates repo during debootstrap
causing us install a second kernel when we do an apt-get dist-upgrade
during build.

Lets use debootstrap to get us a minimal chroot, then add our repos and
install the correct packages from the start.

We also have to reorder the dpkg root.d scripts which configure apt so
they run before we perform our package installs.

Change-Id: I6a592db6f0a01d3b19d8e0786e63f1315a1ef647
Closes-Bug: #1637516
2016-11-30 15:16:46 +11:00
Markos Chandras
35e878b6d9 elements: zypper-minimal: Add ca-certificates-mozilla package
It's important to have the CA certificates on the target for ssl
crypto apps to work. Plus it's also important during bootstrapping
with diskimage-builder as tools like 'pip' etc need the certificates
in place in order to work properly. This fixes opensuse-minimal
image generation with the 'simple-init' element which was causing the
following error:

Download error on https://pypi.python.org/simple/: [SSL:
CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)

Change-Id: Ie94cd3556f8ae523f60ce0155ba18ed752e6fbb6
2016-11-29 16:14:53 +00:00
Jenkins
7644681815 Merge "elements: Add new openssh-server element" 2016-11-28 17:09:15 +00:00
Jenkins
0a453febc6 Merge "Fedora AArch64 (64-bit ARM) support in diskimage-builder" 2016-11-25 12:09:21 +00:00
Jenkins
73a9591632 Merge "yum-minimal: add systemd to initial install" 2016-11-25 11:19:09 +00:00
Ian Wienand
08d6a9f93d yum-minimal: add systemd to initial install
It seems in the grub cleanup in
Iafe3611f4eec3c6357587a6cae6a30a261686ead I managed to unintentionally
drop systemd from the yum-minimal builds.  By not pre-installing grub
we dropped some dependencies; the path is tortured ... grub2 ->
os-prober -> udev -> systemd-udev -> systemd (we don't even want
os-prober!  So this whole thing was working by accident).

This manifests in *very* confusing ways.

Currently centos-minimal builds are failing late in the build with
services unable to enabled.  dib-init-system was actually trying to
tell us that it didn't know what init was installed (because systemd
wasn't actually installed), but unfortunately it was not really
failing.  This meant the service files were not copied correctly from
other elements, and thus fail to be enabled.  I have corrected this
with I076c08190d40c315ad6a6d96a3823e9fc52630be which would at least
alert us earlier.

For Fedora 24, due to a bug in dracut dependencies [1], missing the
systemd-udev package fails the build of the initrd during the kernel
install.  This then results in an initrd-less, unbootable system (see
also Ibaaa81124098f3c6febe48e455d3e1cd0a5f1761).

Add these dependencies explicitly.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1398505

Change-Id: I24ce648485c3d6f3c27ab8f87a638516b3727017
2016-11-25 21:09:11 +11:00
Ian Wienand
ff79dc9793 Catch errors in DIB_INIT_SYSTEM export
When you source a file that just does

 export FOO=$(bar)

you miss any invalid return codes from "bar" (even under -e) because
bash returns the value of the "export", which is 0

On centos-minimal, we stopped bringing in systemd early and this was
causing dib-init-system to not know what init was available.  Since it
did not fail correctly, it lead to confusing errors much later in the
build when service files were not copied correctly.  See also
I24ce648485c3d6f3c27ab8f87a638516b3727017

A dib-lint check is added.  One minor fixup is in 00-set-apt-sources
(this one is less likely to cause problems).  I have run dib-lint over
project-config elements and none use this pattern.

Change-Id: I076c08190d40c315ad6a6d96a3823e9fc52630be
2016-11-23 23:03:50 +00:00
Luong Anh Tuan
29d50bc69b Replace six.iteritems() with .items()
We should avoid using six.iteritems/keys achieve iterators. We can use
dict.items/keys instead, as it will return iterators in PY3 as well.
And dict.items/keys will more readable.

In py2, the performance about list should be negligible, see
https://wiki.openstack.org/wiki/Python3 and
http://lists.openstack.org/pipermail/openstack-dev/2015-June/066391.html

Change-Id: I6353f0a1b423c6dbd0c71343f7919fd0de440e34
2016-11-23 12:03:05 +00:00
Ian Wienand
bc6be85424 Trace package install in package-installs-v2
When running the package install, trace the output so we can see what
packages were installed.

Change-Id: I5442f544ff0ef3ddffdbe6b898d178548d699a41
2016-11-23 19:58:45 +11:00
Ian Wienand
f15550f9fe Special case dib-python in dib-lint
It seems that on Xenial, it does not take much to confuse "file" and
it's mime guessing such that it thinks some files are not python.

"package-installs-v2" is a good example, since it has an interpreter
"dib-python" that "file" doesn't know about, and no extension.  While
looking at this, I've added emacs vars here so it opens in python
mode.

Change-Id: I01994b08c5ad8987925f1eec4062f5b6ee72eb8f
2016-11-23 19:58:43 +11:00
Markos Chandras
e4868d9ad6 elements: Drop unneeded DIB_INIT_SYSTEM usage
DIB_INIT_SYSTEM is exported by the dib-init-system element and contains
the output of the dib-init-system script so there is no need to
re-initialize it during various phases.

Change-Id: I09d6d10742689efe3d8eb9d64b539d6599b46227
2016-11-22 10:47:43 +00:00
Markos Chandras
bbcc22751f elements: Add new openssh-server element
Add new 'openssh-server' element to ensure that openssh server
is installed and enabled during boot. This is mostly useful for
*-minimal images which do not come with openssh installed and/or
enabled in order to keep a small dependency footprint.

Change-Id: Ide15ee04f5de123dbc8ce4bb56d638d8a167c341
2016-11-22 10:07:14 +00:00
Noam Angel
55b6101e1b add option to configure cloud-init to allow password authentication
This patch will configure cloud-init to allow password authentication.
This is usefull in case you use "devuser" element and want to ssh guest
image.

Change-Id: I00e38aa2753f26b4cdd34d0fd85fc8e0de78171f
2016-11-22 09:33:58 +00:00
Markos Chandras
d84df60345 elements: pip-and-virtualenv: Add python-xml dependency
SUSE packages the 'xml' python module as a separate package so make
sure it's pulled in before we attempt to install the pip module
since the latter depends on it. Fixes the following problem when
building with the opensuse-minimal and pip-and-virtualenv elements:

Traceback (most recent call last):
  File "/tmp/get-pip.py", line 19177, in <module>
    main()
  File "/tmp/get-pip.py", line 194, in main
    bootstrap(tmpdir=tmpdir)
  File "/tmp/get-pip.py", line 82, in bootstrap
    import pip
  File "/tmp/tmpOiESjX/pip.zip/pip/__init__.py", line 16, in <module>
  File "/tmp/tmpOiESjX/pip.zip/pip/vcs/subversion.py", line 9, in <module>
  File "/tmp/tmpOiESjX/pip.zip/pip/index.py", line 32, in <module>
  File "/tmp/tmpOiESjX/pip.zip/pip/_vendor/html5lib/__init__.py", line 16, in <module>
  File "/tmp/tmpOiESjX/pip.zip/pip/_vendor/html5lib/html5parser.py", line 6, in <module>
  File "/tmp/tmpOiESjX/pip.zip/pip/_vendor/html5lib/inputstream.py", line 10, in <module>
  File "/tmp/tmpOiESjX/pip.zip/pip/_vendor/html5lib/utils.py", line 10, in <module>
ImportError: No module named xml.etree.ElementTree

Change-Id: I1bec12dfcde05fb07f41bcec994148c3eacbb287
2016-11-21 15:54:18 +00:00
Clark Boylan
c5ec1348c3 Fix runtime ssh host keys script
The script is set -e and set -o pipefail, unfortauntely this intersects
with `yes n`'s non zero exit code behavior when it receives an interrupt
like sigpipe. As a result stop setting pipefail so that we treat those
errors as "normal" and only fail if ssh-keygen fails.

Change-Id: I5447df97c9888cae3007e235e2fea44df61af28e
2016-11-19 18:02:33 -05:00
gecong1973
f9244a8fce Fix a typo
TrivialFix

Change-Id: Iebe82e616eed2d9b9a99a9714230d480adbd055b
2016-11-18 10:48:05 +08:00
Jenkins
eeb9e0e4a6 Merge "Disable all repos in os-refresh-config too" 2016-11-18 00:23:04 +00:00
Jenkins
0fbf131550 Merge "Cleanup yumdownloader repos" 2016-11-18 00:23:00 +00:00
Jenkins
27bd4741bf Merge "debian: install dialog package" 2016-11-18 00:21:23 +00:00
Jenkins
f5d23c975a Merge "Change path for dnf arch override so basearch is not overwritten." 2016-11-18 00:20:50 +00:00
Jenkins
0ea4ea5bed Merge "Don't use ssh-keygen -A for init scripts" 2016-11-17 19:54:25 +00:00
d.marlin
8d7362aa9b Change path for dnf arch override so basearch is not overwritten.
After writing the basearch value to /etc/dnf/vars/basearch the
arch value was overwriting the same file.  This appears to be
incorrect, so changing it to write /etc/dnf/vars/arch, which
matches the subsequent 'yum' code paths.

Change-Id: I5da54f03224c11f9e286f16b68533936c4174c2a
2016-11-17 03:36:54 -05:00
d.marlin
9b4d2a22e4 Fedora AArch64 (64-bit ARM) support in diskimage-builder
Add some checks for AArch64 to avoid the "Unknown architecture" or
"architecture not supported" messages, and allow builds to complete.

Change-Id: I89ba609abaeeb7019eb317cf13473929b2065230
2016-11-16 21:47:26 -05:00
Ben Nemec
fb8cf95b6f Disable all repos in os-refresh-config too
This change was made for pre-install so it applies during the
image build, but wasn't applied to the os-refresh-config script
that would run after deployment.  The same problems apply there,
so we should do the same thing.

Change-Id: I4b8534cc9586eeb588b5c358550e76e27d40556a
Closes-Bug: 1629922
2016-11-16 16:27:24 -06:00
Jenkins
be1e563524 Merge "simplify ARCH param for rhel/centos param can be x86_64 and amd64" 2016-11-15 00:44:00 +00:00
Noam Angel
e88d6b37df add support for SUSE in dhcp-all-interfaces
This patch will add support for SUSE network scripts,
network script in SUSE saved under "/etc/sysconfig/network/ifcfg-*"
see: https://www.suse.com/documentation/sled11/book_sle_admin/data/sec_basicnet_manconf.html

Change-Id: I87ac2e327cee4945c15da9f2e4adc0a8b7650712
2016-11-15 10:20:09 +11:00
Noam Angel
d07d7ed15d simplify ARCH param for rhel/centos param can be x86_64 and amd64
for fedora/rhel/centos the main supported ARCH is x86_64. This patch allow
to call diskimage-builder with the above distro's with param ARCH=x86_64,
And also retain same behaiver when call with ARCH=amd64 as it translate
anyway to x86_64. Doing so wil simplify user expirience.

Change-Id: I229e0912434109b1b48a030bd35ad8dc1096a629
2016-11-15 10:18:14 +11:00
Jenkins
80976d9d44 Merge "Add element for setting sysctl values" 2016-11-14 17:19:04 +00:00
Saverio Proto
5d9d3d5cf0 debian: install dialog package
Without the dialog package is not possible
to properly use an interactive frontend.

debconf will print the following errors:

debconf: unable to initialize frontend: Dialog
debconf: (No usable dialog-like program is installed,
so the dialog based frontend cannot be used. at
/usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 76, <> line 1.)

Change-Id: I0c7142f717cacf7437dbac1e1696f39b00cb4c49
2016-11-14 15:37:39 +01:00
Jenkins
77e14e9313 Merge "elements: runtime-ssh-host-keys: Add openssh-client mapping for SUSE" 2016-11-14 10:01:11 +00:00
Jenkins
a0a19cbbfa Merge "elements: opensuse-minimal: Add support for building Tumbleweed images" 2016-11-14 10:01:05 +00:00
Jenkins
6224824bde Merge "Fix ironic-python-agent image not loading vfat mod" 2016-11-14 09:41:35 +00:00
Jenkins
cc8ab5f4b6 Merge "Avoid disabling rhel-7-server-rh-common-rpms" 2016-11-14 09:32:28 +00:00
Jenkins
5c546ed565 Merge "Install lsb package by map name instead of package name" 2016-11-14 07:13:52 +00:00
Jenkins
3bfff8f905 Merge "elements: zypper-minimal: Mount common pseudo filesystems" 2016-11-14 07:08:23 +00:00
Jenkins
aec2128b38 Merge "elements: zypper-minimal: Refresh repositories after adding the cache" 2016-11-14 07:01:13 +00:00
Ben Nemec
79c27199f2 Install lsb package by map name instead of package name
We have a pkg-map entry for lsb_release, but in package-installs.yaml
we refer to the actual package name instead.  This will happen to
work on Red Hat platforms, but it's actually wrong.

Change-Id: Idb248f96e75fa1090422fa08e5fbb2385cc1f517
2016-11-11 14:30:21 -06:00
Ian Wienand
7e60540c0d Cleanup yumdownloader repos
yumdownloader has to have all the repo XML files, etc, which adds up
to a not totally insignificant 150MiB or so.  Currently we're leaking
this directory for every build, which adds up on regualar builders
like nodepool.

Isolate the call with a separate TMPDIR so we can clean it up after
the initial download.

Change-Id: Ic65e8ca837cc76b7a1bb9f83027b4a5bdd270f75
2016-11-10 17:02:53 +11:00
Jenkins
587d14feed Merge "dhcp-all-interfaces: support Centos/RHEL 6" 2016-11-08 11:21:51 +00:00