Commit Graph

3036 Commits

Author SHA1 Message Date
Dirk Mueller
bfeb9d9e99 zypper: Clean caches and don't cache packages locally
For builds inside the infra, we don't want to pack the cache
inside the image (as it might be different at the time the image
runs). In an opensuse-minimal image this saves about 10MB of image
size.

Change-Id: I5ecabd46f0a662798bda3e4468395ad8308d0055
2017-07-23 17:24:24 +02:00
Jenkins
55971717b6 Merge "elements: openstack-ci-mirrors: Use openSUSE mirrors for gating jobs" 2017-07-22 05:22:34 +00:00
Jenkins
e029af993b Merge "Remove DIB_[DISTRO]_DISTRIBUTION_MIRROR" 2017-07-22 05:22:04 +00:00
Jenkins
f9700225b9 Merge "doc: supported_distros: Add openSUSE Leap 42.2/3 and Tumbleweed" 2017-07-20 11:13:32 +00:00
Jenkins
7a70299668 Merge "Enable console during kernel boot on Power" 2017-07-20 03:55:19 +00:00
Jenkins
d66dbc679c Merge "The correct option for label name in fat and vfat is '-n'" 2017-07-20 03:54:23 +00:00
Markos Chandras
710c20196b doc: supported_distros: Add openSUSE Leap 42.2/3 and Tumbleweed
Depends-On: I08a663fd03c8545de09d650001ab250eaf40e427

Change-Id: I924987457e047beaf13de6ca47d8d4a5fedff513
2017-07-19 14:59:29 +02:00
Ian Wienand
7ffe6856d6
Add -m flag to setfiles for Fedora 26
As described in the comment and associated bugzilla, the behaviour of
setfiles has changed in Fedora 26 to require "-m" situations where
labeled file-systems are mounted below non-labeled file-systems.  Our
loopback/chroot system appears to trigger this nicely, leading to a
setfiles call that does nothing without this.

Change-Id: I276c6f6a4fb44f4bea5004f6b4214f94757728ae
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-07-19 08:01:19 -04:00
Ian Wienand
6ffde2e596 yum-minimal: pre-install coreutils
As described in the referenced bug, the dependency solver in yum
doesn't handle weak dependencies well and in some cases, such as
Fedora 26, can end up choosing coreutils-single (the busybox-esque
single binary) instead of actual coreutils, which then causes problems
with conflicting packages later.

Change-Id: I2907bf3b74c146986b483d52cc6ac437036330b4
2017-07-18 14:51:18 +10:00
Ian Wienand
b8ad9c2e37 Force install during pip-and-virtualenv
On a system where the packaged pip/virtualenv is up-to-date with
upstream (such as Fedora 26 ... for now), we don't reinstall, which
then violates a bunch of assumptions later on.  Force install.

Change-Id: I6ebcda0351997fa7e32f0e6e77a98b2c33764e3f
2017-07-18 14:50:09 +10:00
Ian Wienand
da90ef4743 Fix latest-limit command line
It turns out dnf argparse can't handle negative numbers without "=".
It's actually documented in the man page

    --latest-limit <number> ...  If <number> is negative skip <number>
      of latest packages. If a negative number is used use syntax
      --latest-limit=<number>

But who reads that :)  This started failing with Fedora 26

Change-Id: I884af94c07fa11b010f69863047a04711b14f21e
2017-07-18 13:17:10 +10:00
Jenkins
016606c81d Merge "opensuse-minimal: install glibc-locale" 2017-07-18 00:40:48 +00:00
OpenStack Proposal Bot
23fb6fe142 Updated from global requirements
Change-Id: If1b543d86c1d5a19df3dc0eb5c3018e3968f8c04
2017-07-17 22:00:33 +00:00
Dirk Mueller
59721d3c74 opensuse-minimal: install glibc-locale
We expect LC_ALL for non-C locales to be working inside
images, so always install glibc-locale for openSUSE.

Change-Id: I8fe92773e377539070d9d9fe2960a6202bb80a18
2017-07-17 22:50:25 +02:00
Markos Chandras
6be09152c2 elements: openstack-ci-mirrors: Use openSUSE mirrors for gating jobs
In preparation for promoting the openSUSE jobs to voting ones we should
use the OpenStack mirrors. As such, the opensuse elements are modified
to make use of the DIB_DISTRIBUTION_MIRROR variable which is normally
exported by the openstack-ci-mirrors element.

Change-Id: Ie588c1c1eec13190cfb2ec718ba51f8c9878283f
2017-07-17 10:54:03 +01:00
Jenkins
b736264941 Merge "Update the documentation link for doc migration" 2017-07-17 07:42:04 +00:00
chenxing
f3158553d7 Update the documentation link for doc migration
The documentation has moved due to the docs manuals migration
effort.  See [1]

[1] https://specs.openstack.org/openstack/docs-specs/specs/pike/os-manuals-migration.html

Change-Id: I16fac4a1caea5e4b8c7e2d5adcb93d8a6535e65d
2017-07-17 16:12:12 +10:00
Jenkins
c18a3ff029 Merge "Replace architecture-emulation-binaries with qemu-debootstrap" 2017-07-17 05:36:09 +00:00
Ian Wienand
3457d2f8e8 Remove DIB_[DISTRO]_DISTRIBUTION_MIRROR
We added the DIB_distro_DISTRIBUTION_MIRROR arguments with
I92964b17ec3e47cf97e3a3091f054b2a205ac768 as a way that we could
source a list of mirrors and then have the distro elements choose
which one applied to them.

However, this hasn't worked out to be so useful.  The
openstack-ci-mirrors element is working as a mirror setup script -- it
translates the openstack CI mirror list variables into the generic
"DIB_DISTRIBUTION_MIRROR" as appropriate for each distro's build.
Also, it turns out there's other things that need to be done, such as
turning off gpg checking, which mean the idea of "just export
variables" hasn't turned out as valid ... you need actual code
involved to get it right.

AFAICT we never actually documented these, and they do not seem to be
in use.  They have caused considerable confusion when dealing with new
platforms as we try to keep consistency.  Remove them.

[1] http://codesearch.openstack.org/?q=DIB_.*_DISTRIBUTION_MIRROR&i=nope&files=&repos=

Change-Id: Ifc4ab700631ffdfbe790068558f670f9a11dde5e
2017-07-17 14:47:31 +10:00
Jenkins
5d919bdc8e Merge "doc: Switch from oslosphinx to openstackdocstheme" 2017-07-17 01:03:15 +00:00
Jenkins
787e76b916 Merge "Remove additional Bumblebee repository for opensuse element" 2017-07-17 00:50:46 +00:00
Jenkins
64a8c6e1dc Merge "zypper-minimal: No point in preserving the environment here" 2017-07-17 00:26:11 +00:00
chenxing
644d642e02 doc: Switch from oslosphinx to openstackdocstheme
Per the manuals migration effort, projects should use the
openstackdocstheme [1]

[1] https://specs.openstack.org/openstack/docs-specs/specs/pike/os-manuals-migration.html

Change-Id: I3dbba86369326e831ec5c47e368d04e28bcb93a1
2017-07-16 23:54:49 +00:00
Amrith Kumar
59f416ae20 The correct option for label name in fat and vfat is '-n'
The code in mkfs correctly extends the command line with a '-n' for
vfat but does not currently do it for fat. This means that mkfs for
fat ends up with a '-L' which is what you'd do for everything like
ext[234].

The change just treats fat like vfat in the one place where this check
is required.

Change-Id: If65dfd949acdadff33a564640fb42ea73026a786
Closes-Bug: #1703063
2017-07-15 22:48:52 -04:00
Dirk Mueller
02d33f2ca7 zypper-minimal: No point in preserving the environment here
Change-Id: I46442e841d1f718b683bca4d2a348f0013306907
2017-07-13 22:50:47 +02:00
Dirk Mueller
05ba445ade Remove additional Bumblebee repository for opensuse element
The purpose of the openSUSE element is to build openSUSE distribution
based images, so an additional community repo shouldn't be pulled into
the image. In addition the dkms dependency is blacklisted for SUSE
in the dkms element anyway, so this should be a noop.

Change-Id: I0aa06d9f4f110546032f910e3361840693d02de7
2017-07-11 23:24:05 +02:00
Jenkins
0327d775f1 Merge "pip-and-virtualenv: Install python3 on openSUSE" 2017-07-11 08:11:16 +00:00
Jenkins
997a6ea6cb Merge "Add symlink test for resolv.conf restore" 2017-07-11 08:07:23 +00:00
Rafael Folco
bfdf7dc0f6 Enable console during kernel boot on Power
On Power systems console should be added the kernel command line
in the following order: 'console=tty0 console=hvc0'.
The first one is the graphical console. The last one is the serial
console. The kernel enables all the consoles pointed through the
kernel command line. However, only the last one will receive
input/output during kernel boot. All the other consoles will be
enabled after the boot.

Change-Id: I0069f608e0ab104d3778954e033fb82ed5ea7693
2017-07-07 17:55:56 +00:00
Amrith Kumar
43e32116bd fix readme.rst to reflect correct environment variable
The readme.rst incorrectly refers to the environment variable
DIB_APT_KEYS which should be DIB_ADD_APT_KEYS. See [1] for usage in
code.

This is a minor correction to the readme only, no runnable code is
modified.

[1] http://git.openstack.org/cgit/openstack/diskimage-builder/tree/diskimage_builder/elements/dpkg/extra-data.d/01-copy-apt-keys#n23

Change-Id: I04129cef9f40ec75a206c126bfd40ee61e4e6a2b
2017-07-06 22:54:08 -04:00
Ian Wienand
5fa6e3e13c Add symlink test for resolv.conf restore
We replace the base resolv.conf with an "outside" copy so that
resolving works when we're in the chroot.

Installing resolvconf package modifies the in-chroot resolv.conf to a
symlink (to /var/run) which it wants maintained in the final image.
We have the existing "immutable" check for a created resolv.conf file,
but no eqivalent for a symlink.

This adds a check to see if the resolv.conf is a symlink and leave it
alone if it is, assuming it has been re-created in the chroot.

I have tested this with ubuntu-minimal+resolvconf with
dhcp-all-interfaces and the system seems to work with resolvconf
working correctly.

Change-Id: Idd5a26e9d55979bd951577d5b098ed4bfba91ad3
2017-07-06 13:48:27 +10:00
Jenkins
e8ad2a3799 Merge "elements: pip-and-virtualenv: Use common packages for openSUSE" 2017-07-04 11:20:35 +00:00
Markos Chandras
5fe35b0d7a pip-and-virtualenv: Install python3 on openSUSE
The python3 package actually contains some core modules (like the xml
one) which are not present in the python3-base on which is pulled by
the python3-devel package. As such, it's best to have it installed
similar to python-xml for python2.

Change-Id: I5cd5d1127ae62d6753c2ace44965179c5400bb9a
2017-07-04 08:40:34 +01:00
Jenkins
fad72745d2 Merge "Support for Cloud Images on ppc64le for rhel7 and centos7" 2017-07-04 01:13:24 +00:00
Jenkins
faf83dac3c Merge "bindep.txt: Exclude gnupg2 package on openSUSE" 2017-07-03 23:34:11 +00:00
Jenkins
2ed643a734 Merge "Use the dib python to do cleanup" 2017-06-29 21:22:36 +00:00
Jenkins
6b45497ff6 Merge "Remove centos and rhel elements" 2017-06-29 21:16:57 +00:00
Jenkins
f0fb835db9 Merge "Avoid hanging endlessly on unreachable cache urls" 2017-06-29 08:03:25 +00:00
Chhavi Agarwal
6d69d7909d Support for Cloud Images on ppc64le for rhel7 and centos7
In order to support {CentOS,RHEL}7 for building cloud images we need to
handle the differences in grub packaging from Ubuntu.  We also need to
populate the defualt location for cloud images for CentOS builds.

Change-Id: Ie0d82ff21a42b08c4cb94b7a5635f80bfabf684e
2017-06-29 15:44:26 +10:00
Markos Chandras
49bed39aa7 bindep.txt: Exclude gnupg2 package on openSUSE
gnupg2 does not exist on openSUSE and there is no need to explicitly
install it. Fixes the following problem in the OpenStack CI

2017-06-28 09:28:29.071275 | + sudo PATH=/usr/sbin:/sbin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/bin:/usr/games zypper --non-interactive install gnupg2 squashfs
2017-06-28 09:28:29.124994 | Loading repository data...
2017-06-28 09:28:29.287514 | Reading installed packages...
2017-06-28 09:28:29.713161 | 'gnupg2' not found in package names. Trying capabilities.
2017-06-28 09:28:29.713234 | No provider of 'gnupg2' found.

Change-Id: Ie90c3cf6d478ae4e178df0ce61072e9ee15b2259
2017-06-28 21:27:20 +01:00
Dirk Mueller
959226c55e Avoid hanging endlessly on unreachable cache urls
When a download redirector redirects to a broken mirror, timeout
quickly rather than waiting until the overall job is being timed out.

Change-Id: If7eb63d406aaf61f71aa9203cf708c474aa63fd0
2017-06-28 22:14:55 +02:00
Markos Chandras
c46b6da65f elements: pip-and-virtualenv: Use common packages for openSUSE
The 'packages' variable already contains the packages we need so
use it instead of duplicating the packages.

Change-Id: Id22e1862f9654e66252d03a0fed9839cf004d750
2017-06-28 17:59:25 +01:00
Ian Wienand
859e737ada Remove mirror create
This was just to avoid our initial gate crisis, and has been put into
project-config with I45b4b181369032155f8908ee11641d2327586e6f

Change-Id: I3ab57b4455b39ccc3fa94ef1be2193fa7f082fb6
2017-06-28 18:59:51 +10:00
Ian Wienand
a00d02f6a1 Remove centos and rhel elements
Several people have popped up in IRC recently with failures in these
elements.  Without Python 2.7 available in the image they are
unsupported (OpenStack hasn't supported it for a long time).  Remove
these to avoid further confusion.

The centos/centos7 DISTRO split that has happened with centos-minimal
is unfortunate but I don't think it helps to rename centos7/rhel7 ATM.
To summarise; DISTRO=centos7 means image based build,
DISTRO=centos && DIB_RELEASE=7 means the minimal build.

In the future, I think it is important that the minimal builds and
image builds set the same DISTRO.  This reflects that "upper" layers
shouldn't care about the exact building of the lower layers.  I see
CentOS 8 going one of two ways

1) the changes are so significant, we start separate centos8 /
centos8-minimal elements.  They both set DISTRO=centos8 (and
DIB_RELEASE to point-release maybe?).  This means we have to update
all "if DISTRO == centos || DISTRO == centos7" branches to also check
for "centos8".  Evenually (!)  "centos" goes away for versioned DISTRO
only

2) we restore centos element with DISTRO=centos and DIB_RELEASE=8, and
centos-minimal remains the same.  This means we have to audit all "if
DISTRO == centos" calls to make sure they're appropriate for version 8
(stick a "&& DIB_RELEASE=7" on them all basically).

I'm not sure we can fully decide until we start to see excatly how the
distro switching/matching bits look, but (2) is consistent with Ubuntu
and probably the preferred solution.

Some "rhel" parts have been cleaned up.  More could be done in
rhel-common, but given our lack of coverage of that I'd prefer to
leave it for now.

Change-Id: I6ea784116ef59ca22878c8512c963f29c815a00a
2017-06-28 12:26:24 +10:00
OpenStack Proposal Bot
f8106b2ec0 Updated from global requirements
Change-Id: I1fd3f69273ddc3cc8447622dd48e007c1994aa1c
2017-06-27 12:09:11 +00:00
Jenkins
cd5d08be89 Merge "dib-lint: Ignore editor temp files for linting run" 2017-06-27 03:57:37 +00:00
Tony Breeds
f12b1310ef dib-lint: Ignore editor temp files for linting run
When running dib-lint vi temp files .*.sw? files are included which leads to
false positives.  ALso all editor files are checked when looking at indents,
again this results in false positives.

Exclude those files by checking if they're in the user/project's
.gitignore setup.

Change-Id: I0a48174f22a8dad9e8f15bf3f70835d021a2d46f
2017-06-27 10:46:00 +10:00
Tony Breeds
c383428727 As far as block-device layout is concerned ppc64le == ppc64el
Change-Id: I06a89f256f66eba2f73dd110f5d8a61e0f0a924c
2017-06-27 00:34:33 +00:00
Tony Breeds
7aa6a69983 [doc] Add some notes about PowerPC Architecture names
Change-Id: I58eb3e442344bbbcf7c5d71c13688baaacd70105
2017-06-27 10:27:10 +10:00
Jenkins
c1facd7980 Merge "Fix mkfs use wrong label option for vfat" 2017-06-26 02:41:30 +00:00