Commit Graph

689 Commits

Author SHA1 Message Date
Zuul
709c9e70c9 Merge "Set router solicitation delay with using NM" 2019-07-10 19:14:00 +00:00
Clark Boylan
5b5b78bf59 Set router solicitation delay with using NM
The linux kernel and NetworkManager fight each other over control for
interface management when router advertisements are in use. Long story
short if the linux kernel configures a network interface for ipv6
before NetworkManager attempts to manage that interface then NM will
ignore the interface and not configure ipv4 on it.

This can happen because the kernel is configured to send router
advertisements solicitations which result in router advertisements which
the kernel uses to configure the interface(s). There is a default of a 1
second delay before sending the solicitation which in many cases is long
enough that NM has started before then. However, in slower environments
like those used for testing with qemu this isn't long enough.

Some testing by hand indicates that 15 seconds is about right so
increase the delay to 15 seconds via sysctl.conf.

Note this may increase boot times in ipv6 only environments (though it
is hard to be sure due to how systemd starts everything at once and does
socket activation and the like).

Change-Id: I475a253091cbaf63687b91c748c31a6753bb0f57
2019-07-10 08:33:17 -07:00
Chandan Kumar (raukadah)
08caa8034d [RHEL-8] Set _clear_old_files=0 in install-pip element
When virtualenv and setuptools gots installed from source and rpm
then their installation path lives at different places but when
the python script got called then that time it choses either of
rpm or source based path on system wide installation and leads to
different failure as their methods are not implemeted.

So by setting _clear_old_files to 0 will install
python3-virtualenv python3-pip python3-setuptools from rpms only
and avoid these failures.

Change-Id: I0c162f1fe8168513e352546ab8dd2b68fa65b88c
Signed-off-by: Chandan Kumar (raukadah) <chkumar@redhat.com>
2019-07-08 19:50:25 +05:30
Matthew Thode
de94e07a86
disable autounmask for emerge
autounmask=y (default) changes portage depsolving, causing errors
(mostly often seen in perl and binpkg related issues).

Disabling this functionality for DIB builds is OK as the enviroment is
not passed on post build and the build process is not interactive
anyway.

Change-Id: Ife9ace246bec16864ee4982bc456763af5dff2e8
Signed-off-by: Matthew Thode <mthode@mthode.org>
2019-07-05 12:41:51 -05:00
Lon Hohberger
92f87d0d0a ironic-agent: Use targetcli & python3-devel on rhel8
Change-Id: I2e596f5c3dc23b21441a4eb61c773725ebd25d34
Signed-off-by: Lon Hohberger <lhh@redhat.com>
2019-07-02 21:39:13 -04:00
Matthew Thode
d8f796e153
install gnupg2 by default in debian-minimal
debian-minimal depends on debootstrap which depends on dpkg

This needs to be installed early as dpkg installs the apt keys early via
02-add-apt-keys in pre-install.d

Change-Id: I8580849ceaa7a5152c94f29afa890ac6d6983fb1
2019-07-01 14:28:01 -05:00
Matthew Thode
03933b3ab7
set default sources conf for buster as it now has a release
Change-Id: Ica014a1a267b55e2f29a0c333213e0a4c897108b
2019-07-01 13:59:14 -05:00
Maksim Malchuk
32dd3305d2 Cleanup: remove useless statement
This change removes useless statement accidentally added in the
06576a02f0

Change-Id: I7ea4a24d8c72c9e72f5f87247403af0f9bf69b40
Signed-off-by: Maksim Malchuk <maksim.malchuk@gmail.com>
2019-07-01 10:28:12 +03:00
Zuul
091a4e2c6e Merge "debootstrap: make default network interface names configurable" 2019-06-28 06:30:46 +00:00
Zuul
2c5fc5f8f9 Merge "Enable nodepool testing for opensuse 15.1" 2019-06-28 06:26:58 +00:00
Dirk Mueller
97444ad92e Enable nodepool testing for opensuse 15.1
There are several jobs depending on working opensuse 15.1
images in nodepool, so it makes sense to ensure its working.
Also upgrade the previously marked experimental opensuse-15.0
job as it tests the xenial->opensuse combination, which is
particularly difficult to keep working and we'll need it in
the CI.

Change-Id: Icb6d998756ce5221e017959dcb59b21f0f023454
2019-06-27 19:59:45 +02:00
Zuul
f27a0dfc26 Merge "Add DIB_UBUNTU_KERNEL to ubuntu-minimal" 2019-06-25 22:48:50 +00:00
Michael Johnson
e433aebf7d Add DIB_UBUNTU_KERNEL to ubuntu-minimal
This patch adds a new environment variable to the ubuntu-minimal
element called DIB_UBUNTU_KERNEL that allows you to specify the kernel
meta package that will be using to install the kernel inside the image.
It supports "linux-image-generic" (The default), "linux-image-kvm", and
"linux-image-virtual".
This allows building images that are smaller in size (~200MB smaller
qcow2) that have only the kernel modules necessary for virtual
machines.

Change-Id: I8ce65e1d357d15e8ed8995ad1dcaea02bbd1986f
2019-06-20 10:18:23 -07:00
jacky06
8dd7ca720f Sync Sphinx requirement
1. Sync sphinx dependency with global requirements. It caps python 2 since
sphinx 2.0 no longer supports Python 2.7.
2. Update some URLs to latest
3. Remove the unnecessary space

Change-Id: I5464be9e055feecd80918f691448acf5f100e701
2019-06-18 23:29:52 +08:00
Zuul
4afcba1fea Merge "Move pypi to dib-python" 2019-06-18 06:49:37 +00:00
Zuul
e281c4dae9 Merge "Use architecture-specific grub2 RPMs on RHEL8" 2019-06-18 06:49:35 +00:00
Zuul
c46f8714e3 Merge "Remove the rhel 8 check for xfs" 2019-06-18 06:49:34 +00:00
Zuul
fe618002ff Merge "Update test coverage for openSUSE/-minimal to 15.1" 2019-06-18 06:21:21 +00:00
Dirk Mueller
a81cf9e231 Update test coverage for openSUSE/-minimal to 15.1
Use openSUSE 15.1 as default, which is the latest released stable
openSUSE release.

Remove leftovers for unmaintained openSUSE 42.2 images.

Depends-On: https://review.opendev.org/#/c/660126/
Change-Id: I0b204b7b3d7ae74b6749320b3bfe1ca89d154ebb
2019-06-13 09:20:40 +02:00
Michael Johnson
5f4b764a11 Remove the rhel 8 check for xfs
This patch removes the check and default for rhel 8 requiring
xfs filesystem as rhel 8 images can successfully be built with
ext4 filesystems.

Change-Id: I1a6bfa26324fd43ae0c77c2c977dda0dd56e26e5
2019-06-12 07:01:36 -07:00
Andreas Florath
05af0fc863 debootstrap: make default network interface names configurable
Nowadays, in the time of Predictable Network Interface Names, the
network interface names 'ethX' are not used that often any more.
Depending on the virtualization layer and the guest OS names like
'ens3', 'enp1s0' or 'enp0s31f6' are used.
This patch enables the user to set DIB_NETWORK_INTERFACE_NAMES to a
list of network interfaces which are brought up using DHCP during
(first) boot.

Change-Id: I04cc2ee710f0389a8085b1c91d9329784cb28048
Signed-off-by: Andreas Florath <Andreas.Florath@telekom.de>
2019-06-12 13:53:38 +00:00
Maksim Malchuk
ea9ab89829 Move pypi to dib-python
The latest Fedora/Ubuntu images don't ship python2 by default, so we
need to use our dib-python wrapper for this so we work in python3 only
environments.

This change also correctly creates the pip.conf and .pydistutils.cfg
files with trusted host extracted from the index-url.

Related-bug: 1577105

Change-Id: Ibb5348af3e3bbe46b19affe90a8930a4b4ad4cad
Signed-off-by: Maksim Malchuk <maksim.malchuk@gmail.com>
2019-06-10 20:43:09 +03:00
Dmitry Tantsur
928c6e61f0 ironic-agent: install mdadm on the ramdisk
The newly introduced software RAID support requires it.

Change-Id: Ic438865006f1472abc0c9f4d40cc40c91b4ada71
2019-06-07 14:05:41 +02:00
Lon Hohberger
0cf0942068 Use architecture-specific grub2 RPMs on RHEL8
RHEL8 ships a bunch of grub2-efi-X-modules in its main
repository, each of which provides grub2-efi-modules,
potentially causing nondeterminism when building images.

This changes the DIB elements to always use architecture-
specific RPMs when RHEL8 is selected.

Change-Id: If94f3721195d5ecd80036e4234a3ca223a19c349
Related: https://bugzilla.redhat.com/show_bug.cgi?id=1716672
2019-06-06 10:50:51 -04:00
Zuul
7f469e3e83 Merge "Increase size of EFI system partition (again)" 2019-05-31 09:05:17 +00:00
Zuul
d323928af0 Merge "Makes image caching more resilient" 2019-05-31 08:54:31 +00:00
Zuul
d6f43865ed Merge "fail early when lates build information can not be fetched" 2019-05-31 08:45:50 +00:00
Zuul
21676bd350 Merge "Add option to skip update packages" 2019-05-31 08:13:38 +00:00
Zuul
49930da885 Merge "Deprecate rhel7 in favor of rhel" 2019-05-31 08:01:13 +00:00
Zuul
4367dd2dd3 Merge "Add version-less RHEL element for RHEL7 and RHEL8" 2019-05-31 07:56:50 +00:00
Pierre Riteau
7fd52ba841 Increase size of EFI system partition (again)
When I said in I8594d1fe05242f246a5809740a115ab2f84ac5a3 that 12 MiB
ought to be enough, I should have expected that I would be proven wrong.
While 12 MiB is enough to fit shim-x64 and grub2-efi-x64, yum fails to
update these packages to newer versions:

Transaction check error:
  installing package shim-x64-15-2.el7.centos.x86_64 needs 7MB on the /boot/efi filesystem
  installing package grub2-efi-x64-1:2.02-0.76.el7.centos.1.x86_64 needs 3MB on the /boot/efi filesystem

Error Summary
-------------
Disk Requirements:
  At least 7MB more space needed on the /boot/efi filesystem.

It is recommended that the ESP partition be much bigger. This commit
bumps its size to 550MiB, following guidelines from Rod Smith to avoid
incompatibilities with some EFIs [1].

[1] https://www.rodsbooks.com/efi-bootloaders/principles.html

Change-Id: If9515234f1a803cda32b2482f8abe10ddf0e6d26
2019-05-31 17:10:08 +10:00
Sorin Sbarnea
fb656718fb Makes image caching more resilient
Avoids failing on the first attempt to download the image to cache as
mirrors hosting them can randomly go down, usually with a connection
refused.

Change-Id: I9de9f33c2cc16596d04b35c4eb92621e6a2c7511
2019-05-31 16:31:43 +10:00
Dirk Mueller
421a0fa541 fail early when lates build information can not be fetched
When the mirror returns a error, it was trying to interpret the error
message (e.g. <html><title>Internal server error..) as a download link.
By using -f on curl we get an empty reply and an exit code, which, as
we run in set -e mode, aborts.

Change-Id: Ibaa39aedb7db286f859c4b090114c6a233b150c7
2019-05-31 16:09:25 +10:00
Zuul
ed6dfd87e5 Merge "allow the use of non-bzip compressed stages for building gentoo" 2019-05-31 04:42:07 +00:00
Nir Magnezi
433a374748 Deprecate rhel7 in favor of rhel
The rhel7 element is deprecated and is left only for backward
compatibility.
The rhel element should be used instead. Users should set DIB_RELEASE to
'7' to indicate which release you are using.

The new element is a version-less RHEL element to handle both '7'
and '8' DIB_RELEASE, which aligns with other elements which operate in
the same way such as the Fedora element.

Change-Id: Ic39ed85cacae9942448eb18ad685763f9369c2ed
2019-05-29 12:07:44 +00:00
Nir Magnezi
ee46e2f9b7 Add version-less RHEL element for RHEL7 and RHEL8
Make a version-less RHEL element to handle both '7' and '8' DIB_RELEASE.
The element usage should align with other elements which operate in the
same way such as the Fedora element.

Additionally, this patch adds support for RHEL8 that operates with
Python 3.
As of now, users of diskimage-builder will still be able to use the
'rhel7' element, or migrate to 'rhel' and specify their respective
DIB_RELEASE value.

* mount the xfs file-system for extraction as read-only.  vaguely
  based on explaination in [1] and the fact we only read the image
  data into a tar, so can ignore this.

    XFS (dm-1): Superblock has unknown read-only compatible features (0x4) enabled.

* Use the redhat system python as the dib-python version.  dib was
  ahead of it's time making an abstracted python interpreter for
  system work ;) the system python should work for running the various
  dib element scripts.

[1] https://unix.stackexchange.com/questions/247550/unmountable-xfs-filesystem

Redhat-Bug: https://bugzilla.redhat.com/show_bug.cgi?id=1700253
Co-Authored-By: Ian Wienand <iwienand@redhat.com>
Change-Id: I90540675c70bb475d9db2ae24f81c648a31f3f95
2019-05-29 11:28:53 +03:00
Matthew Thode
afcac1922c
allow the use of non-bzip compressed stages for building gentoo
Upstream is switching to xz so we need to be able to support it.

Change-Id: I382cc3e8038e2e552c553c526a990a01e51aeb12
2019-05-24 09:32:57 -05:00
Zuul
c47a9d5001 Merge "Replace git.openstack.org URLs with opendev.org URLs" 2019-05-24 08:03:47 +00:00
melissaml
a6322c6ed0 Replace git.openstack.org URLs with opendev.org URLs
Change-Id: I03e9162d5a59a2aa1631a9ecf6f6833bb7ac6050
2019-05-16 14:45:52 +08:00
Dirk Mueller
c7ac6ee0cb Update test coverage for openSUSE/-minimal to 15.0
Use openSUSE 15.0 as default, which is the latest released stable
openSUSE release. Switch to https for accessing download.o.org
as encrypted transfers should be used by default.

Remove leftovers for definitely unmaintained openSUSE 13.x images
and split into old/new leap style versioning scheme for clarity.

Change-Id: Iab129eeee2b1a2563f0f0d2cb17bbad57c068e38
2019-05-08 14:59:51 +00:00
Zuul
4665e79245 Merge "openssh-server: harden sshd config" 2019-05-07 19:35:42 +00:00
Paul Belanger
5d60979e93 Use fedora-release-common for fedora 30+
It looks like fedora-release on fedora 30+ has been split into sub
packages. Use fedora-release-common to avoid package conflicts.

Change-Id: I8f8711044fc4074b91939e0a6dfdac4d7a14a35b
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2019-05-07 12:08:50 +00:00
Zuul
8c8b856c27 Merge "Only enable dbus-daemon for fedora-29 and below" 2019-05-07 05:47:57 +00:00
Paul Belanger
38d7574127 Only enable dbus-daemon for fedora-29 and below
In fedora-30 is when we migrate to dbus-broker, fedora-29 is still using
dbus-daemon.

Change-Id: I1e1d3a3826157b8b22386c211eaa58b6439b5f3c
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2019-05-06 17:49:42 +10:00
Paul Belanger
daf5a4e4bd Switch simple-init to support python3
Depending on the version of $DIB_PYTHON_VERSION, we can either use pip /
pip3 to install glean.  This is helpful for newer OSes that might not
want to ship python2 (pip).

Change-Id: I25c5927a1eb55ee16b919dd64403184f335839b6
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2019-05-02 19:38:16 -04:00
Tristan Cacqueray
11ec95b779 openssh-server: harden sshd config
Harden sshd configuration by adding KexAlgorithms, Ciphers and MACs for sshd,
following good pratices on https://infosec.mozilla.org/guidelines/openssh

Change-Id: I3051320d867a5033e82deef10c5e723ca9829884
Co-Authored-By: Nicolas Hicher <nhicher@redhat.com>
2019-05-01 11:42:21 -04:00
caoyuan
0329a6de5e Replace git.openstack.org URLs with opendev.org URLs
Change-Id: Iac5a9da62db84365a769ea07146281866215a9c5
2019-04-29 20:15:25 +10:00
Zuul
6a4bf78e0c Merge "Fix Fedora aarch64 image location" 2019-04-18 12:22:06 +00:00
Zuul
5b47dc3a5b Merge "debian-minimal buster support" 2019-04-18 08:32:47 +00:00
Zuul
2205741de6 Merge "Also use selinuxenabled to check selinux status" 2019-04-18 08:32:15 +00:00
Pedro Alvarez
f034dd00d9 Fix Fedora aarch64 image location
It used to be considered a 'secondary architecture' but that
is not the case anymore.

Change-Id: I8e5e9cfa915c8a3c979ff9db26477c0542d271db
2019-04-09 15:50:55 +00:00
Ian Wienand
105d201e1f debian-minimal buster support
Due to the referenced bug, many versions of debootstrap can't bring up
a buster environment.  Unfortunately, these include versions we use to
do this on Xenial/Bionic nodes.

Also, there isn't backports or security updates, so elide these for
now.

I did get a working build (I haven't gone so far as a full boot+glean)
with this, at least.

Change-Id: If2420e92cb728ab6e91b0d70547da4483679b391
Paritial-Bug: #1822927
2019-04-04 16:10:08 +11:00
Serena Ziviani
19cc00041a Also use selinuxenabled to check selinux status
Currently, the cleanup script is using the existence of the folder
/sys/fs/selinux to check if SELinux is enabled. This, however, is
misleading in case disk-image-builder is used inside a Docker
container on a selinux-enabled host. In this case, the folder exists
in the container but SELinux is disabled.

This patch addresses the problem by checking, in addition to the
check already in place, the output of the command selinuxenabled.

Change-Id: I83e58f2467e60df9f0f00f7b7a58d0e2ce357a9a
Closes-Bug: #1820077
2019-03-28 14:20:24 +01:00
Zuul
36b4bc87f9 Merge "Minor clarifications in centos7 element docs" 2019-03-28 03:50:52 +00:00
Ian Wienand
20c5c98426 Replace openstack.org git:// URLs with https://
This is a mechanically generated change to replace openstack.org
git:// URLs with https:// equivalents.

This is in aid of a planned future move of the git hosting
infrastructure to a self-hosted instance of gitea (https://gitea.io),
which does not support the git wire protocol at this stage.

This update should result in no functional change.

For more information see the thread at

 http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003825.html

Change-Id: Id26bec14c3d94e2f81b2148fc85d17f07866398c
2019-03-22 01:35:42 +00:00
Daniel Abad
965d6f97aa Minor clarifications in centos7 element docs
Change-Id: I6aef77513efa37262269ca24b296acbdc823a039
2019-03-20 17:23:34 +01:00
Zuul
bdfc13a5c0 Merge "[lvm] Add Ubuntu bionic as supported distro" 2019-03-11 09:19:49 +00:00
Zuul
186db05ffb Merge "Add DIB_APT_MINIMAL_CREATE_INTERFACES toggle" 2019-03-05 07:24:24 +00:00
Logan V
8756cbea1b Add DIB_APT_MINIMAL_CREATE_INTERFACES toggle
Add a DIB_APT_MINIMAL_CREATE_INTERFACES boolean to the debootstrap
element which functions identically to
DIB_YUM_MINIMAL_CREATE_INTERFACES in the yum-minimal element.

This can be used to disable the creation of the
/etc/network/interfaces.d/eth[01] dhcp configuration files, which
are not needed on systems where cloud-init or other means are used
to configure networking.

The flag is enabled by default to keep creating the dhcp interface
files, maintaining backwards compatibility.

Change-Id: I1fdaca8350a5ceefd9e437af4fd000ce6a3ee7f3
2019-03-05 16:27:57 +11:00
Gaëtan Trellu
bbde9bb320 [lvm] Add Ubuntu bionic as supported distro
The way how LVM is created on Ubuntu Xenial and Ubuntu Bionic
is the same.

Change-Id: I16d548f6393dd3cdfd5a9befa5c0ef0f6db92df1
2019-03-04 15:05:37 -05:00
Zuul
0f8d340c6c Merge "Update gentoo-releng gpg key" 2019-02-28 00:33:06 +00:00
Matthew Thode
b2cc91d276
Update gentoo-releng gpg key
A new signing subkey is used.

Change-Id: Idc9aceba7ee144fd0307737c24991acfacf68985
2019-02-27 15:18:25 -06:00
Zuul
703549412d Merge "update spelling errors" 2019-02-25 08:49:56 +00:00
Zuul
b2e2d121f0 Merge "set rhel minor release" 2019-02-25 08:11:56 +00:00
Noam Angel
8b83196024 Add option to skip update packages
in same cases it is required to avoid update all existing packages,
doing so can result in release update which is currently not possible
unless you not include "base" element.

"base" element used for most distribution (rhel, debain), and is
necessary for most cloud operations, this patch add 
"DIB_AVOID_PACKAGES_UPDATE" parameter to skip updating all packages.

usecases for this patch can be:
 * Avoid release update when building old release ex. RHEL7.5.
 * build on network-less environment.

usage:
DIB_AVOID_PACKAGES_UPDATE=1

or
DIB_AVOID_PACKAGES_UPDATE=0

Change-Id: I71192b23c8f0bc48b348fe7377bf8a2399b53792
2019-02-25 06:31:30 +00:00
Ian Wienand
37dff9738a Fix opensuse 42.3 pip-and-virtualenv
Related to I041a141366099093805e6052b1bbf64efd277e1e, we also need to
remove this on opensuse.  The files for gate testing are added, but
the test is not added to any jobs at this point in the interests of
gate time.

Change-Id: I1af9e84d76bedcb2607717edc6d2abe2920b0584
2019-02-25 15:37:17 +11:00
Quique Llorente
5b1844acf9 Keep git after ironic-agent post
New versions of pbr depends on git [1] and IPA depends on pbr [2] so
removing git will remove pbr, IPA and friends.

[1] https://src.fedoraproject.org/rpms/python-pbr/blob/master/f/python-pbr.spec#_64
[2] https://github.com/rdo-packages/ironic-python-agent-distgit/blob/rpm-master/openstack-ironic-python-agent.spec#L85

Closes-Bug: #1816017
Depends-On: https://review.openstack.org/637668
Change-Id: I97f6b593e88e1cb81cd4bb2d77787bc012fb8271
2019-02-19 16:32:22 +11:00
Noam Angel
802dc35a61 set rhel minor release
Change-Id: I52a38c16dbbbe9fa1d4d6b6daffde01f63f664e6
2019-02-10 14:59:41 +00:00
Zuul
25ba034a0e Merge "pip-and-virtualenv: handle centos image-based builds" 2019-02-07 07:54:23 +00:00
Ian Wienand
d0906ad473 pip-and-virtualenv: handle centos image-based builds
This fixes a regression in I041a141366099093805e6052b1bbf64efd277e1e
where we starting skipping the removal of old files for image-based
builds (confusingly named centos7 rather than centos for historical
reasons).  Fix the check

Change-Id: I74688a9e91d833b5d654056431729bed0585616c
2019-02-07 10:56:29 +11:00
Zuul
dd8bbd5c4f Merge "fix systemd import-tar for gentoo" 2019-02-04 02:15:09 +00:00
Zuul
46ac931513 Merge "pip-and-virtualenv : only remove system files on centos" 2019-02-02 01:38:48 +00:00
Ian Wienand
ea1735b6a2 pip-and-virtualenv : only remove system files on centos
As described inline, we only want to remove the system package files
on centos; it causes problems on Fedora where some system tools expect
these to be there.

But there is an additional bug -- pip actually removes the system
package files anyway.  To work around this, reinstall the system
package.

Closes-Bug: #1813232
Change-Id: I041a141366099093805e6052b1bbf64efd277e1e
2019-02-01 11:01:45 +11:00
Ian Wienand
7cb5916a76 Enable dbus-broker for Fedora 29
As described in the comments, it seems the transition between
dbus-daemon -> dbus-broker in Fedora 29 has made it so the packages
can get into a state where neither service is enabled.

Explicitly install and enable dbus-broker for F29

Change-Id: I06753043a75be2f635653899c6c251b9fbdd7c67
2019-01-31 18:08:37 +11:00
Matthew Thode
f4a1c7f89f
fix systemd import-tar for gentoo
use a newer version of systemd
Fixes https://review.openstack.org/#/c/608102

Change-Id: I23fd671adb893f3abd9fbc65382f2aec5a317c24
2019-01-27 18:16:51 -06:00
Zuul
379f1bdfc0 Merge "support cracklib in pam for Gentoo's musl profile" 2019-01-21 05:03:48 +00:00
Zuul
ad23eef69c Merge "Change phase to check for dracut-regenerate in iscsi-boot" 2019-01-18 09:57:14 +00:00
Matthew Thode
1bde2591ae
support cracklib in pam for Gentoo's musl profile
Needed for any musl build, is default in other profiles

Change-Id: Ib7cae9124f5846d33c05f26befd8f13646a08610
2019-01-15 09:47:46 -06:00
Zuul
b0d41230e4 Merge "Delete the duplicate words in 50-zipl" 2019-01-10 00:17:53 +00:00
Zuul
8d3fa3a85c Merge "simple-init: allow for NetworkManager support" 2019-01-09 03:38:25 +00:00
Zuul
18c0c42c8d Merge "package-installs: provide for skip from env var" 2019-01-09 03:34:08 +00:00
Zuul
234f000e6b Merge "Update to Fedora 29" 2019-01-08 23:56:43 +00:00
Matthew Thode
cf786bb175
change to python36 for gentoo
fixes build issues since dev-util/glib-utils was updated

Change-Id: Ie8b5c425f846619ab4fc07f5bd1902dc83172a59
2019-01-08 04:59:08 -06:00
weiyj
64a8fc7c58 update spelling errors
Change-Id: Ic206b8247acce1975409329faa29deccd4f886de
2019-01-08 14:31:06 +08:00
Zuul
31f3e9dbc3 Merge "source-repositories: Replace documentation http with https links" 2019-01-08 05:16:18 +00:00
chengebj5238
079a104c1a source-repositories: Replace documentation http with https links
Change-Id: I7db3f2a1ed4e0460db60635ab00367050b0300a5
2019-01-08 15:49:41 +11:00
zhouxinyong
e3d6b8b0ec Delete the duplicate words in 50-zipl
Change-Id: Icaa3678f8b46f2a02eb9254753ed30ab8215aa7f
2019-01-07 10:02:35 +08:00
Yolanda Robla
853e13c6c3 Change phase to check for dracut-regenerate in iscsi-boot
There is an use of get_image_element_array on the environment.d
phase, for the iscsi-boot element.
This function is not available on that step. So moving the check
at next step, extra-data-d, where it is available.

Change-Id: I89cfe565492142c2f7962109360fcbcebadfd469
2019-01-02 11:05:00 +01:00
Zuul
56c72a0139 Merge "Add an element to configure iBFT network interfaces" 2018-11-30 13:46:05 +00:00
Ian Wienand
8ec3750dda simple-init: allow for NetworkManager support
This plumbs through an "--use-nm" flag to glean which instructs it to
setup interface bringup with NetworkManager rather than legacy network
enablement scripts.

In this case, install the NetworkManager package.  In the non-nm case,
also install the network-scripts for Fedora 29 -- this has stopped
being installed by default (it's been deprecated since forever).

As noted in the docs, this is currently really only relevant on the
supported rpm distros which are using the ifcfg-rh NetworkManager
plugin to effectively re-use old config files.  However,
NetworkManager has similar plugins for other platforms, so support can
be expanded if changes are proposed.

Depends-On: https://review.openstack.org/618964
Change-Id: I4d76e88ce25e5675fd5ef48924acd09915a62a4b
2018-11-30 10:02:47 +11:00
Ian Wienand
c52c383f1b package-installs: provide for skip from env var
Provide a "when" option that provides for not installing packages
based on a = or != match on an environment variable.

Unit tests are added.

Change-Id: Ifa824dccaff69fd447f45d54cb4a3083bcabdd86
2018-11-30 10:02:47 +11:00
Ian Wienand
0da1d3a419 Fix unit tests for elements
It looks like we dropped running these probably when we moved the
elements around.  For testtools to find the test scripts we need to
add the __init__.py files to make the directories look like modules.
Also prevent copying any .pyc or cache files in as hooks.

Change-Id: I66d5f6ee62cc4d9ee14c64e819b4db57d035d09f
2018-11-28 11:04:50 +11:00
Dmitry Tantsur
f0f3e3bac4 Add an element to configure iBFT network interfaces
This allows nodes with remote devices configured via iBFT to be
correctly used during Ironic introspection and deployment,
at least for non-multipath configurations.

The new element is added as a dependency for ironic-agent.

Change-Id: If3dac6504d26535593f12e851092065b688ef696
2018-11-20 14:11:11 +01:00
Zuul
d9d59b70da Merge "move selinux-permissive configure to pre-install phase" 2018-11-20 10:34:42 +00:00
Noam Angel
6f1f60983f move selinux-permissive configure to pre-install phase
install-packages is running before install.d phase, there is a chance
that installing a package like "container-selinux" will failed the
build, moving "selinux-permissive" to run at pre-install stage make
more sense.

Change-Id: I32f988be725d4b385c3765c47a00cd57c53d7d71
2018-11-19 13:13:57 +11:00
Zuul
d591c53196 Merge "fix some errors for ill-syntax in README.rst" 2018-11-19 01:59:49 +00:00
Ian Wienand
5085f23fdf Update to Fedora 29
Update builds to Fedora 29.  Remove the openstack gate CI mirror
workaround for pre-28 versions as they're not building in the gate any
more.

Change-Id: Ia6a8ae8d66d69f6add39e571043328e7274ba26c
2018-11-16 09:05:08 +00:00
Zuul
04806ea971 Merge "Increase size of EFI system partition" 2018-11-15 10:14:26 +00:00
zhouxinyong
621cbfca26 fix some errors for ill-syntax in README.rst
Change-Id: I0daf36848e8ddb09fcae6cbd55f460a8e49d209c
2018-11-13 15:05:51 +08:00
Pierre Riteau
257f9f4d1d Increase size of EFI system partition
8 MiB is not enough when using the grub2 element with centos7 images,
which installs binaries from the shim-x64 and grub2-efi-x64 packages
under /boot/efi. 12 MiB ought to be enough for anybody.

Change-Id: I8594d1fe05242f246a5809740a115ab2f84ac5a3
2018-10-31 14:38:28 +00:00
Jesse Pretorius
d59a0c8786 Add ubuntu-systemd-container operating-system element
In order to allow the simple preparation of base images which
can be used for LXC/nspawn machine containers, we add this
element.

Containers inherit a kernel from the host, so there is no need
to build a kernel into the image. All the element needs is a
base init system which, in this case, is systemd.

Change-Id: I45651de2aa1b19bdeee301094f0bdffdd0a3b45c
2018-10-31 14:22:28 +11:00
Zuul
f8c12712cc Merge "Native zuulv3 tests" 2018-10-29 05:45:18 +00:00
Ian Wienand
36d642a6a0 Native zuulv3 tests
This finalises the ports of the legacy jobs to zuul native jobs.

The dib-setup-gate-mirrors role preconfigures the repo templates,
etc. for the openstack-ci-mirrors element.

The dib-functests role runs the tests as specified by dib_functests,
and can run under python2 or 3.

Change-Id: Ied67a31f0d31503d13eccad8662c29740c93f33e
2018-10-29 12:46:15 +11:00
Zuul
1785bd3975 Merge "Fix epel repo rewrite, add to testing" 2018-10-28 23:41:21 +00:00
Ian Wienand
dfd1e4fdaf Fix epel repo rewrite, add to testing
It looks like epel-release switch from "mirrorlist" to "metalink"
(around release 7-10 Jun 2017 according to [1]).  Update our rewrite
matching to handle this "metalink" as well.

Add epel element to the centos7 (image-based) build for testing too

[1] https://koji.fedoraproject.org/koji/buildinfo?buildID=978473

Add epel element so it's tested during the centos7 functional test.

Change-Id: I2d6d4c2ec47bc69d2f16c96b5045b05c435a1af9
2018-10-24 12:39:59 +11:00
Zuul
668978b0c2 Merge "Remove redundant sources change/update" 2018-10-23 09:16:35 +00:00
Zuul
ec025ff03d Merge "Fix DIB_DISTRIBUTION_MIRROR_UBUNTU_IGNORE regex typo" 2018-10-19 04:01:21 +00:00
Zuul
07d263de0b Merge "Allow debootstrap to cleanup without a kernel" 2018-10-19 03:23:09 +00:00
Zuul
22cd19fea7 Merge "ubuntu-common: Update default DIB_RELEASE to bionic" 2018-10-19 03:23:08 +00:00
Zuul
39c6d7c747 Merge "Move common ubuntu environment setting to ubuntu-common element" 2018-10-19 02:50:36 +00:00
Zuul
46ec4992ff Merge "Add support for Fedora 28, remove EOL Fedora 26" 2018-10-18 23:44:51 +00:00
Ian Wienand
86080a7db0 Fix DIB_DISTRIBUTION_MIRROR_UBUNTU_IGNORE regex typo
This should not be quoted.  Introduced in
I6ffbde07fa0e103641ee5c5f9d9e854e5b2168dc

Change-Id: Iac54c86ebdb90351a2dd663b876ab4675ee0435a
2018-10-18 21:49:44 +11:00
Paul Belanger
7173658df2 Add support for Fedora 28, remove EOL Fedora 26
This updates diskimage-builder to support current Fedora releases (27
and 28) and removes support for Fedora 26 which is EOL as of June
2018.

Change-Id: I602b22ed4d5397b39dc1eef67964f6fbdcd93060
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-10-18 19:43:12 +11:00
Zuul
e09faf9a8d Merge "simplify overlay logic for Gentoo" 2018-10-18 05:51:05 +00:00
Zuul
b3d405f624 Merge "simplify python3.6 selection on gentoo" 2018-10-18 05:51:05 +00:00
Zuul
763493e1f1 Merge "enable caching for gentoo builds" 2018-10-18 05:51:03 +00:00
Zuul
2d327c2506 Merge "Set EPEL mirror during openstack-ci-mirrors" 2018-10-18 04:40:33 +00:00
Zuul
fe15b197fb Merge "ubuntu: Add options to ignore mirror components and use insecure repos" 2018-10-18 04:34:00 +00:00
Zuul
56634e0938 Merge "Turn down pkg-map and hook copy tracing output" 2018-10-18 03:56:12 +00:00
Zuul
d3d17b2e43 Merge "Fix DIB ubuntu-minimal running on bionic (18.04)" 2018-10-18 03:08:37 +00:00
Ian Wienand
8b0111477e Set EPEL mirror during openstack-ci-mirrors
We want to set DIB_EPEL_MIRROR for the epel element, which means we
need access to the CI mirror info script in the chroot.  Copy it into
the temp directory with extra-data.d and update the environment file
to find it.

Change-Id: Ia12f0cbdeb194eef3155497ceb5ffc4a452aad76
2018-10-18 13:53:53 +11:00
Ian Wienand
c47ee6e121 ubuntu: Add options to ignore mirror components and use insecure repos
When using the upstream cloud images with the "ubuntu" element, they
have universe and multiverse enabled which we don't mirror.

To use the infra mirrors as a DIB_DISTRIBUTION_MIRROR with this
element, we need to be able to skip redirecting to universe and
multiverse, and additionally enable insecure repos (as we don't gpg
sign our mirrors).

Add and document two new variables with the ubuntu element to do this.
This is then setup by the openstack-ci-mirrors element so that we use
local mirrors duing dib functional testing for the "ubuntu" element.

Change-Id: I6ffbde07fa0e103641ee5c5f9d9e854e5b2168dc
2018-10-18 13:53:48 +11:00
Matthew Thode
3ad8235180 simplify overlay logic for Gentoo
openssl/cryptography versions are updated/stable
musl profiles need newer versions of open-iscsi masked as upstream
doesn't want to work with multiple libcs

Change-Id: If5baf339516390ae332015928557c6bb734486c2
2018-10-18 00:21:31 +00:00
Matthew Thode
1e257f59cc simplify python3.6 selection on gentoo
python 3.6 is now stable in gentoo

Change-Id: Ide96c039b03817c216ab58930ff5c897556e6c8a
2018-10-18 00:21:16 +00:00
Ian Wienand
6c077d7c2a Turn down pkg-map and hook copy tracing output
This is a lot of very low value noise in the logs as these iterate
through all the elements (often doing nothing).  Turn it down and add
an echo so we just see what elements it is working on.

Change-Id: I0687de4722766189db9d4a7bd7d3cfb45d387b62
2018-10-18 11:03:17 +11:00
Matthew Thode
89f95a5351
enable caching for gentoo builds
To facilitate this I've created two new environment variables to set
the environment and default options for package actions.

eval is needed for the export as it preserves quotes.

Change-Id: Ib03651ee8dacd48cd1c135afd57cd31101356056
Signed-off-by: Matthew Thode <mthode@mthode.org>
2018-10-15 23:17:37 -05:00
Ian Wienand
39fb794ebc Allow debootstrap to cleanup without a kernel
Currently there's more-or-less an assumption that a kernel is
installed, so module blacklists are simply echoed into the modprobe
blacklist.  This may not be the case with some ongoing container work.

Although we don't need to blacklist modules for containers, it also
doesn't hurt.  Move the debootstrap element to the new modprobe
element, and allow it to create the blacklist directory.

Change-Id: I0f057caf473951df56a2af9633e3a5b53e0809b1
2018-10-08 16:02:25 +11:00
Pierre Riteau
bacceba41d Fail build due to missing kauditd only when SELinux is enabled
With the check added in commit 7566819139,
diskimage-builder fails to build RPM-based images if kauditd is not
running. However, this is only valid for environments where SELinux is
enabled. If SELinux is disabled (which is identified by an empty _runcon
variable), proceed with running setfiles.

Change-Id: I1b056f20a3a55f7333391207d9e1049d25ece041
Closes-Bug: #1779273
2018-10-05 10:02:14 +01:00
Zuul
3e6d658687 Merge "allow building non-gentoo images on gentoo hosts" 2018-10-03 11:08:11 +00:00
Michael Johnson
330626482a Fix DIB ubuntu-minimal running on bionic (18.04)
Diskimage-builder fails to build ubuntu-minimal images when run on
a Ubuntu bionic-beaver (18.04) instance.
The user gets "Couldn't create tempfiles for splitting up" when
apt-get update is run in the ubuntu-minimal element root.d.

The issue is that the /tmp inside the chroot is not getting the
proper permissions applied from the base-files package. This is likely
because the pip-cache element has already created the directory before
the base-files package is installed.

This patch changes the order of pip-cache to root.d/11-pip-cache so that
it runs after teh base OS root.d elements run.

Change-Id: I6fd1cb2a23422206884165eb502b260f0c1e52f7
2018-10-01 19:59:30 -07:00
Jesse Pretorius
f4c5326b8e ubuntu-common: Update default DIB_RELEASE to bionic
The ubuntu-minimal README states that the latest Ubuntu LTS
is the default, but currently that is not true. This patch
changes the default to the current LTS.

Change-Id: I10f28314d1a5969c20094194637cfe31219d228c
2018-09-20 02:06:04 +01:00
Jesse Pretorius
8c69da65ac Remove redundant sources change/update
The apt sources are set out in root.d/75-ubuntu-minimal-baseinstall
and the cache is updated, cleaned and a dist-upgrade is done there.

As such, this file is unnecessary.

Change-Id: Idab5ede3f235bc204c4bdebf40fbcf4a12e5bc2f
2018-09-20 00:05:07 +00:00
Jesse Pretorius
30493f7655 Move common ubuntu environment setting to ubuntu-common element
The ubuntu, and ubuntu-minimal elements both make use of a common set
of environment settings to determine the distribution name.

The ubuntu-minimal element also does a few extra things which would
appear to apply to both sets and bring in extra architecture support.
As such, these are included in the common element.

This intends to be part of a series of patches which will eventually
create a new element to build a minimal ubuntu-systemd-container
element which can be used for lxc/nspawn containers.

Change-Id: Ia4e620f7d3fa6215484a8d218cea2f28bd1ffaee
2018-09-20 00:04:55 +00:00
Zuul
b86af3dc6a Merge " IPA requires iptables" 2018-09-13 02:05:42 +00:00
Ian Wienand
f6a2452d4c Only append DIB_BOOTLOADER_DEFAULT_CMDLINE to default grub entry
The grub.cfg has two variables [1]

 GRUB_CMDLINE_LINUX : used on all boots
 GRUB_CMDLINE_LINUX_DEFAULT : additionally used on all "normal" boots

The problem with I2298675dda1f699c572b3423e7274bc8bd7c1c9d is that it
appened the values in DIB_BOOTLOADER_DEFAULT_CMDLINE to both of these,
resulting in duplicated arguments.  I don't think we considered that
GRUB_CMDLINE_LINUX_DEFAULT actually already appends to the
GRUB_CMDLINE_LINUX values.

Make DIB_BOOTLOADER_DEFAULT_CMDLINE only append itself to
GRUB_CMDLINE_LINUX_DEFAULT.  That seems to line up sensibly with the
name of the variable.

Documentation is enhanced around this, and a releasenote added.

[1] https://help.ubuntu.com/community/Grub2/Setup

Change-Id: I76b5442a9090c19a6540ed2d4ab324546f241ebf
Closes: #1791736
2018-09-13 09:51:50 +10:00
Zuul
b29aee3383 Merge "Install sudo on Gentoo images by deault" 2018-09-11 00:09:33 +00:00
Zuul
9f93b2ce3d Merge "Fix CentOS image build failure when dib runs on system where audit disabled" 2018-09-06 11:24:59 +00:00
Olivier Bourdon
7566819139 Fix CentOS image build failure when dib runs on system where audit disabled
Without this fix, building a CentOS image on Ubuntu where audit=0 is passed
as a kernel boot parameter will lead to the following error:
disk-image-create centos7 dhcp-all-interfaces cloud-init-nocloud \
    devuser yum epel baremetal
... dib-run-parts Running tmpdir/hooks/cleanup.d/99-selinux-fixfiles-restore
... Error connecting to audit system.

Change-Id: I229d9b72f88bffddca42da57f01c27e902427071
2018-09-04 08:23:02 +02:00
Charalampos Kominos
c85141291e Fix bootloader packages for aarch64
Due to the arm naming convention, building centos images for arm64 and
aarch64 does not yield the same result. In order to locate grub2 on
aarch64 the correct mapping is added.

Change-Id: I1bb227b2523e420e394fec8c52c6c79fcdd31c53
Closes-Bug:#1789414
Signed-off-by: Charalampos Kominos <Charalampos.Kominos@enea.com>
2018-08-31 17:01:47 +02:00
Zuul
b602c05551 Merge "Add netcat to redhat-common map-packages" 2018-08-23 11:26:12 +00:00
Zuul
f33526d20c Merge "better handle existing keywords files/directories" 2018-08-10 06:07:06 +00:00
Carlos Goncalves
138b14b010 Install ca-certificate with redhat-common
Change-Id: I36d44ca8b4f966476657ec2bb1ecc1458bb524fd
2018-08-04 00:30:49 +02:00
Carlos Goncalves
cf553ce0ac Add netcat to redhat-common map-packages
Change-Id: I273038611febe5e30f30dd8d3ba8990dbdc94647
2018-08-03 17:20:19 +02:00
Zuul
53186f23a0 Merge "cache-url requires curl" 2018-07-31 00:20:56 +00:00
Oliver Walsh
a514feba99 Add DIB element to blacklist nouveau
This ensures nouveau is not loaded at boot, which is required when installing
NVIDIA GPU drivers and to avoid issues with PCI passthrough of NVIDIA GPUs.

The option to disable kernel modesets ensures that it can be unloaded again if
it happens to be loaded after boot (e.g manually or implicitly by X).

bp tripleo-vgpu

Change-Id: I60815de86e7b22dfb39555af9d2d53564841e2ab
Related-bug: 1774674
2018-07-25 16:35:58 +01:00
Oliver Walsh
73e27a8504 modprobe DIB_MODPROBE_BLACKLIST should be optional
modprobe element currently fails when DIB_MODPROBE_BLACKLIST is not set.
As there are now two methods to control blacklisting this should be optional.

Change-Id: Ibf3c31a95177ba88c1b93228490c7f36f5b70b57
2018-07-25 16:33:47 +01:00
Zuul
9adf12fe4a Merge "Fix for proper LVM support" 2018-07-24 07:32:11 +00:00
Clint Byrum
d8907e78b5 cache-url requires curl
In some cases cache-url can get pulled in without curl, causing it to
fail.

Co-Authored-By: Adam Harwell <flux.adam@gmail.com>

Change-Id: Ibd66c2ca4f8cc423783555d8a99b1184f43adff2
2018-07-23 09:56:58 +00:00
Zuul
b79952af2a Merge "Add new modprobe element" 2018-07-11 07:21:40 +00:00
Zuul
bbf69a90f3 Merge "Fix /etc/network/interfaces file contents" 2018-07-10 07:46:56 +00:00
Zuul
2343d4b577 Merge "Add keyring if supplied" 2018-07-10 07:46:55 +00:00
Olivier Bourdon
caf565673b Fix for proper LVM support
Without this fix, a LVM based ubuntu-minimal image will fail
booting due to the fact that the boot process will not be able
to retrieve the root filesystem using LABEL=(cloud)img-rootfs

Change-Id: If4ecf65868563f7b799160a58af6312bedf956bf
2018-07-09 14:15:57 +00:00
Hironori Shiina
7e4e6cfff4 Add expected semicolons for dhclient.conf
This patch adds an expected semicolon to an end of statement in
dhclient.conf for dhcp-all-interfaces element. Without this fix, an
error occurs when an image is booted with a message,
'semicolon expected.'.

Change-Id: I8311dbc67cc2815223111da01e7a7517c7d6f059
2018-07-06 13:42:25 +09:00
Sam Yaple
c144246cc9
Add keyring if supplied
When building with debootstrap, debootstrap will use the key to check
that everything is properly signed. It will not `apt-key add` the key
into the final environment, however.

Early adding the key after debootstrap before we need to read from the
private repo again prevents unsigned issues. This also maintains the
integrity of the packages in the environment throughout the build.

Change-Id: I5ca75ae4620c9fb26b512cb30f8cd79fa7a0373a
2018-07-02 14:33:35 -04:00
Zuul
927e8115f6 Merge "Fix bootloader for efi on rhel systems" 2018-06-28 15:02:30 +00:00
Yolanda Robla
31383970c7 Add new modprobe element
This element will replace modprobe-blacklist element. It wil
still have the blacklist functionality, but it also adds
the feature of passing a complete file with settings to the
modprobe.d directory. Adding this functionality, that will
allow elements that depends on this module, to just copy the
specified files to the final directory.

Change-Id: I9a44f7d11520b8b1e604956d3c1db2fc7e2bf457
2018-06-28 13:55:53 +02:00
Matthew Thode
b9f1c7a22f
better handle existing keywords files/directories
The existing directories are needed for stage building (a part of the
Gentoo build process).  Normally these directories are empty, but there
are times where overrides need to be defined.  This commit handles
existing overrides for keywords.  For historical reasons the overrides
were able to be put in different files and directories, this
centralizes them.

This also updates the version of openssl/cryptography that works with
or without bindist.

Change-Id: I62c934ed305a711a4a9a3ef01fa55ad142aebb78
2018-06-25 09:20:52 -05:00
Hoang Trung Hieu
a4648872ba Add iscsi-boot element for CentOS images
This patch adds an element that handles the configuration for
creating a disk capable of being a remote root filesystem through
iSCSI on CentOS images.

Tested on Fujitsu Server and boot with BIOS and UEFI mode successfully.
- Tested Boot-From-Volume + EFI for centos7 with following elements:
  "centos7 vm devuser cloud-init-datasources dhcp-all-interfaces
   iscsi-boot dracut-regenerate block-device-efi"

Co-authored-By: Nguyen Van Trung <trungnv@vn.fujitsu.com>

Change-Id: Ia1f23d722dced6f254fd7aee86abe8066a72fa42
2018-06-25 12:01:52 +00:00
Olivier Bourdon
11d91501d0 Fix /etc/network/interfaces file contents
According to http://bit.ly/2HA4oDO and
the official Ubuntu manual
http://manpages.ubuntu.com/manpages/xenial/man5/interfaces.5.html
source-dir support has been removed from Ubuntu >= 16.04/Xenial

Once an image is generated and booted, moving the dhcp interface(s)
declaration(s) from /etc/network/interfaces into specific subentries
of /etc/network/interfaces.d and calling 'service networking restart'
just make your instance unreachable and all interfaces are left
unconfigured.

This patchset fixes this issue

Change-Id: I6b6b99c81490c874c5db5405c2fbf3c180c87464
2018-06-19 11:26:21 +02:00
Yolanda Robla
61e6566c48 Fix bootloader for efi on rhel systems
When building the image on a non-efi environment, it generates
linux16/initrd16 entries. But to boot from UEFI they need to have
linuxefi/initrdefi entries.
Use sed to replace those entries, in case we have an EFI image.

Change-Id: I47c96450e10f34b91bcc32888532bd7ab87cf316
2018-06-19 08:37:30 +02:00
Zuul
e210f79500 Merge "Don't run setfiles on /boot/efi" 2018-06-15 08:42:13 +00:00
Nguyen Van Trung
8d86ff1bec Don't run setfiles on /boot/efi
setfiles isn't supported on the vfat /boot/efi partition.  Add it to
the skip list.

Tested on Fujitsu Server successfully.

Change-Id: Iab262c4bdb0ecc25ca6b77ee4aff1ce442c0c578
2018-06-15 14:53:38 +10:00
Michael Turek
91e3b72a23 Add iscsi-boot element
This patch adds an element that handles the configuration for
creating a disk capable of being a remote root filesystem through
iSCSI on Ubuntu and Debian images.

Change-Id: Ibf9e39d2bdab530106015f156d23d28029d12b0d
Closes-bug: #1716794
2018-06-14 08:56:03 +07:00
Yolanda Robla
bfc60958bf Fix bootloader packages for rhel
When using uefi in rhel, the package mapping is incorrect.
We need to add specific grub-efi* mappings to use grub2-efi

Change-Id: I2db96ae85fd5e4638c794015b2f8164c018420e3
2018-06-08 17:14:19 +02:00
Zuul
fc3748f49b Merge "Fix encoding issue during processing output" 2018-06-06 07:58:14 +00:00
Zuul
225c09b245 Merge "elements: pip-and-virtualenv: Handle openSUSE Leap 15" 2018-05-31 21:54:59 +00:00
Zuul
c4c2fb746c Merge "Remove duplicate GRUB command line entry" 2018-05-31 16:00:29 +00:00
Markos Chandras
f37e85d547 elements: pip-and-virtualenv: Handle openSUSE Leap 15
We need to handle openSUSE Leap 15 when installing pip and virtualenv
packages. This fixes the following problem when the pip-and-virtualenv
elements is used:

2018-05-31 09:42:12.014 | + [[ opensuse = opensuse ]]
2018-05-31 09:42:12.014 | /tmp/in_target.d/install.d/04-install-pip: line 57: packages: unbound variable

Change-Id: Id7911b0a0836fa8dcc003e23fa515b78fba67126
2018-05-31 10:55:28 +01:00
Zuul
4c04b46db6 Merge "Allow to rebuild arbitrary images" 2018-05-29 17:06:48 +00:00
Zuul
df9402c81c Merge "elements: zypper-minimal: Add support for openSUSE Leap 15.X" 2018-05-25 08:55:53 +00:00
Roman Gorshunov
84eea81efe Allow to rebuild arbitrary images
Patch allows to rebuild arbitrary images, which location, filename and
sha256sum are specified in variables, not only hardcoded $DIB_RELEASE/current.

Change-Id: I05418932a0c40d885fe00a49f1f49d7e86c67518
2018-05-24 10:19:59 +00:00
Zuul
668c93b118 Merge "Replace the ubuntu-minimal trusty test with a bionic one" 2018-05-21 06:01:53 +00:00
Ian Wienand
82eb1ca837 Replace the ubuntu-minimal trusty test with a bionic one
Add a bionic test in replacement of trusty.  We are already building
bionic images in the gate, so this seems like a good time to switch.

Change-Id: I20d4c25e9b79e7326c86767c36be8615ba0888a3
2018-05-21 12:51:31 +10:00
Roman Gorshunov
09af52a08c Remove non-maintained ubuntu-core element
Removing no longer working and no longer maintained ubuntu-core element, which
intent is unclear, and not documented.

Change-Id: Id847591d04fd7cd32c8903967da01ee0d303b267
Closes-Bug: 1771614
2018-05-18 09:28:02 -07:00
Markos Chandras
bb9ad09010 elements: zypper-minimal: Add support for openSUSE Leap 15.X
openSUSE Leap shares the same repo structure with 42.X

Change-Id: I23de11d81020c8aae641dfc01c1cbddf769f5c75
2018-05-18 11:55:56 +01:00
Stanislav Makar
2e6a19a018 Add Ubuntu 18.04 support
Use squashfs for more recent Ubuntu releases

Change-Id: I80df28be6e2a5e03ae1450e84fc05715f21a7750
Closes-bug: 1766850
2018-05-18 14:47:20 +10:00
Matthew Thode
e1f01d513a
IPA requires iptables
iptables -L is used in log collection

Change-Id: Id7e69a9e9f87db6621b928a2104df4b94f10044e
2018-05-17 11:14:44 -05:00
Doug Szumski
c1b1534c87 Remove duplicate GRUB command line entry
Without this change DIB appends a second command line entry to the GRUB
config. This causes the original command line entry to be ignored
when Linux is booted.

The expected behaviour is that DIB appends to the existing entry as
it does for Ubuntu and SUSE.

Following discussion on the review, this also removes the distro specific
switch statement, as update-grub just calls grub-mkconfig, meaning that
there was nothing distro specific in the first place.

Change-Id: I2298675dda1f699c572b3423e7274bc8bd7c1c9d
Closes-Bug: #1771366
2018-05-16 09:25:59 +01:00
Zuul
a8df61edbb Merge "rpm-distro: set the contentdir yum var" 2018-05-14 12:36:11 +00:00
Zuul
d79a7aaf01 Merge "Add pip cache cleanup to pip-and-virtualenv" 2018-05-14 09:03:49 +00:00
Zuul
1412e957ba Merge "Fixes add-apt-keys in dpkg element" 2018-05-14 08:52:44 +00:00
Tristan Cacqueray
25964c5c5b rpm-distro: set the contentdir yum var
Since CentOS-7.5, a new yum variable is needed for SIG repositories.
This change replicates the %post task of the centos-release package
to setup the contentdir yum var. This should fix issues when
repository url uses $contentdir and yum fail with:
http://mirror.centos.org/%24contentdir ... [Errno 14] HTTP Error 404 - Not Found

For more details see:

https://lists.centos.org/pipermail/centos-devel/2018-March/016542.html

This change also drops support for fedora without dnf.

Change-Id: I1819a48b94670577b0c5e29b24cebfb20ea07d28
2018-05-14 05:38:37 +00:00
XiaojueGuan
93b42f0f01 Trivial: update url to new url
Change-Id: I8d26fd0598626c8666a1852724f0620af9dc028d
2018-05-13 23:06:00 +08:00
Oded Le'Sage
10b2a5a4ee Fixes add-apt-keys in dpkg element
This commit addresses the issue described in bug 1768354 when using the
apt-sources element and adding a key to a custom repo, subsequent deb
package installs fail due no update of the repo before package install

Change-Id: I968b3422fab2fb2305426d49215391d8ba7499df
Closes-Bug: 1768354
2018-05-09 12:06:20 -05:00
Michael Johnson
7e79a933db Add pip cache cleanup to pip-and-virtualenv
The pip-and-virtualenv element provides pip inside the created images.
When this pip is used inside the chroot of the image it, by default, creates
a cache directory with the http packages and the resulting wheels.
In the case of the octavia ubuntu-minimal image this cache is using ~50MB of
cache that is not needed after the image finished building.

This package creates a finalise.d task that removes the cache from the
default location.

Change-Id: I4715437b068d04993ef755bd1e27963db1d22417
2018-05-09 09:31:17 -07:00
Matthew Thode
f7818e4a57
Install sudo on Gentoo images by deault
Change-Id: I30640017e3c652a467be421169029045a8b675d2
Close-Bug: #1766484
2018-04-24 11:44:39 -05:00
Tristan Cacqueray
abd63b01aa pip-and-virtualenv: fix install-pip when centos-release-openstack is enabled
When RDO projects repository is installed, the python-setuptools
package is obsoleted by python2-setuptools, this makes the install-pip
script failed:

  Package python-setuptools-0.9.8-7.el7.noarch is obsoleted by
  python2-setuptools-22.0.5-1.el7.noarch which is already installed

Then the "rpm -ql python-setuptools | xargs rm -rf" exit 1.  Check if
we have a record of the updated package obsoleting then old one; if
so, use it.

Change-Id: I2b0051bd9e81908c187098a7b82e120b999b111d
2018-04-23 08:18:04 +10:00
Paul Belanger
99eb9a2d7d
Fix epel element for centos-minimal
We no longer install wget / yum-utils for centos-minimal, this fixes
that.

Change-Id: I8d89026bd48cf7398cc1cbe41e3b7f00f682dbb8
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-04-18 20:15:07 -04:00
Paul Belanger
12a760f7b7 Revert "debootstrap: Call update-initramfs explicitly"
This currently breaks glean on rackspace, revert until we can figure
out why that is.

This reverts commit 43bc352c59.

Change-Id: Iae88a3b0457bab0b8f0fd1febf58732ca95e5dc9
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-04-17 21:47:46 +00:00
Ian Wienand
b423292cd0 Remove installed packages before pip install
The release of pip10 has shown up a few issues here

Firstly, pip10 now refuses to overwrite distutils installed packages,
which includes "python-virtualenv" on centos.  History has shown us
that we want the packages installed and overwritten, to avoid the
packages coming back and messing things up.

Pre-install all the packages, then list the files in the packages with
"rpm" directly and remove them.  This way pip is happy to install.

We need to take better account of the package names for this; on
Fedora things have switch to "python2-virtualenv" instead of
"python-virtualenv" and we can't use an alias to list the package
contents.

This also highlighted that python2-pip is in EPEL for centos, so
enable that when we install it.  Make the epel element a no-op for non
centos/rhe distros.

There is a related change in recent fedora that python3 now installs
binaries into /usr/local/bin.  There are commented swizzles in here to
ensure we retain the status quo of "pip" and "virtualenv" both being
python2 based, with the python3 versions being called explicitly
"pip3" and "virtualenv3" respectively.

Change-Id: I2ffdd9f615ae6b00428c17249e4f216774991b99
2018-04-17 16:09:04 +10:00
Matthew Thode
2dd6dd357a
allow building non-gentoo images on gentoo hosts
Can't use a variable that's not set.

Change-Id: I4a7cf2ac47c2254da2fec778437f67c1fe3707f7
2018-04-12 13:24:59 -05:00
Zuul
0769bfd4aa Merge "Set the dhclient timeout to match DIB_DHCP_TIMEOUT" 2018-04-11 10:12:45 +00:00
Ian Wienand
f52b385818 Don't only install python3-virtualenv
We added this sed in I422490ebe9a9c655552685bc2ff342d288335a9c to
avoid installing python2 packages on python3-only systems and thus
dragging in all of python2.

We made a similar change to python-pip in
I7d8ba9300039cce90965410a4e16ca9e711904c3; however we realised that
the gate (and other consumers) were relying on this element having
installed the python2 & 3 packages for consistency -- otherwise jobs
would install the python-pip packages and overwrite the
pip-from-source and mess everything up.  We reverted that in
I419dbdf4682394db68974944af1e5c432f3e0565 and added some clearer notes
that this element brings in python2 & 3, and if you want something
that doesn't do that then this element isn't for you.

However, we never fixed up the virtualenv package install -- currently
our Xenial images have a global virtualenv installed from source, but
the python-virtualenv packages aren't installed.  Thus if a job does
"apt-get install python-virtualenv" it overwrites the from-source
virtualenv with older parts and again messes everything up.

Probably most jobs just call "virtualenv" and assume it is there;
however in bringing up some rspec test for puppet I have hit this
issue as some modules specify dependencies on the virtualenv packages.

Thus install the python-virtualenv AND python3-virtualenv packages in
this element.

Change-Id: Ia84c38dc3c40a6080e144b563e10abca7dac2881
2018-04-10 12:34:03 +10:00
Derek Higgins
97399e9bb1 Set the dhclient timeout to match DIB_DHCP_TIMEOUT
On initial boot when networking is brought up by cloud-init this
is the timeout that dhclient adheres to. Centos configures
"timeout 300" (for an EC2 bug) in their cloud image, which results
in a 5 minutes delay to boot in cases where no dhcp available (e.g. IPv6
SLAAC). To reduce this boot delay and to provide consistency with
places where we have set other dhcp timeouts set this to DIB_DHCP_TIMEOUT.

Change-Id: I119a002070501c3dfe7c6730b07ee25f422b85b0
Related-Bug: #1758324
2018-04-05 14:29:22 +01:00
Zuul
09e5b2d357 Merge "debootstrap: Call update-initramfs explicitly" 2018-03-29 06:02:58 +00:00
Zuul
c07e7349d5 Merge "Change the GENTOO_PORTAGE_CLEANUP variable default" 2018-03-29 05:33:55 +00:00
Zuul
e8744cf32b Merge "zypper-minimal: Set default locale env to C.UTF-8" 2018-03-29 05:33:52 +00:00
Zuul
e444de83e6 Merge "Clean up dib-python symlink" 2018-03-29 05:33:51 +00:00
Andreas Florath
43bc352c59 debootstrap: Call update-initramfs explicitly
Many elements install additional distribution packages.
In addition the user can provide a set of packages to be installed
via the '-p' switch.
Some of them influence the boot process and therefore the initramfs
needs to be updated. Because the package manager during the image
creation process is configured not to run package scripts, this needs
to be done explicitly.

This issue was found during development and debugging of the
block-device LVM plugin: Even when the e.g. the lvm2 package
was installed in the image, it was missing in the initramfs
because of the missing update.

Change-Id: I7c92033b3ca80cdd23d081002059d83ca3f53bdb
Signed-off-by: Andreas Florath <andreas@florath.net>
2018-03-29 04:14:52 +00:00
Matthew Thode
6a6d78e63c
Change the GENTOO_PORTAGE_CLEANUP variable default
Default the GENTOO_PORTAGE_CLEANUP to True.  By default we should not
ship package info, this bloats the image and is usually outdated by the
time it'd be consumed.

Change-Id: I14c2530d91807cbc6a3806e01c7e4f6f472b190d
2018-03-26 23:26:49 -05:00
Clark Boylan
301eac8e8b Fix element-provides in debian element
The debian element depends on debian-minimal now which provides
operating-system. This means that the debian element can no longer
provide operating-system and doing so results in an error when using the
debian element.

The fix is simple just rely on the fact that debian-minimal provides
operating-system and remove this element-provides from debian.

Fixes-Bug: 1758000
Change-Id: I524feeb82c19046ec987eb1186c7f4568309e559
2018-03-26 10:58:04 -07:00
Zuul
ffc06874ef Merge "install sudo in the devuser element" 2018-03-26 00:58:57 +00:00
Zuul
8ab38dc5cf Merge "Update Fedora defaults to 27" 2018-03-25 23:41:45 +00:00
Zuul
9d751463f1 Merge "enable systemd profile for Gentoo" 2018-03-23 19:01:35 +00:00
Zuul
45ff8175b6 Merge "proliant-tools: add net-tools package to support hpsum utility" 2018-03-23 05:36:12 +00:00
Matthew Thode
cfa7935e43
enable systemd profile for Gentoo
Change-Id: Id3ac1d97b280f10f9938a60c4871d08f59b85002
2018-03-22 15:12:59 -05:00
Matthew Thode
cfa5b237c0
install sudo in the devuser element
The devuser element can set up passwordless sudo, which requiers the
/etc/sudoers.d directory, which requires the sudo package, so we ensure
the sudo package is installed.

Change-Id: I80d6c669d4ac0d97b49d01cb621bf05b8e7f8ef1
2018-03-22 00:16:09 -05:00
Matthew Thode
5e2f3646ad
remove portage git directory
this shrinks thinks by ~50%, from 722M to 352M

Change-Id: I1267cc05700ee28c45a331de7f571b9ee075c6b5
2018-03-16 18:29:40 -05:00
Zuul
651d913fcc Merge "arm64: use HWE kernel and fix console" 2018-03-16 08:31:14 +00:00
Zuul
5e2168aefd Merge "Choose appropriate bootloader for block-device" 2018-03-16 08:31:12 +00:00
Zuul
96af247400 Merge "Add block-device defaults" 2018-03-16 08:31:10 +00:00
Zuul
c60a20b59d Merge "GPT partitioning support" 2018-03-16 08:14:27 +00:00
Anshul Jain
a7135a0d8f proliant-tools: add net-tools package to support hpsum utility
Hpsum utiltity of proliant-tools requires net-tools to be installed
as part of base image. This commit adds support for installation of
net-tools for all distros.

Change-Id: I2a1e81059ed1aee975db78cfa5e61bbf1b98e06f
Closes-bug: 1751777
2018-03-09 02:29:23 -06:00
Zuul
30d5ee2ec7 Merge "Fix for passing user defined value for satellite cert for rhel-common." 2018-03-08 07:05:40 +00:00
Zuul
dda4e2e0f1 Merge "secondary architectures use different url" 2018-03-08 07:05:39 +00:00
Zuul
d2133ec5e7 Merge "Fix for rhel7 iso image creation." 2018-03-08 07:05:38 +00:00
Mark Hamzy
cdb423eeb9 secondary architectures use different url
The Fedora-Cloud-Base*qcow2 images are stored on a different
server for secondary architectures.

Change-Id: I90d48ce4175fd251e8f5ab7a70190ad952256a94
2018-03-01 19:38:14 -06:00
Tobias Henkel
b62ed1823c
Fix encoding issue during processing output
When using the package-installs element there can be some encoding
problems if the package installation emits unparsable output
[1]. However in this case we just want to forward the output to the
console which normally can handle this correctly. In order to fix this
switch off universal_newlines processing such that we just operate on
bytes.

Further we have to decode the lines without setting the locale and
ignoring errors. This is required because print encodes without
setting the locale and thus we need to filter/modify the stream such
that it doesn't crash.

[1] Traceback:
2018-03-01 09:58:00.515 | Traceback (most recent call last):
2018-03-01 09:58:00.515 |   File "/usr/local/bin/package-installs-v2", line 137, in <module>
2018-03-01 09:58:00.515 |     main()
2018-03-01 09:58:00.515 |   File "/usr/local/bin/package-installs-v2", line 130, in main
2018-03-01 09:58:00.515 |     process_output(install_args, follow=True)
2018-03-01 09:58:00.515 |     for line in iter(proc.stdout.readline, ''):
2018-03-01 09:58:00.515 |   File "/usr/lib/python3.5/encodings/ascii.py", line 26, in decode
2018-03-01 09:58:00.515 |     return codecs.ascii_decode(input, self.errors)[0]
2018-03-01 09:58:00.515 | UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 34: ordinal not in range(128)

Change-Id: Ie4af9b4523459a630cfb98d09093bfe9ef7aa61e
2018-03-01 16:09:25 +00:00
Anshul Jain
73a9ba0f1f Fix for rhel7 iso image creation.
Currently rhel7 image creation fails because it tries to copy
default bootloaders which is ubuntu way. This commit updates `iso`
element to correct the path of bootloaders required for rhel image.

Change-Id: I526d75b2db609fc77be0fc778b4d00f2d3df38ec
Closes-bug: 1750725
2018-02-28 06:31:34 -06:00
Zuul
3fb0950808 Merge "update Gentoo Hardened profiles (now stable)" 2018-02-28 11:42:51 +00:00
Zuul
05a44e2181 Merge "Checking link status according to DIB_DHCP_TIMEOUT" 2018-02-28 11:42:50 +00:00
Anshul Jain
b37a1e27cb Fix for passing user defined value for satellite cert for rhel-common.
For 'satellite' mode of registration, rpm for rhel SSL certificate is
hard coded to 'katello-ca-consumer-latest.noarch.rpm'. This commit adds
functionality that provides an option to set this as defined in their
satellite server.

Change-Id: Ib176cfa209f5ac8a4b5da71419327b4237330904
Closes-Bug: 1749947
2018-02-28 02:43:39 -06:00
Ian Wienand
e9ed983324 arm64: use HWE kernel and fix console
Install hwe kernel for ubuntu-minimal.  As noted this is currently
Xenial specific; we need this for initial bring-up so let's tackle
future releases as things progress.

Ensure we use ttyAMA0 for arm64 console too.

Change-Id: Ic607cf8369666dc24929aff6f2ef8a72e7980599
2018-02-23 10:04:48 +11:00
Ian Wienand
7b4c8abce3 Choose appropriate bootloader for block-device
In the prior change we added block-device-[mbr|gpt|efi] elements to
create appropriate disk-layouts.

This adds an environment flag to each so the bootloader can install
the right thing.  The EFI install path is updated to work with this
(this part a copy of I572937945adbb5adaa5cb09200752e323c2c9531)

We do some basic sanity checking in the block-device elements;
e.g. mbr is not suitable for aarch64, and efi is not suitable for
power.

This updates the bootloader to install EFI where appropriate

Co-Authored-By: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
Change-Id: Ib80acbfd9a12efd976c3fa15a5d1081eb0799305
2018-02-23 10:04:44 +11:00
Ian Wienand
adb0341064 Add block-device defaults
This moves the block-device default out of the "vm" element and into a
selection of other elements.  There's "mbr" which retains the status
quo.  There's an EFI version that has the boot/grub partitions as
required.  In between there's the GPT only version, which is useful
for architectures like power without EFI, but still want possible
larger disks using GPT.

Change-Id: I4a566a97d073fc0dda0ab2494ac988fe015800a9
2018-02-23 10:04:40 +11:00
Ian Wienand
55b479b54f GPT partitioning support
This adds support for a GPT label type to the partitioning code.  This
is relatively straight-forward translation of the partition config
into a sgparted command-line and subsequent call.

A unit test is added based on a working GPT/EFI configuration and the
fedora-minimal functional test is updated to build a single-partition
GPT based using the new block-device-gpt override element.  See notes
in the sample configuration files about partition requirements and
types.

Documentation has been updated.

Co-Authored-By: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
Change-Id: I6b819a8071389e7e4eb4874ff7750bd192695ff2
2018-02-23 10:04:26 +11:00
Matthew Thode
6b3c22dd75
update Gentoo Hardened profiles (now stable)
Change-Id: I6d89f27bfd62fc0e86fec1a0bc6cb37f9ec6c82f
Signed-off-by: Matthew Thode <mthode@mthode.org>
2018-02-21 11:06:16 -06:00
Lenny Verkhovsky
f249cec9f3 Checking link status according to DIB_DHCP_TIMEOUT
In slow networks like Infiniband it takes much time for the
interface to get the carrier. This patch enables this service
to run more then 20 seconds and limited by DIB_DHCP_TIMEOUT.

Change-Id: I8a6015567ac25e37b5a5aba4b1fda71170cc144a
2018-02-21 12:52:57 +00:00
Zuul
e8eb291f57 Merge "update gentoo vars for new profile and python" 2018-02-20 05:26:20 +00:00
Paul Belanger
e9e7ac2ee1
Install systemd earlier for Ubuntu Bionic
Like we did in https://review.openstack.org/475206 we need to install
systemd sooner because of the new world order of containers.

Change-Id: Ia60d751fee3af6f8d72ad664107acb337360feca
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-02-17 14:32:35 -05:00
Matthew Thode
bfee8bf094
update gentoo vars for new profile and python
Change-Id: I810b09f98c479e6ebdbf8de6fae31166a3e87667
2018-02-13 17:02:11 -06:00
Zuul
2080055155 Merge "Create rescue user on ironic agent" 2018-02-05 22:43:00 +00:00
Zuul
cb599b13de Merge "upgrade pip before using -c option" 2018-02-05 22:34:00 +00:00
Zuul
66444e8b93 Merge "Add support for Fedora 27, remove EOL Fedora 25" 2018-01-29 05:23:28 +00:00
Tim Flink
f8bcc51b55 Don't install dmidecode on Fedora ppc64le
While Debian-based distros use the label of ppc64el for ppc64 little
endian, Fedora uses ppc64le.

The ironic-agent was doing arch specific package install of lshw over
dmidecode for ppc64 and ppc64el but was attempting to install dmidecode
on Fedora ppc64le which caused the test to fail due to a missing
package.

This change just adds ppc64le to the arch-specific package installation
description for the ironic-agent element.

Change-Id: I38c3c1480bbbb2df817856614e6b740a0c02723a
Closes-Bug: 1744944
2018-01-29 10:53:43 +11:00
Zuul
9bfa45f0c8 Merge "Remove architecture rules on lshw dependency in ironic-agent" 2018-01-28 22:07:53 +00:00
Tim Flink
490cf3aa49 Add support for Fedora 27, remove EOL Fedora 25
This updates diskimage-builder to support current Fedora releases (26
and 27) and removes support for Fedora 25 which is EOL as of December
12, 2017.

Change-Id: I227a607c6c468cc8b7bb154a189e9c8ce2021192
2018-01-23 11:31:21 +00:00
Mark Hamzy
34ff72f253 upgrade pip before using -c option
The installed pip can be an older version which does not support
the -c argument. Therefore, upgrade pip before using -c.

Change-Id: If18d8ea822a62c8551c9c4d47354d58b0299fed2
Closes-Bug: 1744403
2018-01-19 16:46:59 -06:00
Dirk Mueller
4858340b42 Add SUSE Mapping
Change-Id: I8bc12435c62ef7c9c3fa8e21e00738698a532f56
2018-01-11 19:00:40 +01:00
Ian Wienand
e9df83b2b3 Revert "Dont install python-pip for py3k"
This reverts commit ab89c7d69c.

This commit checked for DIB_PYTHON_VERSION and only installed the v3
packages.  This is unfortunately backwards-incompatible, as consumers
such as the openstack gate are relying on this package installing pip
& virtualenv packages for python2 AND python3.

This was sort-of expressed in the docs, where it discusses what the
resulting setup of the system will be, but I've added a note to make
it clearer.

If we want to change this, I think we'll need either a new element, or
a non-defaulting flag.

Change-Id: I419dbdf4682394db68974944af1e5c432f3e0565
2018-01-10 15:48:03 +11:00
sayalilunkad
acc44c0651 Adding mapping for SUSE package
Mapping PyYAML to python-PyYAML as in opensuse.

Change-Id: I54864c96afdb20975bf1e3ac9a117d5263f21f3b
2017-12-20 17:51:33 +01:00
Ian Wienand
af50200863 Update Fedora defaults to 27
Update the Fedora defaults to now released F27

Change-Id: I7655ad010dfa668bd68b7a619d85e5f694f96806
2017-12-07 13:59:51 +11:00
Ian Wienand
dcbb7e7ebb ironic-agent: don't remove make
It turns out make has always been a tacit dependency of openssl as it
ships a Makefile for certificates [1].  This just recently changed to
be a hard dependency in F27, so this now fails as openssl is a
dependency of protected packages such as dnf.  Since it's always been
wrong to remove it, we take it out of the purge list.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=783446

Change-Id: I69efb3a56878ab97c4587bbbf5356bea752f2846
2017-12-07 13:59:51 +11:00
Michael Turek
7054a71f7d Remove architecture rules on lshw dependency in ironic-agent
There's a patch in flight in ironic-python-agent to switch the
default hardware manager to use lshw instead of dmidecode. [0]
This would require lshw to be installed regardless of
architecture. This patch removes the architecture rules from
lshw in the package-installs list.

[0] Ie370331df6bb5ef131c5cb60f458877e2a7ad71a

Change-Id: Idaf05b8efce28cd0cbf339cf693db4f55a693d9b
Partial-Bug: #1715790
2017-12-06 17:02:46 -05:00
K Jonathan Harker
7470ee26e0 zypper: fix package removal
zypper only supports the --no-recommends option during installs, giving
the option during removals results in an error.

When setting ACTION=remove, remove --no-recommends from EXTRA_ARGS, and
set --clean-deps to also remove no-longer-needed dependencies.

Rename EXTRA_ARGS to ACTION_ARGS for increased readability.

Change-Id: Ifbd168992b1a20658b6b4a99ba175234f6c78f6d
2017-12-05 22:59:20 +00:00
Zuul
7beb27ca30 Merge "Add zipl element as s390x architecture bootloader" 2017-12-01 02:35:58 +00:00
Zuul
6ab9655ca3 Merge "Fix /dev/pts mount options handling" 2017-12-01 02:01:46 +00:00
goldyfruit
c247cb41bb Fix wrong epel-release-7* package URL
When "epel" element is used during a build process
with "rhel7" distribution, the build failed
because the "epel-release-7*" package cannot be
installed.

The reason is because the URL is not correct, it
should be:
  URL=$BASE_URL/$RELEASE/x86_64/Packages/e/

Change-Id: I90c26892361f7611645b85f2eddc949b2f0d76fc
Closes-Bug: #1735547
2017-11-30 15:31:10 -05:00
Zuul
bfd61258ba Merge "Add the groundwork for musl profile support" 2017-11-30 09:31:40 +00:00
Zuul
5677a30a5a Merge "Pre-install curl" 2017-11-30 08:16:14 +00:00
Zuul
8182acb0fc Merge "Install fedora-gpg-keys for F27" 2017-11-30 07:54:38 +00:00
Matthew Thode
7223f2ce20 Add the groundwork for musl profile support
At the moment all musl needs in addition to an official stage4 file is a
few keywords and use flag changes.

Change-Id: Ibf4a6d616aca1aef876967e2aa34170c96ac9ef8
2017-11-30 18:17:21 +11:00
Zuul
6a28810ad8 Merge "Enable support for Gentoo overlays" 2017-11-30 06:40:43 +00:00
Zuul
87392cf1c8 Merge "Make preinstall.d more deterministic" 2017-11-30 05:50:03 +00:00
Matthew Thode
c4f83f2311 Enable support for Gentoo overlays
This is intended to eventually support building musl-libc based images,
which need the musl overlay.

Change-Id: I8f5429ffa64e74c860772d9a00ff0b7eebb7721a
2017-11-30 05:14:00 +00:00
Zuul
bbbe762dff Merge "elements: zypper-minimal: Refresh repositories where necessary" 2017-11-30 04:36:06 +00:00
Ian Wienand
1b203f8a38 Pre-install curl
As described, Fedora 27 has a curl-minimal package that comes in to
satisfy the rpm package dependency.  It conflicts with the "real" curl
package -- which is so commonly installed (by infra elements, etc)
that this becomes an annoying problem.  Just pre-install the full curl
package.

Fedora 24 is old enough to not worry about, so remove some old
workarounds to make the flow a little simpler.

Change-Id: I67baf96377109ac4521ba00243a0d91b35fafba0
2017-11-30 15:15:42 +11:00
Ian Wienand
bf8de79940 Install fedora-gpg-keys for F27
The repo GPG keys moved into a separate package [1] which now needs to
be installed.

Since the fedora-release/fedora-repos split is *long* since over,
remove that work-around and add this one.

[1] https://pagure.io/fedora-repos/c/f69f3729511c3eba5f470b1d90ea2bfee372eb29?branch=f27

Change-Id: I9ad28d5bdb78375ae21dbb16e2d8c4effb32cb35
2017-11-30 15:15:37 +11:00
Matthew Thode
c886c4cbe1 Make preinstall.d more deterministic
Reorders the preinstall, making the order more explicit.  Also dedupes
some folder setup.

Change-Id: I423dcba169558ff6037a3382b997675722e77405
2017-11-30 13:48:18 +11:00
Zuul
f74e48799d Merge "Enable gentoo in pip-and-virtualenv element" 2017-11-30 02:10:09 +00:00
Zuul
247c68b5a3 Merge "Clear /etc/machine-id to avoid duplicate machine-ids" 2017-11-30 01:56:26 +00:00
Zuul
d01d3d8832 Merge "Make python changes more reliable" 2017-11-30 01:49:07 +00:00
Zuul
71ca627d30 Merge "Add debian minimal requirement for arm64" 2017-11-29 23:47:12 +00:00
Zuul
c146d2f3b7 Merge "Fix grub2 dependency on arm64" 2017-11-29 23:47:10 +00:00
Andreas Florath
46a07de480 Fix /dev/pts mount options handling
The current implementation - as introduced in
Iee44703297a15b14c715f4bfb7bae67f613aceee - has some shortcomings / bugs,
like:

* the 'grep' check is too sloppy
* when /dev/pts is already mounted multiple times the current implementation
  fails:
  $ mount | grep devpts | sed 's/.*(\(.*\))/\1/'
  rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000
  rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000
  rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000
* code duplication
* Using the undocumented and non-robust output
  of 'mount'.

This patch fixed the above problems.

Change-Id: Ib0c7358772480c56d405659a6a32afd60c311686
Signed-off-by: Andreas Florath <andreas@florath.net>
2017-11-24 07:14:56 +00:00
Matthew Thode
ade82904a1
Make python changes more reliable
We oneshot emerge without calculating dependencies a few things to solve
for possible dependency loops.

Python 3.5 also became stable, so don't need to do special things for
it.

Matched the uninstall with the install lines (no need for a full if
statement).

Change-Id: I7c5e546612ac47d659e73a46a52e34d39ca81949
2017-11-20 23:19:46 -06:00
Zuul
b879a03c75 Merge "Dont install python-pip for py3k" 2017-11-20 05:38:24 +00:00
Markos Chandras
7f051add61 elements: zypper-minimal: Refresh repositories where necessary
We should always refresh the Tumbleweed repositories and the 'update'
one for Leap in order to always have the latest information from the
repositories.

Change-Id: I85db9d8bb7fa153f01222129e9b36fecc2632f57
2017-11-16 16:54:47 +00:00
Markos Chandras
da02f37de1 elements: Respect devpts mount options
This is a continuation for f2cc647dae ("diskimage_builder: lib:
common-functions: Fix options for devpts mount"). We also need to
respect the devpts mount options when the dib elements are mounting
this virtual filesystems themselves.

Change-Id: Iee44703297a15b14c715f4bfb7bae67f613aceee
2017-11-14 08:31:55 +00:00
Gregory Haynes
ab89c7d69c Dont install python-pip for py3k
We want to install python3-pip, not python-pip when we are building a
py3k image less we pull in python2. Once we stop installing python2 we
have to stop calling python2 during pip install.

Change-Id: I7d8ba9300039cce90965410a4e16ca9e711904c3
2017-11-13 23:00:52 +00:00
Adam Harwell
d4fd7f1217 Enable gentoo in pip-and-virtualenv element
Currently it will hit the `else` and try to apt-get, which fails.

Change-Id: I951882cf3897ced165e167f12877c05ee62a5054
2017-11-11 09:00:50 +09:00
Dirk Mueller
4e258bdad6 zypper-minimal: Set default locale env to C.UTF-8
Currently in Leap 42.x the bootup scripts don't actually make use of
locale.conf yet, so we need to set it in /etc/sysconfig/language. For
future distro compatibility the setting in locale.conf is kept in sync.

Also fix default timezone link.

Change-Id: I59e5dccad8a5ae132d3039851e7aa1db86a609d7
2017-11-10 16:09:05 +01:00
Zuul
bc6c928bb9 Merge "Move to a common lock-file directory" 2017-11-07 17:32:28 +00:00
Zhiguo Deng
271dc36f33 Add zipl element as s390x architecture bootloader
s390x architecture uses zipl as bootloader. When used in combination
with the vm element it replaces the existing bootloader element.
It's mandatory for s390x vm images.

Use cases
---------

* Allow users to create s390x images that run on nova with s390x
  libvirt/kvm backend
* Building nodepool images for s390x third party CI

Supported Distros
-----------------
The following listing shows all Distros that officially support
s390x and how those Distros are supported in DIB with this patch.

* SLES - not supported (SLES is not supported in DIB)
* RHEL - not suppoprted (RHEL is not supported as KVM guest on s390x,
                         therefore there's no rhel7 qcow image for s390x available
                         like it is for other archictectures)
* Ubuntu - supported

Ubuntu images can for example be built using the following commands:

  $ disk-image-create ubuntu-minimal zipl vm
  $ disk-image-create ubuntu-minimal zipl
  $ disk-image-create ubuntu zipl vm

Testing
-------

Cross architecture building of s390x images is not supported so far.

The plan is to set up a ThirdParty CI that builds the image for s390x and
provides the logs.

Co-Authored-By: Andreas Scheuring <andreas.scheuring@de.ibm.com>
Co-Authored-By: Holger Smolinsky <holger@smolinski.name>
Co-Authored-By: Zhiguo Deng <bjzgdeng@linux.vnet.ibm.com>
Co-Authored-By: Arne Recknagel <arne.recknagel@hotmail.com>

Closes-Bug: #1730641

Change-Id: I576e7edda68da12e97c60af38f457915efe7b934
2017-11-07 17:19:27 +01:00
Zuul
906a3f4a57 Merge "Use -t devpts for /dev/pts mounts" 2017-11-02 14:20:05 +00:00
Zuul
1b0631da84 Merge "Update proliant-tools to support Gen10 Proliant servers" 2017-11-01 07:22:03 +00:00
Ian Wienand
b25d0337b8 Move to a common lock-file directory
In a couple of places we use flock for critical sections, but we leave
lockfiles around in various locations which can be confusing.

Introduce DIB_LOCKFILES global (under ~/.cache/dib/lockfiles) and
write lockfiles in there.

Fix up removal of the lockfile in the yum path; we just want to make
sure we cleanup the .rpmmacros file, but we don't need to remove the
lockfile as well.

Co-Authored-By: Andreas Florath <andreas@florath.net>

Change-Id: Ie810b2836be521325afe923708d046112e1e1e20
2017-10-26 16:27:59 +11:00
Zuul
8f025691ba Merge "Dont install python-virtualenv for py3k in deb" 2017-10-24 06:33:45 +00:00
Zuul
c5f713b1ec Merge "Change to install a package in 'proliant-tools'" 2017-10-24 05:48:25 +00:00
Yolanda Robla
ba11376328 Create rescue user on ironic agent
Create a new service, that will be launched after ironic
agent has been exited. This will launch an script that will
take the rescue password, and create the rescue user with
that credentials.

Depends-On: I7898ff22800dedba73d7fbfb3801378867abe183
Change-Id: Ic3a241e2789a122d3d966e7e2148306fd0cf6aed
Partial-Bug: 1526449
2017-10-23 12:50:32 +00:00
Andreas Florath
cebfcf85f9 Use -t devpts for /dev/pts mounts
Currently a bind is used when mounting /dev/pts in chroot.
This leads to problems - especially when running DIB in parallel:
It was observed that the /dev/pts mount vanishes from the host
system.

This patch uses '-t devpts' - as it is done for /sys and /proc -
for handling /dev/pts.

Change-Id: Id7775ae6fca6502af800e7b73a00862ef320206b
Signed-off-by: Andreas Florath <andreas@florath.net>
2017-10-23 07:13:02 +00:00
Gregory Haynes
00d7c619e9 Dont install python-virtualenv for py3k in deb
On ubuntu we detect that in python3 we need to install
python3-virtualenv, but append this to the packages to install rather
than replace python-virtualenv which results in both being installed
(and therefore grabbing python2).

Change-Id: I422490ebe9a9c655552685bc2ff342d288335a9c
Closes-Bug: #1724656
2017-10-18 23:11:55 +00:00
Zhangfei Gao
6cc155fd66 Add debian minimal requirement for arm64
Debian system building fails, fixed by adding arm64 package.

DIB_RELEASE=jessie disk-image-create debian ironic-agent \
    grub2 devuser -a arm64 -o deploy-jessie
dib-run-parts Running /tmp/dib_build.v5FEtaKx/hooks/cleanup.d/99-extract-kernel-and-ramdisk
ls: cannot access /tmp/dib_build.v5FEtaKx/mnt/boot/vmlinu*: No such file or directory

Change-Id: I610d767785df49fed954f12854be5ae78ff9baa6
Signed-off-by: Zhangfei Gao <zhangfei.gao@linaro.org>
2017-10-16 13:39:50 +08:00
Zhangfei Gao
a8635d3a3b Fix grub2 dependency on arm64
Unable to locate package reported when
DIB_RELEASE=jessie disk-image-create debian ironic-agent \
grub2 devuser -a arm64 -o deploy-jessie

E: Unable to locate package grub-pc-bin
E: Unable to locate package shim-signed
E: Unable to locate package grub-efi-amd64-signed

Fix the issue via adding arch dependency and arm64 packages

Change-Id: I40650a887b575a9c2b00a8c5036c35354d548673
Signed-off-by: Zhangfei Gao <zhangfei.gao@linaro.org>
2017-10-16 13:39:34 +08:00
Matthew Thode
e29f031bec
Update Gentoo element for element changes
There have been a few changes over the past few months, here we make the
following changes.

* change from backtrack=99 to complete-graph as a more correct flag
* make python version selection more in line with what gentoo supports
* set up python before stuff gets pip installed
* ensure we have the proper pip so we can install pip packages as root
* ensure we have the proper use flags for the disk formatting changes
* set DIB_RELEASE like other distros
* fix openssh-server element for gentoo

Change-Id: I17202de3016616ce34c8cbead7d0fb047a64e96b
2017-10-08 12:02:46 -05:00
Anshul Jain
afb7084a4d Update proliant-tools to support Gen10 Proliant servers
This commits make update to ssacli version to point to latest
ssacli release that has support for HPE P/E-Class SR Gen10 controllers.

Change-Id: Ia9a0eaec78d601f56b4036e57601554b87f21acc
Closes-Bug: 1721185
2017-10-04 07:02:07 +00:00
Ian Wienand
df00e9adcb Add initramfs-tools for ubuntu-minimal
A small update was made to 4.4.0-96.119 that dropped the
initramfs-tools dependency from the kernel [1].  This had the
unfortunate affect of removing the initramfs from ubuntu-minimal and
making it unbootable, since we specify the root device via LABEL=.
Add the package explicitly alongside the kernel.

Also, small fix to pass unit tests

[1] https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1700972

Change-Id: I57a0f08cd5e082ecdf8dba0ab34fb3062c50836d
2017-09-21 10:42:11 +02:00
Aparna
7bc2b23290 Change to install a package in 'proliant-tools'
This commit adds change in 'proliant-tools' element to
install a package 'unzip' which is required to perform
SUM based firmware update for HPE Proliant servers.

Change-Id: Ib8f6d18402439edd93d100cc7a4fb2094c863715
2017-09-18 08:04:34 +00:00
Jenkins
22e03f9820 Merge "Add missing package dependency for yaml" 2017-09-15 13:52:35 +00:00
Jenkins
254875dbde Merge "Create /etc/machine-id for fedora" 2017-09-15 04:05:51 +00:00
Jenkins
6e266399fb Merge "Use latest Fedora .qcow URL" 2017-09-15 02:36:24 +00:00
Ian Wienand
768c5e188c Create /etc/machine-id for fedora
As described in the comment, we need to create the /etc/machine-id for
the image-based build when systemd isn't updated (as is usually the
case for a new distro)

Work on clearing this out continues, but this brings it to parity with
fedora-minimal.

Change-Id: Icbbbabb4114d4d95909648d8e39a6bae6d2a7b7b
Depends-On: I761e425f8a658669d9b8a70ce4260cec263ea51a
2017-09-15 11:54:01 +10:00
Ian Wienand
7774260b76 Use latest Fedora .qcow URL
The URL we are using seems to have disappeared.  Update this to
download.fedoraproject.org.  The new URL requires a "subrelease" now,
add it, along with a note on where it comes from.

Change-Id: I761e425f8a658669d9b8a70ce4260cec263ea51a
2017-09-15 11:06:22 +10:00
Yolanda Robla
da5c926fe9 Add missing package dependency for yaml
This element was assuming that yaml was included as package,
but there are systems not including it. So properly add yaml
as a dependency.

Change-Id: I72da2776674a3963657052b9a9715abcb4fab1e2
Partially-Fixes-Bug: #1715686
2017-09-13 14:16:41 +02:00
Yolanda Robla
3ff8d1e10b Move the ordering of the dracut regenerate command
When using combined with rhel7 image, the unregister of repos
has already happened, because it is executed under 60- ordering.
As dracut-regenerate may need to install extra packages for it,
it causes this step to fail, because it cannot find repos where
to pull the packages from.

Change-Id: I35e37df7990ad76a5004cb90fdd863ec743a5483
2017-09-06 12:52:05 +02:00
Ben Nemec
72d0d22cdf Remove nested quotes from TAROPTS
Per the bug report, these seem to be causing issues with maintaining
file capabilities.  They aren't necessary so let's just remove them.

Change-Id: I06c90fdc85655986142b936cadbe04d75dd27427
Closes-Bug: 1714604
2017-09-01 17:29:15 -05:00
Jenkins
39d84d2059 Merge "Use [[ for =~ matches" 2017-08-29 05:59:42 +00:00
Ian Wienand
c448864901 Use [[ for =~ matches
Avoid incorrect use of [ with =~ matching

I guess this doesn't trip "-e" because it's in an if-conditional.  I'm
looking at making bashate detect this; maybe we can run bashate over
things we know are scripts

Change-Id: Ia3fe2b978fae5bdaadbb1789058180d3ad950d00
2017-08-28 17:01:03 +10:00
Julia Kreger
6d64a2aee6 Fix cylical systemd config for dhcp-all-interfaces
In Ubuntu/Debian, the default dependencies cannot be relied
upon as we enter into a cyclical dependency relationship which
prevents the unit from starting.

Added the required configuration to the systemd unit file.

This issue has also been observed in glean[0], which has a nearly
identical unit file for interface start-up.

[0]: https://review.openstack.org/#/c/485748
Closes-Bug: #1708685

Change-Id: I23ac9510d1a21c7073bd33f76ba66fa04a8be035
2017-08-25 15:51:23 +10:00
Jenkins
ea23aa13a5 Merge "Add netbase to ensure /etc/protocols is placed for debian" 2017-08-15 00:37:27 +00:00