Commit Graph

638 Commits

Author SHA1 Message Date
Zuul
c46f8714e3 Merge "Remove the rhel 8 check for xfs" 2019-06-18 06:49:34 +00:00
Zuul
fe618002ff Merge "Update test coverage for openSUSE/-minimal to 15.1" 2019-06-18 06:21:21 +00:00
Dirk Mueller
a81cf9e231 Update test coverage for openSUSE/-minimal to 15.1
Use openSUSE 15.1 as default, which is the latest released stable
openSUSE release.

Remove leftovers for unmaintained openSUSE 42.2 images.

Depends-On: https://review.opendev.org/#/c/660126/
Change-Id: I0b204b7b3d7ae74b6749320b3bfe1ca89d154ebb
2019-06-13 09:20:40 +02:00
Michael Johnson
5f4b764a11 Remove the rhel 8 check for xfs
This patch removes the check and default for rhel 8 requiring
xfs filesystem as rhel 8 images can successfully be built with
ext4 filesystems.

Change-Id: I1a6bfa26324fd43ae0c77c2c977dda0dd56e26e5
2019-06-12 07:01:36 -07:00
Dmitry Tantsur
928c6e61f0 ironic-agent: install mdadm on the ramdisk
The newly introduced software RAID support requires it.

Change-Id: Ic438865006f1472abc0c9f4d40cc40c91b4ada71
2019-06-07 14:05:41 +02:00
Zuul
7f469e3e83 Merge "Increase size of EFI system partition (again)" 2019-05-31 09:05:17 +00:00
Zuul
d323928af0 Merge "Makes image caching more resilient" 2019-05-31 08:54:31 +00:00
Zuul
d6f43865ed Merge "fail early when lates build information can not be fetched" 2019-05-31 08:45:50 +00:00
Zuul
21676bd350 Merge "Add option to skip update packages" 2019-05-31 08:13:38 +00:00
Zuul
49930da885 Merge "Deprecate rhel7 in favor of rhel" 2019-05-31 08:01:13 +00:00
Zuul
4367dd2dd3 Merge "Add version-less RHEL element for RHEL7 and RHEL8" 2019-05-31 07:56:50 +00:00
Pierre Riteau
7fd52ba841 Increase size of EFI system partition (again)
When I said in I8594d1fe05242f246a5809740a115ab2f84ac5a3 that 12 MiB
ought to be enough, I should have expected that I would be proven wrong.
While 12 MiB is enough to fit shim-x64 and grub2-efi-x64, yum fails to
update these packages to newer versions:

Transaction check error:
  installing package shim-x64-15-2.el7.centos.x86_64 needs 7MB on the /boot/efi filesystem
  installing package grub2-efi-x64-1:2.02-0.76.el7.centos.1.x86_64 needs 3MB on the /boot/efi filesystem

Error Summary
-------------
Disk Requirements:
  At least 7MB more space needed on the /boot/efi filesystem.

It is recommended that the ESP partition be much bigger. This commit
bumps its size to 550MiB, following guidelines from Rod Smith to avoid
incompatibilities with some EFIs [1].

[1] https://www.rodsbooks.com/efi-bootloaders/principles.html

Change-Id: If9515234f1a803cda32b2482f8abe10ddf0e6d26
2019-05-31 17:10:08 +10:00
Sorin Sbarnea
fb656718fb Makes image caching more resilient
Avoids failing on the first attempt to download the image to cache as
mirrors hosting them can randomly go down, usually with a connection
refused.

Change-Id: I9de9f33c2cc16596d04b35c4eb92621e6a2c7511
2019-05-31 16:31:43 +10:00
Dirk Mueller
421a0fa541 fail early when lates build information can not be fetched
When the mirror returns a error, it was trying to interpret the error
message (e.g. <html><title>Internal server error..) as a download link.
By using -f on curl we get an empty reply and an exit code, which, as
we run in set -e mode, aborts.

Change-Id: Ibaa39aedb7db286f859c4b090114c6a233b150c7
2019-05-31 16:09:25 +10:00
Zuul
ed6dfd87e5 Merge "allow the use of non-bzip compressed stages for building gentoo" 2019-05-31 04:42:07 +00:00
Nir Magnezi
433a374748 Deprecate rhel7 in favor of rhel
The rhel7 element is deprecated and is left only for backward
compatibility.
The rhel element should be used instead. Users should set DIB_RELEASE to
'7' to indicate which release you are using.

The new element is a version-less RHEL element to handle both '7'
and '8' DIB_RELEASE, which aligns with other elements which operate in
the same way such as the Fedora element.

Change-Id: Ic39ed85cacae9942448eb18ad685763f9369c2ed
2019-05-29 12:07:44 +00:00
Nir Magnezi
ee46e2f9b7 Add version-less RHEL element for RHEL7 and RHEL8
Make a version-less RHEL element to handle both '7' and '8' DIB_RELEASE.
The element usage should align with other elements which operate in the
same way such as the Fedora element.

Additionally, this patch adds support for RHEL8 that operates with
Python 3.
As of now, users of diskimage-builder will still be able to use the
'rhel7' element, or migrate to 'rhel' and specify their respective
DIB_RELEASE value.

* mount the xfs file-system for extraction as read-only.  vaguely
  based on explaination in [1] and the fact we only read the image
  data into a tar, so can ignore this.

    XFS (dm-1): Superblock has unknown read-only compatible features (0x4) enabled.

* Use the redhat system python as the dib-python version.  dib was
  ahead of it's time making an abstracted python interpreter for
  system work ;) the system python should work for running the various
  dib element scripts.

[1] https://unix.stackexchange.com/questions/247550/unmountable-xfs-filesystem

Redhat-Bug: https://bugzilla.redhat.com/show_bug.cgi?id=1700253
Co-Authored-By: Ian Wienand <iwienand@redhat.com>
Change-Id: I90540675c70bb475d9db2ae24f81c648a31f3f95
2019-05-29 11:28:53 +03:00
Matthew Thode
afcac1922c
allow the use of non-bzip compressed stages for building gentoo
Upstream is switching to xz so we need to be able to support it.

Change-Id: I382cc3e8038e2e552c553c526a990a01e51aeb12
2019-05-24 09:32:57 -05:00
Zuul
c47a9d5001 Merge "Replace git.openstack.org URLs with opendev.org URLs" 2019-05-24 08:03:47 +00:00
melissaml
a6322c6ed0 Replace git.openstack.org URLs with opendev.org URLs
Change-Id: I03e9162d5a59a2aa1631a9ecf6f6833bb7ac6050
2019-05-16 14:45:52 +08:00
Zuul
3d3ba26edd Merge "Use megabyte granularity for image extra space" 2019-05-15 06:51:10 +00:00
Logan V
87a18f51e3 Use megabyte granularity for image extra space
I want to use the new --image-extra-size flag[1] but my use-case
calls for megabyte granularity of this value. Rather than adding
60% to an 800MB image, maybe I only want to add 100 or 200MB, etc.

[1] https://review.opendev.org/#/c/655127/

Change-Id: I8fb9685d60ebb1260d5efcf03c5c23c561c24384
2019-05-15 13:38:25 +10:00
Dirk Mueller
c7ac6ee0cb Update test coverage for openSUSE/-minimal to 15.0
Use openSUSE 15.0 as default, which is the latest released stable
openSUSE release. Switch to https for accessing download.o.org
as encrypted transfers should be used by default.

Remove leftovers for definitely unmaintained openSUSE 13.x images
and split into old/new leap style versioning scheme for clarity.

Change-Id: Iab129eeee2b1a2563f0f0d2cb17bbad57c068e38
2019-05-08 14:59:51 +00:00
Zuul
4665e79245 Merge "openssh-server: harden sshd config" 2019-05-07 19:35:42 +00:00
Paul Belanger
5d60979e93 Use fedora-release-common for fedora 30+
It looks like fedora-release on fedora 30+ has been split into sub
packages. Use fedora-release-common to avoid package conflicts.

Change-Id: I8f8711044fc4074b91939e0a6dfdac4d7a14a35b
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2019-05-07 12:08:50 +00:00
Zuul
fa34eb7fe4 Merge "Support defining the free space in the image" 2019-05-07 10:14:01 +00:00
Zuul
8bf37a064e Merge "Allow specification of filesystem journal size" 2019-05-07 10:14:00 +00:00
Zuul
8c8b856c27 Merge "Only enable dbus-daemon for fedora-29 and below" 2019-05-07 05:47:57 +00:00
Paul Belanger
38d7574127 Only enable dbus-daemon for fedora-29 and below
In fedora-30 is when we migrate to dbus-broker, fedora-29 is still using
dbus-daemon.

Change-Id: I1e1d3a3826157b8b22386c211eaa58b6439b5f3c
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2019-05-06 17:49:42 +10:00
Paul Belanger
daf5a4e4bd Switch simple-init to support python3
Depending on the version of $DIB_PYTHON_VERSION, we can either use pip /
pip3 to install glean.  This is helpful for newer OSes that might not
want to ship python2 (pip).

Change-Id: I25c5927a1eb55ee16b919dd64403184f335839b6
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2019-05-02 19:38:16 -04:00
Tristan Cacqueray
11ec95b779 openssh-server: harden sshd config
Harden sshd configuration by adding KexAlgorithms, Ciphers and MACs for sshd,
following good pratices on https://infosec.mozilla.org/guidelines/openssh

Change-Id: I3051320d867a5033e82deef10c5e723ca9829884
Co-Authored-By: Nicolas Hicher <nhicher@redhat.com>
2019-05-01 11:42:21 -04:00
Tobias Henkel
778d007150 Support defining the free space in the image
Currently diskimage-builder supports two ways to specify the image
size. One is defining a fixed image size using DIB_IMAGE_SIZE, the
other one is auto-detection while adding a security margin of 60% as
free space. This means when building larger images (e.g. >100GB) with
unknown size upfront we end up with much wasted space, IO and network
traffic when uploading the images to several cloud providers. This can
be optimized by adding a third way by defining DIB_IMAGE_EXTRA_SIZE to
specify the free space in GB. This makes it possible to easily build
images of varying sizes while still minimizing the overhead by keeping
the free space constant to e.g. 1GB.

Change-Id: I114c739d11d0cfe3b8d8abc6df5ff989edfb67f2
2019-04-29 20:18:43 +10:00
caoyuan
0329a6de5e Replace git.openstack.org URLs with opendev.org URLs
Change-Id: Iac5a9da62db84365a769ea07146281866215a9c5
2019-04-29 20:15:25 +10:00
Logan V
11142f75b4 Allow specification of filesystem journal size
In many cases, the statically sized 64MB journal is far below the
e2fstools default calculation[0] which calls for a 64MB journal only
on filesystems smaller than 16GB. On bare metal in particular, the
correct default journal size will often be in the 512MB-1GB range.

Since we cannot know what the target system is, this should be a
tunable parameter that the user can set depending on the intended
image usage.

Add a DIB_JOURNAL_SIZE envvar and --mkfs-journal-size parameter
to the image creation so users can override the default journal
size.

[0] https://git.kernel.org/pub/scm/fs/ext2/e2fsprogs.git/tree/lib/ext2fs/mkjournal.c#n333

Change-Id: I65fa13a088eecdfe61636678578577ea2cfb3c0c
2019-04-29 17:00:30 +10:00
Zuul
6a4bf78e0c Merge "Fix Fedora aarch64 image location" 2019-04-18 12:22:06 +00:00
Zuul
5b47dc3a5b Merge "debian-minimal buster support" 2019-04-18 08:32:47 +00:00
Zuul
2205741de6 Merge "Also use selinuxenabled to check selinux status" 2019-04-18 08:32:15 +00:00
Pedro Alvarez
f034dd00d9 Fix Fedora aarch64 image location
It used to be considered a 'secondary architecture' but that
is not the case anymore.

Change-Id: I8e5e9cfa915c8a3c979ff9db26477c0542d271db
2019-04-09 15:50:55 +00:00
Ian Wienand
105d201e1f debian-minimal buster support
Due to the referenced bug, many versions of debootstrap can't bring up
a buster environment.  Unfortunately, these include versions we use to
do this on Xenial/Bionic nodes.

Also, there isn't backports or security updates, so elide these for
now.

I did get a working build (I haven't gone so far as a full boot+glean)
with this, at least.

Change-Id: If2420e92cb728ab6e91b0d70547da4483679b391
Paritial-Bug: #1822927
2019-04-04 16:10:08 +11:00
Serena Ziviani
19cc00041a Also use selinuxenabled to check selinux status
Currently, the cleanup script is using the existence of the folder
/sys/fs/selinux to check if SELinux is enabled. This, however, is
misleading in case disk-image-builder is used inside a Docker
container on a selinux-enabled host. In this case, the folder exists
in the container but SELinux is disabled.

This patch addresses the problem by checking, in addition to the
check already in place, the output of the command selinuxenabled.

Change-Id: I83e58f2467e60df9f0f00f7b7a58d0e2ce357a9a
Closes-Bug: #1820077
2019-03-28 14:20:24 +01:00
Zuul
36b4bc87f9 Merge "Minor clarifications in centos7 element docs" 2019-03-28 03:50:52 +00:00
Ian Wienand
20c5c98426 Replace openstack.org git:// URLs with https://
This is a mechanically generated change to replace openstack.org
git:// URLs with https:// equivalents.

This is in aid of a planned future move of the git hosting
infrastructure to a self-hosted instance of gitea (https://gitea.io),
which does not support the git wire protocol at this stage.

This update should result in no functional change.

For more information see the thread at

 http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003825.html

Change-Id: Id26bec14c3d94e2f81b2148fc85d17f07866398c
2019-03-22 01:35:42 +00:00
Daniel Abad
965d6f97aa Minor clarifications in centos7 element docs
Change-Id: I6aef77513efa37262269ca24b296acbdc823a039
2019-03-20 17:23:34 +01:00
Ian Wienand
5284564071 Unmount internal mounts on finalise errors
This is only one line, but it takes a lot to untangle ...  basically
the current "correct" path is:

---
 mk_build_dir()
  -> sets trap trap_cleanup EXIT

 ... stuff ..

 mount_proc_dev_sys
  -> mounts $TMP_MOUNT_PATH/<proc,dev.sysfs>

 pre-finalise.d
 finalise.d

 unmount_image $TMP_BUILD_DIR/mnt # nb == $TMP_MOUNT_PATH
  -> unmount_dir()
   -> recursive unmount everything inside TMP_MOUNT_PATH

 TMP_IMAGE_PATH=$(dib-block-device getval image-path)
 export TMP_IMAGE_PATH

 dib-block-device umount
 dib-block-device cleanup

 ... actually cleanup directories ...
---

Our current failure exit trap does:

---
 dib-block-device umount
 unmount_image
 ...
---

Note this is the *opposite* of what is done in the correct exit path.
In the failure case, if a script fails in the finalise stages it leads
to /proc, /sys, /dev etc. still being mounted inside the image; the
"dib-block-device umount" call doesn't know anything about these
mounts and tries to unmount the parent directory, and we get a hard
failure with a busy mount, and all the mounts are subsequently leaked.

Note that "unmount_dir", which is ultimately called by
"unmount_image", already knows to skip those mounts that
"dib-block-device umount" manages (this is the DIB_MOUNTPOINTS list).
This is further evidence it should be called *before* the
dib-block-device umount.

Change-Id: Ibef3ce9d1167b9c4ff3d5717b113cd3ed374f5e3
2019-03-13 16:38:49 +11:00
Zuul
bdfc13a5c0 Merge "[lvm] Add Ubuntu bionic as supported distro" 2019-03-11 09:19:49 +00:00
Zuul
186db05ffb Merge "Add DIB_APT_MINIMAL_CREATE_INTERFACES toggle" 2019-03-05 07:24:24 +00:00
Logan V
8756cbea1b Add DIB_APT_MINIMAL_CREATE_INTERFACES toggle
Add a DIB_APT_MINIMAL_CREATE_INTERFACES boolean to the debootstrap
element which functions identically to
DIB_YUM_MINIMAL_CREATE_INTERFACES in the yum-minimal element.

This can be used to disable the creation of the
/etc/network/interfaces.d/eth[01] dhcp configuration files, which
are not needed on systems where cloud-init or other means are used
to configure networking.

The flag is enabled by default to keep creating the dhcp interface
files, maintaining backwards compatibility.

Change-Id: I1fdaca8350a5ceefd9e437af4fd000ce6a3ee7f3
2019-03-05 16:27:57 +11:00
Gaëtan Trellu
bbde9bb320 [lvm] Add Ubuntu bionic as supported distro
The way how LVM is created on Ubuntu Xenial and Ubuntu Bionic
is the same.

Change-Id: I16d548f6393dd3cdfd5a9befa5c0ef0f6db92df1
2019-03-04 15:05:37 -05:00
Zuul
0f8d340c6c Merge "Update gentoo-releng gpg key" 2019-02-28 00:33:06 +00:00
Matthew Thode
b2cc91d276
Update gentoo-releng gpg key
A new signing subkey is used.

Change-Id: Idc9aceba7ee144fd0307737c24991acfacf68985
2019-02-27 15:18:25 -06:00