Commit Graph

3773 Commits

Author SHA1 Message Date
Logan V
11142f75b4 Allow specification of filesystem journal size
In many cases, the statically sized 64MB journal is far below the
e2fstools default calculation[0] which calls for a 64MB journal only
on filesystems smaller than 16GB. On bare metal in particular, the
correct default journal size will often be in the 512MB-1GB range.

Since we cannot know what the target system is, this should be a
tunable parameter that the user can set depending on the intended
image usage.

Add a DIB_JOURNAL_SIZE envvar and --mkfs-journal-size parameter
to the image creation so users can override the default journal
size.

[0] https://git.kernel.org/pub/scm/fs/ext2/e2fsprogs.git/tree/lib/ext2fs/mkjournal.c#n333

Change-Id: I65fa13a088eecdfe61636678578577ea2cfb3c0c
2019-04-29 17:00:30 +10:00
Ian Wienand
d18e73147e Document the various global filesystem options
These options have always been a bit of a mess since v2 when we added
all the block-device flexibility.  Add some explicit documentation to
try and help explain the relationship between these options and the
block-device config.

Change-Id: I49affcbef868d644f673b833bef8310cf25cfd0f
2019-04-29 17:00:04 +10:00
Dirk Mueller
08b9f7ebb8 Update to https:// version of *openstack.org urls
The non-encrypted urls are just redirecting to https, so we
can save one roundtrip by linking the encrypted versions directly.

Change-Id: I88fe8b8c8ccfb5471f59f7898a69bf62cb6cfcaf
2019-04-28 03:03:52 +02:00
OpenDev Sysadmins
0be0a827f7 OpenDev Migration Patch
This commit was bulk generated and pushed by the OpenDev sysadmins
as a part of the Git hosting and code review systems migration
detailed in these mailing list posts:

http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003603.html
http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004920.html

Attempts have been made to correct repository namespaces and
hostnames based on simple pattern matching, but it's possible some
were updated incorrectly or missed entirely. Please reach out to us
via the contact information listed at https://opendev.org/ with any
questions you may have.
2019-04-19 19:26:30 +00:00
Zuul
dee382209a Merge "Constraint networkx to <2.3 for Python 2" 2019-04-18 16:33:30 +00:00
Zuul
6a4bf78e0c Merge "Fix Fedora aarch64 image location" 2019-04-18 12:22:06 +00:00
Zuul
5b47dc3a5b Merge "debian-minimal buster support" 2019-04-18 08:32:47 +00:00
Zuul
2205741de6 Merge "Also use selinuxenabled to check selinux status" 2019-04-18 08:32:15 +00:00
Fatih Degirmenci
8d7e9b3805 Constraint networkx to <2.3 for Python 2
NetworkX released 2.3, dropping support for Python 2.

https://github.com/networkx/networkx/blob/master/doc/news.rst

Depends-On: https://review.openstack.org/#/c/652094

Change-Id: I59afe81d7868605434d2a42658c5b09a6a546732
2019-04-13 22:11:58 +02:00
Pedro Alvarez
f034dd00d9 Fix Fedora aarch64 image location
It used to be considered a 'secondary architecture' but that
is not the case anymore.

Change-Id: I8e5e9cfa915c8a3c979ff9db26477c0542d271db
2019-04-09 15:50:55 +00:00
Ian Wienand
105d201e1f debian-minimal buster support
Due to the referenced bug, many versions of debootstrap can't bring up
a buster environment.  Unfortunately, these include versions we use to
do this on Xenial/Bionic nodes.

Also, there isn't backports or security updates, so elide these for
now.

I did get a working build (I haven't gone so far as a full boot+glean)
with this, at least.

Change-Id: If2420e92cb728ab6e91b0d70547da4483679b391
Paritial-Bug: #1822927
2019-04-04 16:10:08 +11:00
Serena Ziviani
19cc00041a Also use selinuxenabled to check selinux status
Currently, the cleanup script is using the existence of the folder
/sys/fs/selinux to check if SELinux is enabled. This, however, is
misleading in case disk-image-builder is used inside a Docker
container on a selinux-enabled host. In this case, the folder exists
in the container but SELinux is disabled.

This patch addresses the problem by checking, in addition to the
check already in place, the output of the command selinuxenabled.

Change-Id: I83e58f2467e60df9f0f00f7b7a58d0e2ce357a9a
Closes-Bug: #1820077
2019-03-28 14:20:24 +01:00
Zuul
36b4bc87f9 Merge "Minor clarifications in centos7 element docs" 2019-03-28 03:50:52 +00:00
Zuul
c644508610 Merge "Replace openstack.org git:// URLs with https://" 2019-03-25 00:18:35 +00:00
Ian Wienand
62c02d16b5 Replace openstack.org git:// URLs with https://
This is a mechanically generated change to replace openstack.org
git:// URLs with https:// equivalents.

This is in aid of a planned future move of the git hosting
infrastructure to a self-hosted instance of gitea (https://gitea.io),
which does not support the git wire protocol at this stage.

This update should result in no functional change.

For more information see the thread at

 http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003825.html

Change-Id: I40e1a77a1f5c9b5a17aab849fc74a11b5a36854c
2019-03-24 20:33:30 +00:00
Ian Wienand
20c5c98426 Replace openstack.org git:// URLs with https://
This is a mechanically generated change to replace openstack.org
git:// URLs with https:// equivalents.

This is in aid of a planned future move of the git hosting
infrastructure to a self-hosted instance of gitea (https://gitea.io),
which does not support the git wire protocol at this stage.

This update should result in no functional change.

For more information see the thread at

 http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003825.html

Change-Id: Id26bec14c3d94e2f81b2148fc85d17f07866398c
2019-03-22 01:35:42 +00:00
Daniel Abad
965d6f97aa Minor clarifications in centos7 element docs
Change-Id: I6aef77513efa37262269ca24b296acbdc823a039
2019-03-20 17:23:34 +01:00
Ian Wienand
5284564071 Unmount internal mounts on finalise errors
This is only one line, but it takes a lot to untangle ...  basically
the current "correct" path is:

---
 mk_build_dir()
  -> sets trap trap_cleanup EXIT

 ... stuff ..

 mount_proc_dev_sys
  -> mounts $TMP_MOUNT_PATH/<proc,dev.sysfs>

 pre-finalise.d
 finalise.d

 unmount_image $TMP_BUILD_DIR/mnt # nb == $TMP_MOUNT_PATH
  -> unmount_dir()
   -> recursive unmount everything inside TMP_MOUNT_PATH

 TMP_IMAGE_PATH=$(dib-block-device getval image-path)
 export TMP_IMAGE_PATH

 dib-block-device umount
 dib-block-device cleanup

 ... actually cleanup directories ...
---

Our current failure exit trap does:

---
 dib-block-device umount
 unmount_image
 ...
---

Note this is the *opposite* of what is done in the correct exit path.
In the failure case, if a script fails in the finalise stages it leads
to /proc, /sys, /dev etc. still being mounted inside the image; the
"dib-block-device umount" call doesn't know anything about these
mounts and tries to unmount the parent directory, and we get a hard
failure with a busy mount, and all the mounts are subsequently leaked.

Note that "unmount_dir", which is ultimately called by
"unmount_image", already knows to skip those mounts that
"dib-block-device umount" manages (this is the DIB_MOUNTPOINTS list).
This is further evidence it should be called *before* the
dib-block-device umount.

Change-Id: Ibef3ce9d1167b9c4ff3d5717b113cd3ed374f5e3
2019-03-13 16:38:49 +11:00
Zuul
bdfc13a5c0 Merge "[lvm] Add Ubuntu bionic as supported distro" 2019-03-11 09:19:49 +00:00
Zuul
186db05ffb Merge "Add DIB_APT_MINIMAL_CREATE_INTERFACES toggle" 2019-03-05 07:24:24 +00:00
Logan V
8756cbea1b Add DIB_APT_MINIMAL_CREATE_INTERFACES toggle
Add a DIB_APT_MINIMAL_CREATE_INTERFACES boolean to the debootstrap
element which functions identically to
DIB_YUM_MINIMAL_CREATE_INTERFACES in the yum-minimal element.

This can be used to disable the creation of the
/etc/network/interfaces.d/eth[01] dhcp configuration files, which
are not needed on systems where cloud-init or other means are used
to configure networking.

The flag is enabled by default to keep creating the dhcp interface
files, maintaining backwards compatibility.

Change-Id: I1fdaca8350a5ceefd9e437af4fd000ce6a3ee7f3
2019-03-05 16:27:57 +11:00
Gaëtan Trellu
bbde9bb320 [lvm] Add Ubuntu bionic as supported distro
The way how LVM is created on Ubuntu Xenial and Ubuntu Bionic
is the same.

Change-Id: I16d548f6393dd3cdfd5a9befa5c0ef0f6db92df1
2019-03-04 15:05:37 -05:00
Zuul
0f8d340c6c Merge "Update gentoo-releng gpg key" 2019-02-28 00:33:06 +00:00
Matthew Thode
b2cc91d276
Update gentoo-releng gpg key
A new signing subkey is used.

Change-Id: Idc9aceba7ee144fd0307737c24991acfacf68985
2019-02-27 15:18:25 -06:00
Zuul
703549412d Merge "update spelling errors" 2019-02-25 08:49:56 +00:00
Zuul
b2e2d121f0 Merge "set rhel minor release" 2019-02-25 08:11:56 +00:00
Noam Angel
8b83196024 Add option to skip update packages
in same cases it is required to avoid update all existing packages,
doing so can result in release update which is currently not possible
unless you not include "base" element.

"base" element used for most distribution (rhel, debain), and is
necessary for most cloud operations, this patch add 
"DIB_AVOID_PACKAGES_UPDATE" parameter to skip updating all packages.

usecases for this patch can be:
 * Avoid release update when building old release ex. RHEL7.5.
 * build on network-less environment.

usage:
DIB_AVOID_PACKAGES_UPDATE=1

or
DIB_AVOID_PACKAGES_UPDATE=0

Change-Id: I71192b23c8f0bc48b348fe7377bf8a2399b53792
2019-02-25 06:31:30 +00:00
Ian Wienand
37dff9738a Fix opensuse 42.3 pip-and-virtualenv
Related to I041a141366099093805e6052b1bbf64efd277e1e, we also need to
remove this on opensuse.  The files for gate testing are added, but
the test is not added to any jobs at this point in the interests of
gate time.

Change-Id: I1af9e84d76bedcb2607717edc6d2abe2920b0584
2019-02-25 15:37:17 +11:00
Quique Llorente
5b1844acf9 Keep git after ironic-agent post
New versions of pbr depends on git [1] and IPA depends on pbr [2] so
removing git will remove pbr, IPA and friends.

[1] https://src.fedoraproject.org/rpms/python-pbr/blob/master/f/python-pbr.spec#_64
[2] https://github.com/rdo-packages/ironic-python-agent-distgit/blob/rpm-master/openstack-ironic-python-agent.spec#L85

Closes-Bug: #1816017
Depends-On: https://review.openstack.org/637668
Change-Id: I97f6b593e88e1cb81cd4bb2d77787bc012fb8271
2019-02-19 16:32:22 +11:00
Noam Angel
802dc35a61 set rhel minor release
Change-Id: I52a38c16dbbbe9fa1d4d6b6daffde01f63f664e6
2019-02-10 14:59:41 +00:00
Zuul
25ba034a0e Merge "pip-and-virtualenv: handle centos image-based builds" 2019-02-07 07:54:23 +00:00
Ian Wienand
d0906ad473 pip-and-virtualenv: handle centos image-based builds
This fixes a regression in I041a141366099093805e6052b1bbf64efd277e1e
where we starting skipping the removal of old files for image-based
builds (confusingly named centos7 rather than centos for historical
reasons).  Fix the check

Change-Id: I74688a9e91d833b5d654056431729bed0585616c
2019-02-07 10:56:29 +11:00
Zuul
dd8bbd5c4f Merge "fix systemd import-tar for gentoo" 2019-02-04 02:15:09 +00:00
Zuul
46ac931513 Merge "pip-and-virtualenv : only remove system files on centos" 2019-02-02 01:38:48 +00:00
Ian Wienand
ea1735b6a2 pip-and-virtualenv : only remove system files on centos
As described inline, we only want to remove the system package files
on centos; it causes problems on Fedora where some system tools expect
these to be there.

But there is an additional bug -- pip actually removes the system
package files anyway.  To work around this, reinstall the system
package.

Closes-Bug: #1813232
Change-Id: I041a141366099093805e6052b1bbf64efd277e1e
2019-02-01 11:01:45 +11:00
Ian Wienand
7cb5916a76 Enable dbus-broker for Fedora 29
As described in the comments, it seems the transition between
dbus-daemon -> dbus-broker in Fedora 29 has made it so the packages
can get into a state where neither service is enabled.

Explicitly install and enable dbus-broker for F29

Change-Id: I06753043a75be2f635653899c6c251b9fbdd7c67
2019-01-31 18:08:37 +11:00
Matthew Thode
f4a1c7f89f
fix systemd import-tar for gentoo
use a newer version of systemd
Fixes https://review.openstack.org/#/c/608102

Change-Id: I23fd671adb893f3abd9fbc65382f2aec5a317c24
2019-01-27 18:16:51 -06:00
Zuul
29f5759a4d Merge "[Configuration] Add missing py37 and corrected default envlist." 2019-01-22 20:22:38 +00:00
Carlos Goncalves
54d6cd0823 Add python3-setuptools to bindep.txt for Fedora
setuptools is required by python3-setools (a dependency of
policycoreutils-python) but is not in setools RPM dependency list in
Fedora 28. See [1] for DIB output.

This is a temporary workaround. It was fixed in Fedora rawhide and
Fedora 29 [2], but Fedora 28 is still unresolved.

[1] http://logs.openstack.org/81/600381/10/check/octavia-v2-dsvm-scenario-\
    fedora-latest/4ac1e5c/controller/logs/dib-build/amphora-x64-haproxy.\
    qcow2_log.txt.gz#_2019-01-21_10_15_49_816
[2] https://bugzilla.redhat.com/show_bug.cgi?id=1623371

Change-Id: Ie8c827d8217d885c3031c4f45aa2556951b01e83
2019-01-22 13:00:06 +01:00
Zuul
379f1bdfc0 Merge "support cracklib in pam for Gentoo's musl profile" 2019-01-21 05:03:48 +00:00
Zuul
11f1dd653f Merge "[Core] Change openstack-dev to openstack-discuss." 2019-01-20 23:41:51 +00:00
Zuul
ad23eef69c Merge "Change phase to check for dracut-regenerate in iscsi-boot" 2019-01-18 09:57:14 +00:00
Matthew Thode
1bde2591ae
support cracklib in pam for Gentoo's musl profile
Needed for any musl build, is default in other profiles

Change-Id: Ib7cae9124f5846d33c05f26befd8f13646a08610
2019-01-15 09:47:46 -06:00
Michael Johnson
cfba9ea79d Make sure $TMP_BUILD_DIR/mnt is owned by root
The path $TMP_BUILD_DIR/mnt becomes the / inside the chroot during
the chroot phases of diskimage-builder. Previously this path was being
created using the account running diskimage-builder. This account may
not be valid inside the chroot. This causes path validation, when running
on a Ubuntu bionic host, to fail.
This patch chown's the $TMP_BUILD_DIR/mnt to root.root to make sure
that / is owned by a valid account inside the chroot.

Change-Id: Ifedc136baa67c7952942aed2c8cb1041902fef91
Closes-Bug: 1811113
2019-01-09 20:08:15 -08:00
Zuul
8ca7b6b9b9 Merge "Add policycoreutils-python-utils to bindep" 2019-01-10 00:22:27 +00:00
Zuul
b0d41230e4 Merge "Delete the duplicate words in 50-zipl" 2019-01-10 00:17:53 +00:00
Zuul
75cb5f6db7 Merge "Only cap pylint for python2" 2019-01-10 00:17:52 +00:00
Natal Ngétal
07657a73e9 [Configuration] Add missing py37 and corrected default envlist.
Change order of default tox envlist in order of likeliness to fail on a
new change and add py37 and py36.

Change-Id: I9d21561991c1d956a62de72c906cf91305ce5328
2019-01-09 11:41:22 +01:00
Natal Ngétal
14f67f3463 [Core] Change openstack-dev to openstack-discuss.
The openstack-dev address is replaces by openstack-discuss.

Change-Id: Iba766d86db3c3f98a52e8dd17b61ac16083d2333
2019-01-09 11:40:30 +01:00
Zuul
8d3fa3a85c Merge "simple-init: allow for NetworkManager support" 2019-01-09 03:38:25 +00:00