I just saw in the trace output of a failure
> grep -o 'CentOS-.[^>]*GenericCloud-.[^>]*.qcow2'
> sort -r
> head -1
sort: fflush failed: 'standard output': Broken pipe
sort: write error
i.e. the "head -1" has exited after reading one line, but "sort -r"
still wants to write and thus has hit a pipe failure, and because we
run with "-o pipefail" this has halted the script.
This seems like it has been there more or less forever, maybe we just
got lucky hitting it now? Anyway, we can work around this by using a
process substitution and passing the output of this into head, this
way we won't hit a pipe failure.
I also updated the fedora path as it does the same thing.
Change-Id: I44d97e5bb31702aacf396e0229329a2ef9c64f2f
This reverts the mirror removal in
I817b412b7f06523df635e8b16111bc1081b40f66 and updates the test to F35,
which is mirrored.
Change-Id: I00d24690f57bedd3fc5ebbc18de0ed874ad1e4ef
The fedora and fedora-minimal elements currently test Fedora <=34. The
opendev ci mirrors no longer mirror Fedora 34 which means we cannot rely
on those mirrors for these tests. Remove the mirror configuration when
testing these older Fedora builds.
Change-Id: I817b412b7f06523df635e8b16111bc1081b40f66
The reality is that "stable" is what is tested. This tries to give
enough info that users can ascertain what tests are running at any
given time and hence what elements are known to be working.
Additional, clarify the Fedora position in the README as now described
by above.
Closes: #1653561
Change-Id: Ifb91b9089790897861bd7e671c3dba59adac239d
Fedora cloud images have sub-releases in their filename. It is not
exacly clear how this is generated but we do know how we can determine
the greatest programatically.
Change-Id: I7fc56897c681fe037db211c290edcdd23cdd5d5b
As of grub2 >= 2.02-95 on redhat family distros, calling grub2-install
on an EFI partition will fail with: "this utility cannot be used for
EFI platforms because it does not support UEFI Secure Boot."
This version of grub is now in centos8-stream and non-eus repos of
RHEL-8. It is not currently possible to build whole-disk UEFI images
on these distros, and when this package is promoted this will also
affect centos8 and RHEL-8 eus. The grub maintainers made this change
because the grub2-install generated /boot/efi/EFI/BOOT/BOOTX64.EFI
will never be capable of booting with Secure Boot.
This change defines a $EFI_BOOT_DIR for every distro element. When
directory /boot/efi/$EFI_BOOT_DIR exists a grub.cfg file in will be
generated there. This change also installs the shim package on redhat
family distros, which installs a copy of the shim bootloader to
/boot/efi/EFI/BOOT/BOOTX64.EFI. Using centos as an example, this
allows UEFI to boot the shim /boot/efi/EFI/BOOT/BOOTX64.EFI which
then chains to /boot/efi/EFI/centos/grubx64.efi.
If /boot/efi/$EFI_BOOT_DIR doesn't exist (such as for Ubuntu,
/boot/efi/EFI/ubuntu) the current behaviour of running grub-install to
generate /boot/efi/EFI/BOOT/BOOTX64.EFI will continue. For distros
such as Ubutnu where packaging does not populate /boot/efi/EFI/ubuntu
with .efi files, secure boot can be added in the future by copying
.efi files to /boot/efi/EFI/ubuntu and copying the shim file to
/boot/efi/EFI/BOOT/BOOTX64.EFI.
Change-Id: I90925218ff2aa4c4daffcf86e686b6d98d6b0f21
The python3/python3-pyyaml packages both are never installed and dnf
itself never updated when $DIB_DISTRIBUTION_MIRROR set and used.
This change fix the order of the operations:
1. yum/dnf configure.
2. *.repo patching.
3. yum/dnf update/install execution.
Change-Id: Ifbbf1f0190fe8c8a77fb3be820e8056447e755f6
Signed-off-by: Maksim Malchuk <maksim.malchuk@gmail.com>
The base URL of EPEL repository installed by the epel-release package in
CentOS 8 at least now defaults to https.
The error seen when building an CentOS 8 image was:
"Error: Cannot find a valid baseurl for repo: epel"
This patch fixes it so that it will always match regardless of being
http or https.
Change-Id: I9ec5536ee72047c929a1ef6950ff4e9092842a4c
Update builds to Fedora 29. Remove the openstack gate CI mirror
workaround for pre-28 versions as they're not building in the gate any
more.
Change-Id: Ia6a8ae8d66d69f6add39e571043328e7274ba26c
This updates diskimage-builder to support current Fedora releases (27
and 28) and removes support for Fedora 26 which is EOL as of June
2018.
Change-Id: I602b22ed4d5397b39dc1eef67964f6fbdcd93060
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
This updates diskimage-builder to support current Fedora releases (26
and 27) and removes support for Fedora 25 which is EOL as of December
12, 2017.
Change-Id: I227a607c6c468cc8b7bb154a189e9c8ce2021192
As described in the comment, we need to create the /etc/machine-id for
the image-based build when systemd isn't updated (as is usually the
case for a new distro)
Work on clearing this out continues, but this brings it to parity with
fedora-minimal.
Change-Id: Icbbbabb4114d4d95909648d8e39a6bae6d2a7b7b
Depends-On: I761e425f8a658669d9b8a70ce4260cec263ea51a
The URL we are using seems to have disappeared. Update this to
download.fedoraproject.org. The new URL requires a "subrelease" now,
add it, along with a note on where it comes from.
Change-Id: I761e425f8a658669d9b8a70ce4260cec263ea51a
Fedora 26 is now the latest release:
https://fedoraproject.org/wiki/Releases/26/Schedule
We are building and using these in infra now
Change-Id: I012c2d28255be274e88abc2751d968bafaf76fbb
Depends-On: Ieba5f69020a13681074f72cfca2955071801b63a
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
We added the DIB_distro_DISTRIBUTION_MIRROR arguments with
I92964b17ec3e47cf97e3a3091f054b2a205ac768 as a way that we could
source a list of mirrors and then have the distro elements choose
which one applied to them.
However, this hasn't worked out to be so useful. The
openstack-ci-mirrors element is working as a mirror setup script -- it
translates the openstack CI mirror list variables into the generic
"DIB_DISTRIBUTION_MIRROR" as appropriate for each distro's build.
Also, it turns out there's other things that need to be done, such as
turning off gpg checking, which mean the idea of "just export
variables" hasn't turned out as valid ... you need actual code
involved to get it right.
AFAICT we never actually documented these, and they do not seem to be
in use. They have caused considerable confusion when dealing with new
platforms as we try to keep consistency. Remove them.
[1] http://codesearch.openstack.org/?q=DIB_.*_DISTRIBUTION_MIRROR&i=nope&files=&repos=
Change-Id: Ifc4ab700631ffdfbe790068558f670f9a11dde5e
fedora/centos-minimal don't obey DIB_DISTRIBUTION_MIRROR currently. I
don't really want them too -- we want to be able to separate the
mirrors used during the build process from those embedded into the
final image. Add DIB_YUM_MINIMAL_BOOTSTRAP_REPOS which is a directory
with repo files to use during the install.
This introduces setup-gate-mirrors.sh which is intended to setup
repo/sources/whatever files in the openstack gate that point to the
local region mirror. It pulls the info from the mirror_info.sh script
on each CI node.
The openstack-ci-mirrors element is updated to export these variables.
elements are updated to depend on it. Tests are restored
Change-Id: I7604fc4d41cb1483be16b8d628a24e8fc764f515
The dib-run-parts element was copying our internal version of
dib-run-parts into /usr/local/bin to be used running scripts inside
the target chroot. However, it never cleaned up after itself. This
means all images were left with an unmanaged local install of
dib-run-parts.
This copies dib-run-parts into the hooks directory of the chroot and
runs it from there. It is cleaned up automatically on the exit path.
The dib-run-parts element is no longer required and it has been
removed from all dependencies. It is left with a deprecation notice
in the README. For compatability we convert it to simply install
dib-utils.
Codesearch shows no users depending on this unintentional implicit
install. Note os-refresh-config depends on dib-utils and thus will
have an explicitly installed version.
Partial-Bug: #1673144
Change-Id: Ia2e96c00a4246c04beb96c17f83b8aefb69219ca
Currently we have all our elements and library files in a top-level
directory and install them into
<root>/share/diskimage-builder/[elements|lib] (where root is either /
or the root of a virtualenv).
The problem with this is that editable/development installs (pip -e)
do *not* install data_files. Thus we have no canonical location to
look for elements -- leading to the various odd things we do such as a
whole bunch of guessing at the top of disk-image-create and having a
special test-loader in tests/test_elements.py so we can run python
unit tests on those elements that have it.
data_files is really the wrong thing to use for what are essentially
assets of the program. data_files install works well for things like
config-files, init.d files or dropping documentation files.
By moving the elements under the diskimage_builder package, we always
know where they are relative to where we import from. In fact,
pkg_resources has an api for this which we wrap in the new
diskimage_builder/paths.py helper [1].
We use this helper to find the correct path in the couple of places we
need to find the base-elements dir, and for the paths to import the
library shell functions.
Elements such as svc-map and pkg-map include python unit-tests, which
we do not need tests/test_elements.py to special-case load any more.
They just get found automatically by the normal subunit loader.
I have a follow-on change (I69ca3d26fede0506a6353c077c69f735c8d84d28)
to move disk-image-create to a regular python entry-point.
Unfortunately, this has to move to work with setuptools. You'd think
a symlink under diskimage_builder/[elements|lib] would work, but it
doesn't.
[1] this API handles stuff like getting files out of .zip archive
modules, which we don't do. Essentially for us it's returning
__file__.
Change-Id: I5e3e3c97f385b1a4ff2031a161a55b231895df5b
