When using the upstream cloud images with the "ubuntu" element, they
have universe and multiverse enabled which we don't mirror.
To use the infra mirrors as a DIB_DISTRIBUTION_MIRROR with this
element, we need to be able to skip redirecting to universe and
multiverse, and additionally enable insecure repos (as we don't gpg
sign our mirrors).
Add and document two new variables with the ubuntu element to do this.
This is then setup by the openstack-ci-mirrors element so that we use
local mirrors duing dib functional testing for the "ubuntu" element.
Change-Id: I6ffbde07fa0e103641ee5c5f9d9e854e5b2168dc
Patch allows to rebuild arbitrary images, which location, filename and
sha256sum are specified in variables, not only hardcoded $DIB_RELEASE/current.
Change-Id: I05418932a0c40d885fe00a49f1f49d7e86c67518
This moves the block-device default out of the "vm" element and into a
selection of other elements. There's "mbr" which retains the status
quo. There's an EFI version that has the boot/grub partitions as
required. In between there's the GPT only version, which is useful
for architectures like power without EFI, but still want possible
larger disks using GPT.
Change-Id: I4a566a97d073fc0dda0ab2494ac988fe015800a9
In a couple of places we use flock for critical sections, but we leave
lockfiles around in various locations which can be confusing.
Introduce DIB_LOCKFILES global (under ~/.cache/dib/lockfiles) and
write lockfiles in there.
Fix up removal of the lockfile in the yum path; we just want to make
sure we cleanup the .rpmmacros file, but we don't need to remove the
lockfile as well.
Co-Authored-By: Andreas Florath <andreas@florath.net>
Change-Id: Ie810b2836be521325afe923708d046112e1e1e20
The image download tests have long been too unreliable for the gate.
We need to cache the base images similar to how devstack caches it's
testing images. Let's move them to non-voting jobs for the time
being.
This means that the gate jobs are now all based on "-minimal" and are
using infra mirrors. Unfortunately, there is still some unreliability
because we currently have issues with infra mirrors being very slow
after AFS updates, leading to job timeouts. But we're on the right
path...
Also, I noticed we don't have tests of the "ubuntu" image-download
based tests, which were tacitly being tested by apt-sources before we
moved that to -minimal. Add simple tests for these.
Change-Id: Ie33ee49656872467ef68d753210032156bb6b2cb
This patch removes three nearly-copies of debootstrap documentation
and fixes some documentation aspects.
Change-Id: Ief7794f5c1abad73788c063af6c862472cd34744
Signed-off-by: Andreas Florath <andreas@florath.net>
The dib-run-parts element was copying our internal version of
dib-run-parts into /usr/local/bin to be used running scripts inside
the target chroot. However, it never cleaned up after itself. This
means all images were left with an unmanaged local install of
dib-run-parts.
This copies dib-run-parts into the hooks directory of the chroot and
runs it from there. It is cleaned up automatically on the exit path.
The dib-run-parts element is no longer required and it has been
removed from all dependencies. It is left with a deprecation notice
in the README. For compatability we convert it to simply install
dib-utils.
Codesearch shows no users depending on this unintentional implicit
install. Note os-refresh-config depends on dib-utils and thus will
have an explicitly installed version.
Partial-Bug: #1673144
Change-Id: Ia2e96c00a4246c04beb96c17f83b8aefb69219ca
Currently there is no description of dependencies in the generated
documentation of the elements: therefore a user of an element does not
know which other elements are automatically included and e.g. which
configuration options are available. In addition there are some
copy&pastes of parts of the README.rst scattered thought different
Ubuntu and Debian specific elements.
This patch adds a semi-automatic generation of dependency information
of all elements. Nevertheless these are not automatically included.
The author of the element's README.rst can decide if and where the
dependency information should appear and can use the descriptor
.. element_deps::
for this.
This patch adds the dependency information for some Debian and
Ubuntu patches - and creates the base for later removing the
duplicated parts.
A call is added to element_dependencies._find_all_elements() to
populate reverse dependencies for Element objects.
(This is a reworking of I31d2b6050b6c46fefe37378698e9a330025db430 for
the feature/v2 branch)
Change-Id: Iebb83916fed71565071246baa550849eef40560b
Currently we have all our elements and library files in a top-level
directory and install them into
<root>/share/diskimage-builder/[elements|lib] (where root is either /
or the root of a virtualenv).
The problem with this is that editable/development installs (pip -e)
do *not* install data_files. Thus we have no canonical location to
look for elements -- leading to the various odd things we do such as a
whole bunch of guessing at the top of disk-image-create and having a
special test-loader in tests/test_elements.py so we can run python
unit tests on those elements that have it.
data_files is really the wrong thing to use for what are essentially
assets of the program. data_files install works well for things like
config-files, init.d files or dropping documentation files.
By moving the elements under the diskimage_builder package, we always
know where they are relative to where we import from. In fact,
pkg_resources has an api for this which we wrap in the new
diskimage_builder/paths.py helper [1].
We use this helper to find the correct path in the couple of places we
need to find the base-elements dir, and for the paths to import the
library shell functions.
Elements such as svc-map and pkg-map include python unit-tests, which
we do not need tests/test_elements.py to special-case load any more.
They just get found automatically by the normal subunit loader.
I have a follow-on change (I69ca3d26fede0506a6353c077c69f735c8d84d28)
to move disk-image-create to a regular python entry-point.
Unfortunately, this has to move to work with setuptools. You'd think
a symlink under diskimage_builder/[elements|lib] would work, but it
doesn't.
[1] this API handles stuff like getting files out of .zip archive
modules, which we don't do. Essentially for us it's returning
__file__.
Change-Id: I5e3e3c97f385b1a4ff2031a161a55b231895df5b
