This adds arm64 ubuntu-minimal Jammy functests and x86 ubuntu image
based Jammy functests. To make this happen we have to install
debootstrap from debian unstable on the functest nodes in order to get
access to a debootstrap that knows what jammy is.
As we ramp up Jammy support in our tools having good testing will be
helpful.
Change-Id: I1d1dc752ce176457d0656cbd50e27a2721ca9856
This change:
- adds a note regarding an error when building focal ubuntu-minimal
images on operating systems with older versions of debootstrap
- adds a reference to where the DIB_RELEASE variable definition can be
found
Closes-Bug: #1941831
Change-Id: Ibc1e04dba0562c4f4909a8cb8af041d9b8ac45c4
As noted inline, this works around potential issues by being a strong
indication you are in a container (e.g. [1]). Since nothing should be
changing anything on the host/build system, this is a generically
safer way to operate.
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1975588
Change-Id: Ic6802c4ffc2e825f129af10717860a2d1770fe80
* Add "DIB_UBUNTU_MIRROR_DISTS":
Default: ``updates,security,backports``
Notes: For some deployment, is may be required
to disable backport|update|etc packages
integration.
Change-Id: Ic7dcd29ea658a66763b4422915149e4d3fe663cc
We are at the point that all distributions we are building have Python
3, so any tools running in the chroot can assume Python 3 exists.
This makes dib-python redundant; mark it as deprecated and start to
remove it from elements where it is no longer required.
Change-Id: I5d852843ec65d3b04444b77c54c5b82424455cd8
This reverts commit 6e549c33ac.
It uses the new multiple-parameter matching format from
Idff7b067ad4255e6fc4138f7eff313a81b75c8ba to actually do what it says.
Change-Id: I4656ff1a5c46bcfbd8587f2f541825f4ad08820f
This reverts commit 14ff8f942c.
This seems to not be installing the kernel at all, and needs further
investigation.
Change-Id: Ifd809d4b67aff5d80f979235db246a16af0375b3
Only install the HWE kernel by default for Xenial. This was actually
installing the 16.04 HWE kernel on Bionic by accident, since it seems
to have that package; however it was breaking Focal.
On the other distros, just install the default generic kernel. Let's
KISS for now if we can ...
Change-Id: I34e27d821fbefe274e7b007f37b0bd34db2e1d26
This patch adds a new environment variable to the ubuntu-minimal
element called DIB_UBUNTU_KERNEL that allows you to specify the kernel
meta package that will be using to install the kernel inside the image.
It supports "linux-image-generic" (The default), "linux-image-kvm", and
"linux-image-virtual".
This allows building images that are smaller in size (~200MB smaller
qcow2) that have only the kernel modules necessary for virtual
machines.
Change-Id: I8ce65e1d357d15e8ed8995ad1dcaea02bbd1986f
The apt sources are set out in root.d/75-ubuntu-minimal-baseinstall
and the cache is updated, cleaned and a dist-upgrade is done there.
As such, this file is unnecessary.
Change-Id: Idab5ede3f235bc204c4bdebf40fbcf4a12e5bc2f
The ubuntu, and ubuntu-minimal elements both make use of a common set
of environment settings to determine the distribution name.
The ubuntu-minimal element also does a few extra things which would
appear to apply to both sets and bring in extra architecture support.
As such, these are included in the common element.
This intends to be part of a series of patches which will eventually
create a new element to build a minimal ubuntu-systemd-container
element which can be used for lxc/nspawn containers.
Change-Id: Ia4e620f7d3fa6215484a8d218cea2f28bd1ffaee
When building with debootstrap, debootstrap will use the key to check
that everything is properly signed. It will not `apt-key add` the key
into the final environment, however.
Early adding the key after debootstrap before we need to read from the
private repo again prevents unsigned issues. This also maintains the
integrity of the packages in the environment throughout the build.
Change-Id: I5ca75ae4620c9fb26b512cb30f8cd79fa7a0373a
Add a bionic test in replacement of trusty. We are already building
bionic images in the gate, so this seems like a good time to switch.
Change-Id: I20d4c25e9b79e7326c86767c36be8615ba0888a3
Install hwe kernel for ubuntu-minimal. As noted this is currently
Xenial specific; we need this for initial bring-up so let's tackle
future releases as things progress.
Ensure we use ttyAMA0 for arm64 console too.
Change-Id: Ic607cf8369666dc24929aff6f2ef8a72e7980599
This moves the block-device default out of the "vm" element and into a
selection of other elements. There's "mbr" which retains the status
quo. There's an EFI version that has the boot/grub partitions as
required. In between there's the GPT only version, which is useful
for architectures like power without EFI, but still want possible
larger disks using GPT.
Change-Id: I4a566a97d073fc0dda0ab2494ac988fe015800a9
Like we did in https://review.openstack.org/475206 we need to install
systemd sooner because of the new world order of containers.
Change-Id: Ia60d751fee3af6f8d72ad664107acb337360feca
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
A small update was made to 4.4.0-96.119 that dropped the
initramfs-tools dependency from the kernel [1]. This had the
unfortunate affect of removing the initramfs from ubuntu-minimal and
making it unbootable, since we specify the root device via LABEL=.
Add the package explicitly alongside the kernel.
Also, small fix to pass unit tests
[1] https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1700972
Change-Id: I57a0f08cd5e082ecdf8dba0ab34fb3062c50836d
We added the DIB_distro_DISTRIBUTION_MIRROR arguments with
I92964b17ec3e47cf97e3a3091f054b2a205ac768 as a way that we could
source a list of mirrors and then have the distro elements choose
which one applied to them.
However, this hasn't worked out to be so useful. The
openstack-ci-mirrors element is working as a mirror setup script -- it
translates the openstack CI mirror list variables into the generic
"DIB_DISTRIBUTION_MIRROR" as appropriate for each distro's build.
Also, it turns out there's other things that need to be done, such as
turning off gpg checking, which mean the idea of "just export
variables" hasn't turned out as valid ... you need actual code
involved to get it right.
AFAICT we never actually documented these, and they do not seem to be
in use. They have caused considerable confusion when dealing with new
platforms as we try to keep consistency. Remove them.
[1] http://codesearch.openstack.org/?q=DIB_.*_DISTRIBUTION_MIRROR&i=nope&files=&repos=
Change-Id: Ifc4ab700631ffdfbe790068558f670f9a11dde5e
This adds "openstack-ci-mirrors" element which performs various
settings to get builds using local mirrors. As a first step, we
convert ubuntu-minimal jobs
The main trick is that since infra mirrors are created with rerepo
they are not signed (they are recreated, not cloned, and not signing
is seen as a feature in that it deters external use). So we need to
instruct debootstrap to ignore signing and also turn it off for
in-chroot apt. Other than that, the existing DIB_DISTRIBUTION_MIRROR
works to redirect installs.
Remove "restricted" as it's not mirrored, and I don't think we want it
in here by default.
(I think DIB_DISTRIBUTION_MIRROR is a bit of an anti-pattern, because
it leaves the mirrors in the final image -- just because you use them
to build, doesn't mean you want them at runtime). But we don't need
to fix that now, and we don't use any created images.)
This pauses fedora testing until the next change, which moves to using
local mirrors for testing on fedora/centos
Change-Id: I778bd05a1e615c27edf1c9f0a1409119a6b3a850
Ubuntu 12.04 LTS reached its regular End of Life on April 28, 2017.
Depends-On: I5e145095a10db112bb27516bfe652d2cdc052a61
Change-Id: I64af4c5183d77a75dcd062895d19b0a1330c8da8
This patch removes three nearly-copies of debootstrap documentation
and fixes some documentation aspects.
Change-Id: Ief7794f5c1abad73788c063af6c862472cd34744
Signed-off-by: Andreas Florath <andreas@florath.net>
diskimage-builder usually provides defaults that work out of the box.
One default that does not work outside of x86 land is Ubuntu distro
mirror url. Considering there are only two valid default options, we can
automatically choose a better default.
This patch changes behavior only for architectures known to be using
http://ports.ubuntu.com/ubuntu-ports. All others still would use
http://archive.ubuntu.com/ubuntu as default. It provides some guarantee
that we do not introduce a regression.
Change-Id: If95a64bac0c88f30736da4bae7f1fdce126c0bf6
We somewhat discussed skipping qcow2 generation previously in
I9372e195913798a851c96e62eee89029e067baa1. As recent issues with PPC
testing have shown, we are not actually testing the "vm" element and
hence the bootloader path in the functional tests.
I don't think we need to test this on every element; it overlaps
somewhat with the testing done by the nodepool jobs which build full
images and boot them. I also didn't want to introduce a separate run
for this. Thus it seems valuable to at least have one element
enhanced to do this installation and conversion in our default tests
for basic sanity.
This disables qcow generation by default, as per the other change, but
allows an element to drop a file that will override the output
formats. The Xenial element is modified to produce a qcow2 using
this, and also introduces a dependency on the "vm" element so it tries
to install the bootloader.
We now exit if the .qcow2 fails to build as well.
Change-Id: I1a6acefe52f8c696c39b2d592fdc7ae32a87e6fe
Currently there is no description of dependencies in the generated
documentation of the elements: therefore a user of an element does not
know which other elements are automatically included and e.g. which
configuration options are available. In addition there are some
copy&pastes of parts of the README.rst scattered thought different
Ubuntu and Debian specific elements.
This patch adds a semi-automatic generation of dependency information
of all elements. Nevertheless these are not automatically included.
The author of the element's README.rst can decide if and where the
dependency information should appear and can use the descriptor
.. element_deps::
for this.
This patch adds the dependency information for some Debian and
Ubuntu patches - and creates the base for later removing the
duplicated parts.
A call is added to element_dependencies._find_all_elements() to
populate reverse dependencies for Element objects.
(This is a reworking of I31d2b6050b6c46fefe37378698e9a330025db430 for
the feature/v2 branch)
Change-Id: Iebb83916fed71565071246baa550849eef40560b
Currently we have all our elements and library files in a top-level
directory and install them into
<root>/share/diskimage-builder/[elements|lib] (where root is either /
or the root of a virtualenv).
The problem with this is that editable/development installs (pip -e)
do *not* install data_files. Thus we have no canonical location to
look for elements -- leading to the various odd things we do such as a
whole bunch of guessing at the top of disk-image-create and having a
special test-loader in tests/test_elements.py so we can run python
unit tests on those elements that have it.
data_files is really the wrong thing to use for what are essentially
assets of the program. data_files install works well for things like
config-files, init.d files or dropping documentation files.
By moving the elements under the diskimage_builder package, we always
know where they are relative to where we import from. In fact,
pkg_resources has an api for this which we wrap in the new
diskimage_builder/paths.py helper [1].
We use this helper to find the correct path in the couple of places we
need to find the base-elements dir, and for the paths to import the
library shell functions.
Elements such as svc-map and pkg-map include python unit-tests, which
we do not need tests/test_elements.py to special-case load any more.
They just get found automatically by the normal subunit loader.
I have a follow-on change (I69ca3d26fede0506a6353c077c69f735c8d84d28)
to move disk-image-create to a regular python entry-point.
Unfortunately, this has to move to work with setuptools. You'd think
a symlink under diskimage_builder/[elements|lib] would work, but it
doesn't.
[1] this API handles stuff like getting files out of .zip archive
modules, which we don't do. Essentially for us it's returning
__file__.
Change-Id: I5e3e3c97f385b1a4ff2031a161a55b231895df5b
