Commit Graph

602 Commits

Author SHA1 Message Date
Zuul
5d16300f00 Merge "Download latest CentOS cloud image" 2020-07-05 21:06:43 +00:00
Zuul
9ab6811e84 Merge "Add support for CentOS 8 Stream" 2020-07-05 21:06:41 +00:00
yatinkarel
e999368d2c Disable all enabled epel repos in CentOS8
EPEL centos8 have different epel repos like
epel, epel-modular, epel-playground etc, so
we need to disable all not just epel.

Also ensure other epel repositories in CentOS7
like epel-testing are also disabled.

Related-Bug: #1885315
Change-Id: I02b3b83fa2047702d5f069d3ca1c9c0bcc1dab52
2020-06-30 10:15:30 +05:30
Carlos Goncalves
1f9619cdb5 Download latest CentOS cloud image
DIB was retrieving the oldest cloud image file which, presently, means
retrieving CentOS 8.1 instead of CentOS 8.2. Even though DIB runs a
system update and so catches up to latest, this takes bandwidth, time
and final image space (8.1 + system update = 765M qcow2, vs 8.2 + system
update = 518M qcow2).

This patch fixes that by taking the first image name in a descending
order list.

Change-Id: I648fe19f1f76c03c97492b6ac7be6381f6f9261b
2020-06-22 11:03:36 +02:00
Carlos Goncalves
367dfc9294 Add support for CentOS 8 Stream
This patch adds support for CentOS 8 Stream [1] to the centos-minimal
element. Users should set DIB_RELEASE=8-stream.

[1] https://www.centos.org/stream/

Change-Id: Id0825de735ab957c10daf35fb3c641f850cc6847
2020-06-22 10:36:30 +02:00
Matthew Thode
52a8b2fa01
update grub cmdline to current kernel parameters
vga=normal is depreciated, use gfxpayload=text instead

Change-Id: I2e5be3b07acce14a9f3d47e24938d9da3c5b6c48
2020-06-14 00:39:42 -05:00
Zuul
3614900572 Merge "Fix yumdownloader cache dir" 2020-06-10 16:24:53 +00:00
Zuul
a0714c2300 Merge "Debuntu: add apt-transport-https" 2020-06-09 10:33:56 +00:00
Carlos Goncalves
9d9dd9249c Fix yumdownloader cache dir
TMPDIR env is not being honored as cache dir in EL 8.

Change-Id: I8281675ec5f0951b3e190a8d6727744a1a5cd8d7
2020-06-07 21:58:49 +02:00
Ian Wienand
0c94eef7be Revert "dib-lint: use yamllint to parse YAML files"
This reverts commit 6ee2995214 and
e85c2a6f03.

I missed that if you pip install and then run dib-lint, it's not going
to pick up the .yamllint file shipped here.  Thus it gives spurious
errors.

The reason for this was simply better duplicate key detection in yaml
files, which caused us problems with the kernel installs.  However, at
this point it seems just the old "does it load" test from pyyaml will
be enough.

Change-Id: I87a9fc9bb119cfeffad48fc0fa0df31f0181825d
2020-05-28 16:44:49 +10:00
Ian Wienand
d2eaaaf622 Merge "Pre-install xz package in opensuse chroot" 2020-05-28 02:06:30 +00:00
Matthew Thode
041bdd331a
use stage3 instead of stage4 for gentoo builds
The main reason for using the stage4 is now gone (kernel compile).
Install and use the distro provided binary kernel package.

In addition to this, set the locale and timezone, beyond that very
little was done in the gentoo stage4.

Change-Id: I541b7d9b807e2357398ae1c249b1978958dd1137
Signed-off-by: Matthew Thode <mthode@mthode.org>
2020-05-27 18:50:16 -05:00
Colleen Murphy
19b3586d08 Pre-install xz package in opensuse chroot
As of recently, opensuse-minimal images fail to build because of an
error installing the kernel-default package:

> Problem: kernel-default-5.6.12-1.3.x86_64 requires mkinitrd >= 2.7.1, but this requirement cannot be provided
>   not installable providers: dracut-050+suse.61.g0fe0e854-1.1.i586[repo-oss]
>                    dracut-050+suse.61.g0fe0e854-1.1.x86_64[repo-oss]

The problem is there is a recently added package `busybox-links` which
provides a subpackage `busybox-xz` which provides the /usr/bin/xz
utility. Since this is available, the `aaa_base` package installs it
during the root.d base installation phase to fulfill it's dependency on
/usr/bin/xz. On the other hand, the dracut package explicitly requires
the `xz` package, and this is not co-installable with the `busybox-xz`
package, so the dracut package is not installable during the install.d
phase. This change explicitly adds the `xz` package to the initial
chroot provisioning phase so that the /usr/bin/xz requirement is already
fulfilled and `busybox-xz` does not get installed.

Change-Id: Iba8c301eb496657873963e1aa99736aacf87cb00
2020-05-27 10:45:03 -07:00
Ian Wienand
339d713527 Revert "Revert "ubuntu-minimal : only install 16.04 HWE kernel on xenial""
This reverts commit 6e549c33ac.

It uses the new multiple-parameter matching format from
Idff7b067ad4255e6fc4138f7eff313a81b75c8ba to actually do what it says.

Change-Id: I4656ff1a5c46bcfbd8587f2f541825f4ad08820f
2020-05-27 06:18:02 +10:00
Ian Wienand
b71d1c60d2 package-installs : allow a list of parameters
The change Ia6f10741fa6be24b11d6991c8a6b6e07951ff68d introduced having
"when:" as a list of values.  However, this was actually not
sufficient to express the logic required for arm64/x86_64/xenial
kernel matching we wanted.

Because the package name is a key, we can't have multiple entires in
the package-map YAML files.  This means we can't do more advanced
matching and thus we need to be able to match through multiple
parameters.  Similar to Ia6f10741fa6be24b11d6991c8a6b6e07951ff68d we
modify the matching rules to allow a list.

A an example of using this is provided in the README.rst, and this
same example worked through by the unit tests.

This also slightly updates the matching logic to be more sequential.
After each check we either continue on or log the failure and continue
to the next check (rather than set a list of flags then check that at
the end).  This makes it much easier to understand what is being
matched in the logging output from the tool.

Change-Id: Idff7b067ad4255e6fc4138f7eff313a81b75c8ba
2020-05-27 06:17:57 +10:00
Ian Wienand
6ee2995214 dib-lint: use yamllint to parse YAML files
This gives us better linting of YAML files that just opening them.
This would have detected the duplicate keys in
I34e27d821fbefe274e7b007f37b0bd34db2e1d26.

The .yamllint is taken from zuul-jobs where it is also used as a
fairly sane set of default rules.

A few minor newline fixes are added.

Change-Id: I96d6644ae24f7deb84fa50fefbda0f0d33e0e009
2020-05-26 12:04:09 +10:00
Ian Wienand
6e549c33ac Revert "ubuntu-minimal : only install 16.04 HWE kernel on xenial"
This reverts commit 14ff8f942c.

This seems to not be installing the kernel at all, and needs further
investigation.

Change-Id: Ifd809d4b67aff5d80f979235db246a16af0375b3
2020-05-25 18:48:51 +10:00
Ian Wienand
7539e241da ubuntu-minimal: Add Ubuntu Focal test build
Add test builds for Focal on x86 and ARM64

Change-Id: Idb23f0e00d37c7447441ea002aad078e8c61f969
2020-05-21 14:03:54 +10:00
Ian Wienand
14ff8f942c ubuntu-minimal : only install 16.04 HWE kernel on xenial
Only install the HWE kernel by default for Xenial.  This was actually
installing the 16.04 HWE kernel on Bionic by accident, since it seems
to have that package; however it was breaking Focal.

On the other distros, just install the default generic kernel.  Let's
KISS for now if we can ...

Change-Id: I34e27d821fbefe274e7b007f37b0bd34db2e1d26
2020-05-21 14:03:54 +10:00
Ian Wienand
c5acc91574 ubuntu-minimal: fix HWE install for focal
On Focal, install the 20.04 HWE kernel if linux-image-hwe is
specified.

Change-Id: I5d536e8f64e7b987415849d8cd4da7959bba7af7
2020-05-21 14:03:54 +10:00
Ian Wienand
138d3f9b81 package-installs: allow when filter to be a list
Allow the "when:" statements to be a list of values, which are
effectively anded together to filter the package install.

Change-Id: Ia6f10741fa6be24b11d6991c8a6b6e07951ff68d
2020-05-21 14:03:49 +10:00
Zuul
534b799de8 Merge "pip-and-virtualenv : fix fedora 30 install" 2020-04-24 05:34:35 +00:00
Zuul
0d0a5909e7 Merge "yum-minimal: strip env vars in chroot calls" 2020-04-24 05:34:33 +00:00
Ian Wienand
a8a5b1ac77 pip-and-virtualenv : fix fedora 30 install
This should be installing the python2 and python3 packages (that's
what pip-and-virtualenv is designed to do), but we dropped the +=
accidentally in ee9ad32b6f.

However, we've moved on anyway and after
I7a6a342461d6001c25e55638ba9b7438c28f2519 F31 doesn't support this
element.  fedora-latest is already updated to f31 in the opendev gate.
Remove the testing as it is no longer relevant.

Change-Id: Id696a90baa1eb05cb4c08501f8dac3665d395682
2020-04-24 12:42:35 +10:00
Ian Wienand
df3ad26f58 yum-minimal: strip env vars in chroot calls
This showed up with dnf in containers when TMPDIR was set; dnf started
trying to write to this directory while in the chroot.

We already do stripping like this in run_in_target -- but this is a
bit of a unique place because it's actually setting up the initial
chroot so the target doesn't actually exist yet; so we just hard-code
it in place here.

Change-Id: If7310cb820846da903bf60daa4486c8bf7cb0136
2020-04-24 12:38:42 +10:00
Ian Wienand
ac7fba7040 pip-and-virtualenv: drop f31 & tumbleweed, rework suse 15 install
This is an alternative approach to commit
68bb43535e.  I think this proposes a
better overall solution that the prior change which had the Python 3
packages being installed, but did not specify the _do_py3 flag to do
the installation steps that redirect the various tool installations.

Fedora 31+ doesn't have python2, and Tumbleweed does have some Python
2 support but there seems to be no reason to bother updating this
element for either with infra very close to removing this completely
[1].  Error out on these platforms, and add a release note.

The 15 path should include the python2 and python3 packages, along
with the flags to do the "cleanup"; i.e. forced removal of distutils
packages that pip 10+ won't touch.  As mentioned in the original
change, the six package causes problems here, but we can clear that
too by explicitly listing it instead of letting it come in via
dependencies.  Again, this element will be removed from the infra 15
builds ASAP; but we can release with this to provide a roll-back point
if we need to revert the removal to fix things temporarily.

Add it to the testing path as well.

[1] https://docs.opendev.org/opendev/infra-specs/latest/specs/cleanup-test-node-python.html

Change-Id: I7a6a342461d6001c25e55638ba9b7438c28f2519
2020-04-23 08:10:26 +10:00
Ian Wienand
3f823488c5 Add centos aarch64 tests
Change-Id: I2fffeeffa82c505a34c5c9972724b557f3b58975
2020-04-16 14:15:50 +10:00
Dirk Mueller
68bb43535e opensuse: fix python 2.x install
openSUSE Tumbleweed is dropping python2-* packages so we need
to stop intalling them. We can also stop installing those
for Leap 15. which avoids a pip uninstall issue (as python2-six
was still built with distribute).

Change-Id: Ie93c8addb26aab3d0154c4b5b52423799abede91
2020-04-03 18:22:33 +02:00
Ian Wienand
434d2db5d4 Debuntu: add apt-transport-https
I don't see anywhere we bring this in, especially on a minimal build.
In 2020 it seems like a base dependency, put it alongside
software-properties-common that installs the other apt helper bits.

Change-Id: I5b079eac4912cb4a164e9aa6158ed106a28f576c
2020-04-02 10:11:35 +11:00
Zuul
ac250e5ab0 Merge "centos 8 image build: fix mirror" 2020-03-31 05:03:06 +00:00
Zuul
dc67ccfe12 Merge "Mellanox element: removed ibutils,libibcm,libmlx4-dev" 2020-03-30 22:10:13 +00:00
Ian Wienand
56dc0a2c31 centos 8 image build: fix mirror
We're ending up with "centoscentos" in the mirror location and the
build fails; strip out the $contentdir from the original too.

Change-Id: If09dbbd8028ea510d2ab0d3d8afe484cea611df5
2020-03-31 08:57:15 +11:00
Zuul
21f9ea4f78 Merge "Add Fedora 31 support and test jobs" 2020-03-30 01:26:32 +00:00
Lee Yarwood
ee9ad32b6f Add Fedora 31 support and test jobs
Change-Id: Iad0261ac8db001ffa4d3a4dd6df05fe923402a69
2020-03-30 08:47:00 +11:00
Zuul
56d9956cbd Merge "Add python-stow-versions element" 2020-03-27 13:49:21 +00:00
Lenny Verkhovsky
2f4e1886e5 Mellanox element: removed ibutils,libibcm,libmlx4-dev
Those packages will be installed as part of dependencies

Change-Id: I640dc55c818dd8538bfcd199c33f9af93aae7608
2020-03-23 00:06:53 +02:00
Harald Jensås
1ac31afd62 Use rpm -e instead of dnf for cleaning old kernels
If the running kernel of the system building the image
matches the kernel that is to be removed dnf will fail.

Repalce use use of dnf with rpm -e.

Closes-Bug: #1623409
Change-Id: Ie2481ea8a02b7b0720e46fa179f24badf4aa25c5
2020-03-19 22:35:23 +01:00
Dmitriy Rabotyagov
1de6fe4ba7 Add python-stow-versions element
This element is designed to install latest minor versions of different
python releases, like py27, py35, py36, py37, py38
into stow directory, and later easily enable them with stow.

Change-Id: Iab6d20e7643e549b53c629fb430e58b1c5e72991
2020-03-19 21:21:01 +02:00
Monty Taylor
c113703050 Add support for build-only packages
Sometimes an element needs packages installed so that it can
perform tasks but those package are not appropriate for the
final image content. Add a "build-only" flag to package-install-squash
which will cause package to be installed at the beginning of the
phase and then uninstalled at the end of the phase.

Change-Id: Ie01b795991710c93f6b669c8f14b57eb4412c1d5
2020-03-18 14:40:44 -05:00
Zuul
61b72ca2c2 Merge "Add ensure-venv element, install glean with it" 2020-03-10 05:08:54 +00:00
Zuul
991586c20a Merge "Uncap hacking" 2020-03-10 01:16:02 +00:00
Ian Wienand
82cfcfe551 Add ensure-venv element, install glean with it
All the platforms we care about now have python3 with venv (even
centos7 now) packaged somehow.  Add an ensure-venv element to make
sure that "python3 -m venv" works.  Any other elements that wish to
install non-distribution-packaged Python utilities can use this to
keep them separate from the main system installs.

Port glean to use this, and drop its dependency on pip-and-virtualenv.

Change-Id: Ic16f134fe34293bb68e7c632dd320f523366320d
2020-03-10 11:57:43 +11:00
Zuul
18f46bde30 Merge "fix iscsi-boot element exiting build even if dracut-regenerate used" 2020-02-26 04:05:57 +00:00
Zuul
8deca122f9 Merge "Change tgt pkg-map to target-restore CentOS/RHEL-8" 2020-02-26 04:05:56 +00:00
Zuul
8ad2956911 Merge "Do not include efibootmgr and efivars for ppc architectures" 2020-02-26 04:02:42 +00:00
Bob Fournier
14a3776885 Do not include efibootmgr and efivars for ppc architectures
These don't exist and are not needed for ppc.

Change-Id: I9ba53e22583b148b43f74cd9b8ec79402796d09b
2020-02-25 13:27:13 -05:00
Chandan Kumar (raukadah)
31a32ff6ee Change tgt pkg-map to target-restore CentOS/RHEL-8
In ironic-agent tgt pkg-map, scsi-target-utils was used for
redhat family which is not used in CentOS/RHEL-8.

As per https://opendev.org/openstack/ironic-python-agent-builder/src/branch/master/dib/ironic-python-agent-ramdisk/pkg-map#L5
targetcli got removed and switch to target-restore. It syncs
the same element for RHEL-8 and adds the same for CentOS-8.

Closes-Bug: #1864427

Change-Id: I95a0c09a6739af23cd1e8c88dded198bd69cc53e
Signed-off-by: Chandan Kumar (raukadah) <chkumar@redhat.com>
2020-02-24 14:17:24 +05:30
Ian Wienand
28ebd24844 Uncap hacking
This causes problems for other projects incorporating dib; we don't
have a specific need for a cap.

Fix a few issues, mostly spacing or regex matches.  No functional
changes.

W503 and W504 relate to leaving artithmetic operators at the start or
end of lines, and are mutually exclusive and, due to "ignore"
overriding the defaults both get enabled.  It seems everyone gets this
wrong (https://gitlab.com/pycqa/flake8/issues/466).  Don't take a
position on this and ignore both.

Use double # around comments including YAML snippets using "# type: "
which now gets detected as PEP484/mypy type hints.

Change-Id: I8b7ce6dee02dcce31c82427a2441c931d136ef57
2020-02-24 10:34:46 +11:00
Carlos Goncalves
8226384cf0 Add CentOS 8 support
* Add "centos" element, a CentOS version-independent element. This is in
  line with the same work done for RHEL in Stein cycle.
* Deprecate the centos7 element. CentOS 7 support itself it not
  deprecated though. The new "centos" element provides the same support
  level as the "centos7" element.
* Add functional testing

The default CentOS version is 8. You can adjust it using the DIB_RELEASE
environment variable.

Change-Id: I373ba2296c4613765676e59aabd9c651345298d1
2020-02-19 10:44:56 +01:00
Zuul
9d08848f25 Merge "Fix cache-url -f" 2020-02-18 22:55:23 +00:00