Now that we are explicit about what python version we intend to use
for dib we can have package installs optionally install packages
depending on this. Add a new dib_python_version that matches on the
DIB_PYTHON_VERSION string set by dib-python.
Co-Authored-By: Adam Harwell <flux.adam@gmail.com>
Change-Id: I70659aab7d12924bdb9bc0489a7f02d5fd0dbb39
We currently have this as a 01- script which causes it to race with
package-installs (the deps are installed after the script runs).
Change-Id: I7b04b4c186eaae783b8e2bda1aa724c0d7823eab
systemd doesn't like it when service files have the executable bit
so this causes it to spam the journal with messages like:
Configuration file /usr/lib/systemd/system/dhcp-interface@.service is
marked executable. Please remove executable permission bits.
Proceeding anyway.
Removing the executable bit from the install permissions should
eliminate those messages.
Change-Id: Ie1bc39465b3fcb55dcda5cee9e46a128a6ccffcb
Right now dib-python works by trying to find any python on a system in
an order of precedence. A much better way is if we are explicit about
the python we intend to be there which will allow us to make better
decisions in other elements (such as allowing for package-installs to
take into account DIB_PYTHON_VERSION) as well as allow for users to
specify a preferred python version.
Co-Authored-By: Adam Harwell <flux.adam@gmail.com>
Change-Id: Ie609de51cc5fcde701296c9474e315981d9778a2
Most of our distros support DIB_DISTRIBUTION_MIRROR for setting the
mirror to be used when building. This has a major downside where if
someone would like to create a static configuration for building various
distro images it cannot set DIB_DISTRIBUTION_MIRROR (think of the case
where we want to run our tests).
By adding support for DIB_$DISTRO_DISTRIBUTION_MIRROR this problem is
resolved.
Change-Id: I92964b17ec3e47cf97e3a3091f054b2a205ac768
Adds conflict checking to the sysctl-write-value script
to detect settings from multiple elements conflicting.
Change-Id: If312d199388036d6f4103e94dca99249cb3bcbaf
Files in $element/environment.d are meant to be sourced, so drop
the executable bit. Moreover, drop the executable bit from a couple
of other scripts that are either meant to be sourced or simply because
they are configuration files.
Change-Id: I7f724dd9d409f4a835a136f12f48a84aa9acc41e
This element adds python-brick-cinderclient-ext to the make customized image
to support cinder local attach/detach functionality. Currently it has the
dependency on known bug<https://launchpad.net/bugs/1623549>, which would be
resolved with next release of python-brick-cinderclient-ext.
Change-Id: Idfe83bafa2843c781c18b83f1a3aece3ae852f78
Debootstrap only supports one apt repository to install packages from.
As a result, we do not consider the updates repo during debootstrap
causing us install a second kernel when we do an apt-get dist-upgrade
during build.
Lets use debootstrap to get us a minimal chroot, then add our repos and
install the correct packages from the start.
We also have to reorder the dpkg root.d scripts which configure apt so
they run before we perform our package installs.
Change-Id: I6a592db6f0a01d3b19d8e0786e63f1315a1ef647
Closes-Bug: #1637516
It's important to have the CA certificates on the target for ssl
crypto apps to work. Plus it's also important during bootstrapping
with diskimage-builder as tools like 'pip' etc need the certificates
in place in order to work properly. This fixes opensuse-minimal
image generation with the 'simple-init' element which was causing the
following error:
Download error on https://pypi.python.org/simple/: [SSL:
CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)
Change-Id: Ie94cd3556f8ae523f60ce0155ba18ed752e6fbb6
It seems in the grub cleanup in
Iafe3611f4eec3c6357587a6cae6a30a261686ead I managed to unintentionally
drop systemd from the yum-minimal builds. By not pre-installing grub
we dropped some dependencies; the path is tortured ... grub2 ->
os-prober -> udev -> systemd-udev -> systemd (we don't even want
os-prober! So this whole thing was working by accident).
This manifests in *very* confusing ways.
Currently centos-minimal builds are failing late in the build with
services unable to enabled. dib-init-system was actually trying to
tell us that it didn't know what init was installed (because systemd
wasn't actually installed), but unfortunately it was not really
failing. This meant the service files were not copied correctly from
other elements, and thus fail to be enabled. I have corrected this
with I076c08190d40c315ad6a6d96a3823e9fc52630be which would at least
alert us earlier.
For Fedora 24, due to a bug in dracut dependencies [1], missing the
systemd-udev package fails the build of the initrd during the kernel
install. This then results in an initrd-less, unbootable system (see
also Ibaaa81124098f3c6febe48e455d3e1cd0a5f1761).
Add these dependencies explicitly.
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1398505
Change-Id: I24ce648485c3d6f3c27ab8f87a638516b3727017
When you source a file that just does
export FOO=$(bar)
you miss any invalid return codes from "bar" (even under -e) because
bash returns the value of the "export", which is 0
On centos-minimal, we stopped bringing in systemd early and this was
causing dib-init-system to not know what init was available. Since it
did not fail correctly, it lead to confusing errors much later in the
build when service files were not copied correctly. See also
I24ce648485c3d6f3c27ab8f87a638516b3727017
A dib-lint check is added. One minor fixup is in 00-set-apt-sources
(this one is less likely to cause problems). I have run dib-lint over
project-config elements and none use this pattern.
Change-Id: I076c08190d40c315ad6a6d96a3823e9fc52630be
It seems that on Xenial, it does not take much to confuse "file" and
it's mime guessing such that it thinks some files are not python.
"package-installs-v2" is a good example, since it has an interpreter
"dib-python" that "file" doesn't know about, and no extension. While
looking at this, I've added emacs vars here so it opens in python
mode.
Change-Id: I01994b08c5ad8987925f1eec4062f5b6ee72eb8f
DIB_INIT_SYSTEM is exported by the dib-init-system element and contains
the output of the dib-init-system script so there is no need to
re-initialize it during various phases.
Change-Id: I09d6d10742689efe3d8eb9d64b539d6599b46227
Add new 'openssh-server' element to ensure that openssh server
is installed and enabled during boot. This is mostly useful for
*-minimal images which do not come with openssh installed and/or
enabled in order to keep a small dependency footprint.
Change-Id: Ide15ee04f5de123dbc8ce4bb56d638d8a167c341
This patch will configure cloud-init to allow password authentication.
This is usefull in case you use "devuser" element and want to ssh guest
image.
Change-Id: I00e38aa2753f26b4cdd34d0fd85fc8e0de78171f
SUSE packages the 'xml' python module as a separate package so make
sure it's pulled in before we attempt to install the pip module
since the latter depends on it. Fixes the following problem when
building with the opensuse-minimal and pip-and-virtualenv elements:
Traceback (most recent call last):
File "/tmp/get-pip.py", line 19177, in <module>
main()
File "/tmp/get-pip.py", line 194, in main
bootstrap(tmpdir=tmpdir)
File "/tmp/get-pip.py", line 82, in bootstrap
import pip
File "/tmp/tmpOiESjX/pip.zip/pip/__init__.py", line 16, in <module>
File "/tmp/tmpOiESjX/pip.zip/pip/vcs/subversion.py", line 9, in <module>
File "/tmp/tmpOiESjX/pip.zip/pip/index.py", line 32, in <module>
File "/tmp/tmpOiESjX/pip.zip/pip/_vendor/html5lib/__init__.py", line 16, in <module>
File "/tmp/tmpOiESjX/pip.zip/pip/_vendor/html5lib/html5parser.py", line 6, in <module>
File "/tmp/tmpOiESjX/pip.zip/pip/_vendor/html5lib/inputstream.py", line 10, in <module>
File "/tmp/tmpOiESjX/pip.zip/pip/_vendor/html5lib/utils.py", line 10, in <module>
ImportError: No module named xml.etree.ElementTree
Change-Id: I1bec12dfcde05fb07f41bcec994148c3eacbb287
The script is set -e and set -o pipefail, unfortauntely this intersects
with `yes n`'s non zero exit code behavior when it receives an interrupt
like sigpipe. As a result stop setting pipefail so that we treat those
errors as "normal" and only fail if ssh-keygen fails.
Change-Id: I5447df97c9888cae3007e235e2fea44df61af28e
After writing the basearch value to /etc/dnf/vars/basearch the
arch value was overwriting the same file. This appears to be
incorrect, so changing it to write /etc/dnf/vars/arch, which
matches the subsequent 'yum' code paths.
Change-Id: I5da54f03224c11f9e286f16b68533936c4174c2a
Add some checks for AArch64 to avoid the "Unknown architecture" or
"architecture not supported" messages, and allow builds to complete.
Change-Id: I89ba609abaeeb7019eb317cf13473929b2065230
This change was made for pre-install so it applies during the
image build, but wasn't applied to the os-refresh-config script
that would run after deployment. The same problems apply there,
so we should do the same thing.
Change-Id: I4b8534cc9586eeb588b5c358550e76e27d40556a
Closes-Bug: 1629922
for fedora/rhel/centos the main supported ARCH is x86_64. This patch allow
to call diskimage-builder with the above distro's with param ARCH=x86_64,
And also retain same behaiver when call with ARCH=amd64 as it translate
anyway to x86_64. Doing so wil simplify user expirience.
Change-Id: I229e0912434109b1b48a030bd35ad8dc1096a629
Without the dialog package is not possible
to properly use an interactive frontend.
debconf will print the following errors:
debconf: unable to initialize frontend: Dialog
debconf: (No usable dialog-like program is installed,
so the dialog based frontend cannot be used. at
/usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 76, <> line 1.)
Change-Id: I0c7142f717cacf7437dbac1e1696f39b00cb4c49
We have a pkg-map entry for lsb_release, but in package-installs.yaml
we refer to the actual package name instead. This will happen to
work on Red Hat platforms, but it's actually wrong.
Change-Id: Idb248f96e75fa1090422fa08e5fbb2385cc1f517
yumdownloader has to have all the repo XML files, etc, which adds up
to a not totally insignificant 150MiB or so. Currently we're leaking
this directory for every build, which adds up on regualar builders
like nodepool.
Isolate the call with a separate TMPDIR so we can clean it up after
the initial download.
Change-Id: Ic65e8ca837cc76b7a1bb9f83027b4a5bdd270f75
while using disk-image-builder for building overcloud images for TripleO
using RDO, this repository is (in my opinion) wrongly disabled because
contains certain dependencies needed by RDO packages.
Example: python-cheetah is required for python-nova, but is not
available through RDO repository but only from
rhel-7-server-rh-common-rpms
Closes-Bug: #1638938
Change-Id: I76824c8ec02590397f1ff1d4f177ad061c7bf441
Signed-off-by: Luca Lorenzetto <lorenzetto.luca@gmail.com>
Mount all the usual /dev /sys /proc pseudo filesystems during the
root.d phase in order to make sure they are available for the rpm
post-installation phases.
Change-Id: I28221debf1036d9eb5137161757eb30811eafab1
On Centos and RHEL 6 the init system is upsart but but networking is using
sysv compatabiliy and a code path the handle this situation.
We can't use DISTRO_NAME because the centos-minimal element sets it to
centos for CentOS 7 but the centos element sets it to centos for CentOS 6.
Change-Id: Ib8e33ed78b3d6a5737eb7449bccef2d33f72b131
Closes-Bug: #1638527
The refresh operation must happen after the cache has been added in
order to ensure that whatever is in the cache is still relevant to
the current build and we are not using stale packages.
Change-Id: Iafd718e9738f85b8c235806c027665730f44d89b
Closes-Bug: 1589450
Load the vfat driver as a Pre Exec action for systemd before starting
ironic-python-agent in order to allow reading of parameters.txt file
required for the ironic-python-agent to find its configuration.
Change-Id: Ibf74dd1b2678ea76e0676711a7aa5ba6b88d5421
AFAICT this is no longer necessary. I've tested minimal and image
builds and they seem to work.
The original problem seems to be with installing the package in the
chroot, although it was never quite clear it ever affected the Red Hat
path.
This code is currently broken (see
I884cb1e78ad8c31d985f3fc94a58091b993edd7d). This is proposed as an
alternative to I74eed074494134334d5e49042bb5214bd0dd7339.
Related-Bug: #1627000
Change-Id: Iafe3611f4eec3c6357587a6cae6a30a261686ead
Recommended packages are usually useful but we normally don't need
them in order to have a working system. As a result, avoid pulling
them in when doing a regular package installation or a distribution
update. Extra packages can be pulled in using the usual '-p' parameter
or from within the elements that actually need them. The results of
this change are quite significant, resulting to gains from a few dozen
of MBs up to a few hundred depending on the selected elements.
Change-Id: I5838829c631990c7a1f3b67548accd9a603fe20c
When debugging, this is very noisy for very little value. If we need
to specifically debug this script we can turn up the level.
Change-Id: Ie15f16397c37e718aa919853697cbf2c5c08503c
Because environment files are sourced into the current environment,
they shouldn't be setting global settings like tracing else they
affect every preceeding import. This is quite confusing when only
half your imports are traced in the logs, because it was either turned
on, or off, by a preceeding environment import.
There is a corresponding dib-run-parts change in
I29f7df1514aeb988222d1094e8269eddb485c2a0 that will greatly increase
debugability for environment files by deliberately logging what files
are sourced and consistently turning on tracing around their import.
This isn't strictly necessary (since dib-run-parts with the prior
change will just turn tracing off after import anyway) but it's a
decent cleanup for consistency. A bare-minimum dib-lint check is
added. Documentation is updated.
Change-Id: I10f68be0642835a04af7e5a2bc101502f61e5357
We are running into race conditions with glean, which ssh-keygen -A is
not handling properly. So, create a new script to first check if the
file exists, then use 'yes' to disable overwriting of existing files.
Change-Id: Ie82e1e3f832fcc8f32c7e1335c5f0ee16d36f9a8
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Theres a pretty standard workflow for setting a sysctl value which will
be applied on image boot which was written by tripleo. Lets move this in
tree as other folks (like Octavia) would like to depend on it.
Change-Id: I3c266870d417cdba3196f5fa65c4cd634ab13173
cloud-init-local needs to be run in the boot runlevel because it
modifies services in the default runlevel. When a runlevel is started
it is cached, so modifications that happen to the current runlevel while
you are in it are not acted upon.
Change-Id: Ifeae0071fc9e738ec223ec0df271559ad6e0196b
Add a new opensuse-minimal element to build small and highly
configurable openSUSE based images using the zypper-minimal element
as the main building mechanism
Change-Id: Iebfc4ad4aff763e511b093f1607b55851ccbddcb
All SUSE-based elements can benefit from the mkinitrd phase to move it
to a more generic location.
Change-Id: Ife171d462a393b6ac0bf2c5eaa48ea25eaf4d1cc
