During the creation of a disk image (e.g. for a VM), there is the need
to create, setup, configure and afterwards detach some kind of storage
where the newly installed OS can be copied to or directly installed
in.
This patch implements partitioning handling.
Change-Id: I0ca6a4ae3a2684d473b44e5f332ee4225ee30f8c
Signed-off-by: Andreas Florath <andreas@florath.net>
expand_dependencies() was a public interface so we should try and
preserve backwards compat. However, since the interface is really
broken, add a new exported function "get_elements" that instack can
switch to. This returns the canonical list of elements without
duplicates, and gives the path to each element too.
This highlighted that the unit tests were really a bit wrong. They're
testing inner functions when we have an "API" in the get_elements()
function. Convert all unit-tests to use this function instead. Since
this is a library call, convert the sys.exit() calls to raised
exceptions.
Refactor the variable output into a separate function so we can do a
sanity check on it.
The added flake8 ignores are for the "over-indented for ... indent"
which happens a lot with these new longer lines. Most other projects
ignore them.
This is an alternative proposal to
I15609389c18adf3017220fc94552514d195b323a
Change-Id: If97bcd21e45de1b5ed91023fdc441a4617051a6b
We do not have the concept of "not installed" in v2, so remove the
obsolete code looking for a now non-existent variable.
Also, log the version at startup. This can help when
debugging from logs
Change-Id: I964c4cf207c10666afc5bc7ab9f2bfb9b1897c1e
Debootstrap only supports one apt repository to install packages from.
As a result, we do not consider the updates repo during debootstrap
causing us install a second kernel when we do an apt-get dist-upgrade
during build.
Lets use debootstrap to get us a minimal chroot, then add our repos and
install the correct packages from the start.
We also have to reorder the dpkg root.d scripts which configure apt so
they run before we perform our package installs.
Change-Id: I6a592db6f0a01d3b19d8e0786e63f1315a1ef647
Closes-Bug: #1637516
Do md5 and sha256 in parallel to speed things up for larger images.
Change-Id: Ib782fe54e4286ba2749a7ab7247f5d41a887a370
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Because we're still fundamentally a python program calling a
shell-script, there's some oddities like not having the virtualenv
bin/ in the $PATH if we call disk-image-create directly.
We can detect this, however, and activate the virtualenv before we
fork the disk-image-create shell script so everything "just works".
See also nodepool change I0537cbf167bb18edf26f84ac269cbd9c8a1ea6a2
Change-Id: Ibfea6cf6a6fd0c7f1e468d501c61ae0b58992042
It's important to have the CA certificates on the target for ssl
crypto apps to work. Plus it's also important during bootstrapping
with diskimage-builder as tools like 'pip' etc need the certificates
in place in order to work properly. This fixes opensuse-minimal
image generation with the 'simple-init' element which was causing the
following error:
Download error on https://pypi.python.org/simple/: [SSL:
CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)
Change-Id: Ie94cd3556f8ae523f60ce0155ba18ed752e6fbb6
It seems in the grub cleanup in
Iafe3611f4eec3c6357587a6cae6a30a261686ead I managed to unintentionally
drop systemd from the yum-minimal builds. By not pre-installing grub
we dropped some dependencies; the path is tortured ... grub2 ->
os-prober -> udev -> systemd-udev -> systemd (we don't even want
os-prober! So this whole thing was working by accident).
This manifests in *very* confusing ways.
Currently centos-minimal builds are failing late in the build with
services unable to enabled. dib-init-system was actually trying to
tell us that it didn't know what init was installed (because systemd
wasn't actually installed), but unfortunately it was not really
failing. This meant the service files were not copied correctly from
other elements, and thus fail to be enabled. I have corrected this
with I076c08190d40c315ad6a6d96a3823e9fc52630be which would at least
alert us earlier.
For Fedora 24, due to a bug in dracut dependencies [1], missing the
systemd-udev package fails the build of the initrd during the kernel
install. This then results in an initrd-less, unbootable system (see
also Ibaaa81124098f3c6febe48e455d3e1cd0a5f1761).
Add these dependencies explicitly.
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1398505
Change-Id: I24ce648485c3d6f3c27ab8f87a638516b3727017
It seems that on Xenial, it does not take much to confuse "file" and
it's mime guessing such that it thinks some files are not python.
"package-installs-v2" is a good example, since it has an interpreter
"dib-python" that "file" doesn't know about, and no extension. While
looking at this, I've added emacs vars here so it opens in python
mode.
Change-Id: I01994b08c5ad8987925f1eec4062f5b6ee72eb8f
Add new 'openssh-server' element to ensure that openssh server
is installed and enabled during boot. This is mostly useful for
*-minimal images which do not come with openssh installed and/or
enabled in order to keep a small dependency footprint.
Change-Id: Ide15ee04f5de123dbc8ce4bb56d638d8a167c341
SUSE packages the 'xml' python module as a separate package so make
sure it's pulled in before we attempt to install the pip module
since the latter depends on it. Fixes the following problem when
building with the opensuse-minimal and pip-and-virtualenv elements:
Traceback (most recent call last):
File "/tmp/get-pip.py", line 19177, in <module>
main()
File "/tmp/get-pip.py", line 194, in main
bootstrap(tmpdir=tmpdir)
File "/tmp/get-pip.py", line 82, in bootstrap
import pip
File "/tmp/tmpOiESjX/pip.zip/pip/__init__.py", line 16, in <module>
File "/tmp/tmpOiESjX/pip.zip/pip/vcs/subversion.py", line 9, in <module>
File "/tmp/tmpOiESjX/pip.zip/pip/index.py", line 32, in <module>
File "/tmp/tmpOiESjX/pip.zip/pip/_vendor/html5lib/__init__.py", line 16, in <module>
File "/tmp/tmpOiESjX/pip.zip/pip/_vendor/html5lib/html5parser.py", line 6, in <module>
File "/tmp/tmpOiESjX/pip.zip/pip/_vendor/html5lib/inputstream.py", line 10, in <module>
File "/tmp/tmpOiESjX/pip.zip/pip/_vendor/html5lib/utils.py", line 10, in <module>
ImportError: No module named xml.etree.ElementTree
Change-Id: I1bec12dfcde05fb07f41bcec994148c3eacbb287
The script is set -e and set -o pipefail, unfortauntely this intersects
with `yes n`'s non zero exit code behavior when it receives an interrupt
like sigpipe. As a result stop setting pipefail so that we treat those
errors as "normal" and only fail if ssh-keygen fails.
Change-Id: I5447df97c9888cae3007e235e2fea44df61af28e
In the error case, we get a spew of output as this check goes though
every pid checking if its in the chroot. Disable tracing around the
call.
Change-Id: Ie84f12974755c0c2c51d7e7697337ed9b32a4a1c
After writing the basearch value to /etc/dnf/vars/basearch the
arch value was overwriting the same file. This appears to be
incorrect, so changing it to write /etc/dnf/vars/arch, which
matches the subsequent 'yum' code paths.
Change-Id: I5da54f03224c11f9e286f16b68533936c4174c2a
Add some checks for AArch64 to avoid the "Unknown architecture" or
"architecture not supported" messages, and allow builds to complete.
Change-Id: I89ba609abaeeb7019eb317cf13473929b2065230
This change was made for pre-install so it applies during the
image build, but wasn't applied to the os-refresh-config script
that would run after deployment. The same problems apply there,
so we should do the same thing.
Change-Id: I4b8534cc9586eeb588b5c358550e76e27d40556a
Closes-Bug: 1629922
It has been observed that some chroot operations spawn additional
processes which rely on chroot files. More specifically, zypper, uses
gpg-agent to import and validate gpg keys for its repositories. This
gpg-agent process may stay alive for longer which prevents unmounting of
the tmpfs directory since the gpg-agent process still uses libraries etc
which were present in the chroot. We try to solve this by using walking
all the pids in /proc to find out the running processes in the chroot and
kill them gracefully. If that fails for whatever reason, then we simply
keep trying to umount the tmpfs directory before we give up.
The gpg-agent process usually terminates soon after its home directory
disappears but on fast systems we can reach the 'umount tmpfs' point
before gpg-agent terminates by itself. The solution is generic enough so
other 'chroot processes' can also be handled appropriately.
Change-Id: Iccf332678c79266113e76f062884fc5ee79e515d
for fedora/rhel/centos the main supported ARCH is x86_64. This patch allow
to call diskimage-builder with the above distro's with param ARCH=x86_64,
And also retain same behaiver when call with ARCH=amd64 as it translate
anyway to x86_64. Doing so wil simplify user expirience.
Change-Id: I229e0912434109b1b48a030bd35ad8dc1096a629
Without the dialog package is not possible
to properly use an interactive frontend.
debconf will print the following errors:
debconf: unable to initialize frontend: Dialog
debconf: (No usable dialog-like program is installed,
so the dialog based frontend cannot be used. at
/usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 76, <> line 1.)
Change-Id: I0c7142f717cacf7437dbac1e1696f39b00cb4c49
