The quickstart should be the first bit of developer documentation, not
the last. Also add in a short blurb for the developer docs so we dont
have two doc titles back to back.
Change-Id: Icb5683b8eb22e759fefb1cb2252ed445dea5f7dd
Being able to discover DIB's version from the command itself is
convenient. This patch adds a --version option to the disk-image-create
command, failing gracefully if diskimage-builder is not installed.
This adds an explicit dependency on pbr to the requirements since this
is required to run diskimage_builder/version.py outside of a test
environment.
This patch consciously chooses to only provide the long-form option
and no '-v' to allow for the future possibility that a '-v' might
indicate '--verbose' in the future.
Change-Id: I9fc084774d6c7a39a944b07680b3eb8be8e34f9c
This patch fixes the profile autodetect. If the profile was passed at install
time, it would overridden by the environment.d script. This was fixed by
accepting the value passed by the user, if one was passed.
The enviroment.d file is run outside of chroot, because of this eselect will
not be available on non-gentoo systems. We change the behavior to work
elsewhere, it was not working on non-gentoo systems and giving bad results on
gentoo systems (returing values from the host, not the guest being built).
This patch also fixes shm detection by checking if it is mounted already.
Change-Id: Ie58d8e19529a731bfbc9eeb4bb246988d1aaa772
Newer distros, such as debian jessie and ubuntu xenial, do not provide
ifupdown by default, but simple-init depends on it. Add it to the pile.
Change-Id: I6f4876863c67c65a82464d4e0593015cdc839c5c
This simplifies and enhances the functional-test runner script for
much better interactive behaviour and to give us the ability to better
choose what is running in CI.
Firstly, I have split the image-output testing into a separate script.
This is not actually part of the functional testing of elements and is
both logically and functionally different. It currently does not run
in upstream CI because we don't have docker in the images. I have
nothing against it, but it can be it's own thing.
run_functests.sh is overhauled to have a useful interactive interface,
e.g.
---
$ ./run_functests.sh -h
run_functests.sh [-h] [-l] <test> <test> ...
-h : show this help
-l : list available tests
<test> : functional test to run
Special test 'all' will run all tests
$ ./run_functests.sh -l
The available functional tests are:
apt-sources/test-sources
debian/build-succeeds
fedora/build-succeeds
fedora/build-succeeds-f21
ironic-agent/build-succeeds-fedora
---
As described there, you can run a single test, a number of tests, the
default tests (as CI will do) or all tests. Running all tests is too
much for regular CI, but currently the only way to stop a low priority
test running, or temporarily pause is to remove it completely --
clearly sub-optimal (see I93c2990472e88ab3e5ff14db56b4ff1b4dd965ef).
There is nothing complicated about this, and to further simplify I
have merged the runner functions back into run_functests.sh which
remains a very modest ~150 lines, with most of that being argument
sanity. With that and the image-format cleanup, we can remove the
indirection of the 3 small library files.
For consistency, I have renamed the "dib_functions_test" (that tests
things from the dib functions library) with a run_* prefix.
Because the default list is the same as the current functional tests
run, this does not modify the status-quo. I plan to modify this,
however, to run fedora-minimal & centos-minimal tests in a future
change, as these are required to be stable for openstack ci.
Documentation is updated, and a README.rst is added in the tests
directory for discoverability.
Change-Id: I86d208bd34ff09a29fdb916a4e7ef740c7f65af8
This is breaking deployments that use registration. I
believe os-refresh-config scripts don't source environment.d
files, so the variable is unset there.
Given that this a blocking issue for RHEL deployments, I'm
pushing a quick revert and we can come up with a proper fix
when this isn't stopping other work.
This reverts commit 71bd8b3a33.
Change-Id: I87504660352220d45f5445bb933edc1c28885fcb
This patch update the find_interface to lookup
for InfiniBand interface according to it's BOOTIF
Closes-Bug: #1532534
Change-Id: I21b91cfd10888ac036f6347a0a44cdca422830a6
In certain cases, with packages cached that need an upgrade,
when performing that action the system hangs waiting for
a prompt.
Add force-confdef option, that will let dpkg overwrite
configuration packages that were not modified. In combination
with already existing force-confold flag, it will allow
to avoid any conffile prompt.
Change-Id: Ifb177f9ac2c9ad29f8b92309c5b8cfe8e60a4e14
In order to avoid conflict with installing dependencies for IPA from
pypi and distribution packages this patch propose installing IPA in a
virtual environment.
Closes-Bug: #1506792
Change-Id: I6a4c6403813d86f4110e98449ddd874109101b9e
By default we create a tarball of any debootstrap rootfs we create. For
the majority of use cases this is a large performance hit for no
benefit. Lets make this an opt-in feature.
Change-Id: I58fc485aacacaa17243bf9ce760ed91256d1f182
python3 is a hard requirement of dnf so can't be removed [1]
grubby is also required for kernel installs on Fedora. For too much
detail see I1a6e45d04755515286b3d49f8280c16b527e2f48; but the kernel,
via dracut, now has this as a "recommends" due to people removing it
and making unbootable systems.
[1] http://logs.openstack.org/76/248976/2/check/gate-dib-dsvm-functests-devstack-f21/734c8bd/console.html
Change-Id: I5867ecd57834eece9477aa9ea4b8bdd70e238084
The debian-minimal element creates /etc/apt/sources.list solely with
the 'main' component. I need to add 'non-free' and 'contribs'. I tried
to pass them via DIB_DEBIAN_COMPONENTS but it is not recognized.
Make debian-minimal to honor DIB_DEBIAN_COMPONENTS. Note that it is
comma separated for 'debootstrap', so replace commas with spaces to fit
the needs of sources.list.
Example usage:
DIB_DEBIAN_COMPONENTS='main,non-free,contrib'
Will debootstrap with the three components then when debian-minimal is
realized pre install a sources.list that has:
deb http://example.org/debian jessie main contrib non-free
Change-Id: I1dca2e8ffd31044a6b441ccb277298601e62f67c
Gentoo provides eclean-dist via the gentoolkit package
eclean-dist isn't needed anyway was /usr/portage is removed anyway
Removed redundant package update in cleanup.
Change-Id: Icf4f9ed549b9a6d923448d927d7c42bcf8d6091b
This action has been reordered so cleanup can occur before copy to blockdev
occurs. Documentation has been added about the ordering of this element in
relation to cleanup actions as well.
Change-Id: I3f9334a3669ee588d7fa7129202c97fa22fdb050
While it does save 1MiB of space, it might also pull python packages
depending on it. E.g. it makes impossible to install python-hardware
on the IPA image for advanced introspection.
Change-Id: Iab80dde63e6de62a5e45dcf404b4f9f633e50ac3
We were getting some subtle issues in fedora-minimal builds that
turned out to be because /var/run was not a symlink to /run.
Upon further investigation, it turns out that yum is creating a
/var/run directory for it's pid file when it starts working in the
empty chroot (which I verified by stracing it)
---
5905 stat("/home/ubuntu/tmp/dib-tmp/image.Ac4VZZsl/mnt/var/run", 0x7ffddffa0330) = -1 ENOENT (No such file or directory)
5905 mkdir("/home/ubuntu/tmp/dib-tmp/image.Ac4VZZsl/mnt/var/run", 0755) = 0
5905 open("/home/ubuntu/tmp/dib-tmp/image.Ac4VZZsl/mnt/var/run/yum.pid", O_WRONLY|O_CREAT|O_EXCL, 0644) = 6
---
Because this happens *before* we install "filesystem" (the package),
we mess up it's symlinking.
To work-around this, pre-install the trio of base packages (setup,
basesystem, filesystem) with rpm from outside the chroot.
Change-Id: I411b6ec9d91d95d3a0f98e76853086af3b70abe8
The Ubuntu Xenial cloud server images set the mode of
/var/lib/apt/lists/partial to 700, so when mounted it's inaccessible to
an unprivileged user, resulting in an error:
find: `/tmp/image.aDQKdkRi/mnt/var/lib/apt/lists/partial': Permission denied
There's no reason an image should come with anything already in
/var/lib/apt/lists/partial, so just avoid trying to descend into that
directory when fixing the apt translations packages.
Change-Id: Id27f0166bfb09d67200f337a5ffff2f2037b7c1c
This is a slight refactor I found useful when debugging. The
udevadmin info query will be visbile in the output of "set -x" which
helps, and is the logs/journal.
We can also reduce some calls by keeping the value and just grepping
with a herefile.
This also does some error checking and bails out if it does not see
what it needs to continue.
Change-Id: I39c4d262f9c5ce53f6b83d95b1363a74834cf2c8
Tucked away in systemd-udev-settle.service is the following comment
# This service can dynamically be pulled-in by legacy services which
# cannot reliably cope with dynamic device configurations, and
# wrongfully expect a populated /dev during bootup.
The info that the growroot script is querying is populated via udev,
particularly the blkid bits of [1]. This creates a race-condition
where sometimes udev has been triggered and the rules have applied and
sometimes not. Obviously in the first case, the root disk is not
grown correctly.
systemd-udev-settle is mostly disabled on distros because it can cause
an increase in boot-time for systems with lots of disks; this is not
our situation so it makes basically no difference.
That said, I will investigate if some systemd people know even better
ways to do this (possibly the service should depend on block .device
targets in systemd, and then filter out and only apply to the root
disk?)
[1] https://github.com/systemd/systemd/blob/master/rules/60-persistent-storage.rules#L66
Change-Id: I453e3afcd953dfc29ab6c42ddc81e940cfa70ee0
A TODO was placed on the partitioning section of the vm element to
replace sfdisk with a saner (and less arcane) way of partitioning. It
suggested parted for replacement. This changeset should reproduce the
same disk label and partition layout as sfdisk, but with less ioctl
errors and version dependency. It will also ensure partition alignment.
Change-Id: I5d8d75131458b73bfb05f80f1bfa7e2970e004b3
We currently install pip from package in the simple-init element.
We should really allow users to select whether to install pip from
git or package.
Change-Id: Ia5e62b9635af90d81227274a1dd8f20474cdbf73
As described in the comment, there is a dnf equivalent of this command
that doesn't require us installing yum-utils (which drags in yum on
dnf-only systems such as f23)
This is a small consequence to this -- due to us not installing
yum-utils some installs will now be completely yum free. This causes
a breakage in ironic-agent 99-remove-extra-packages where we remove
the yum package. There is a long-standing bug/feature where missing
packages in a group of packages do not cause yum/dnf to exit with
failure, but uninstalling a single package will. Because we have made
the systems yum-free, the uninstall of yum can fail in this corner
case.
It has always been like this, so I'm in favour of the "ain't broke"
approach. To work-around this, I have just put yum into the existing
list of packages to be cleaned up. I have added a note to the yum
installer taking note of this behaviour for future reference.
Change-Id: I8bbdc07ccdb89a105b4fc70d5a215077c42fcd03
InfiniBand interface takes more time to bring up then
Ethernet interface. This patch just increase the retries
to 20 times, to make it work for InfiniBand as well.
Change-Id: I5c4842696207885552413ea2d053f2e90bd6803c
I noticed when running functional tests via a interactive shell they
would end up hanging; I tracked it down to waiting for input on this
tar command. We need to specify "-f" so it opens the file ...
---
| + tar -t /tmp/tmp.qid2Ygfu7v/image.tar
| + grep -q /tmp/dib-test-should-fail
| tar: This does not look like a tar archive
| tar: /tmp/tmp.qid2Ygfu7v/image.tar: Not found in archive
| tar: Exiting with failure status due to previous errors
---
I guess we never noticed because it's always running in CI where stdin
is closed and it just get that nothing read above. This probably was
reversing the logic of the "should have failed" tests ... but I don't
think we take that path currently for any tests.
Change-Id: Idad2c172797395d45c0d750ec687011cc1fbf52a
