Running the functional tests is time consuming. This patch adds the
option `-j <job count>` to the tests/run_functests.sh: when given the
test run in parallel up the <job count> jobs.
When using this, be sure to have enough resources (CPUs, RAM and HD
space) on the host.
In addition there was the need to change two things:
o Global /tmp/dib-test-should-fail was move to temporary build
directory of each execution.
o Because the logs might now interleave, each log line has now a
prefix of the name of the testcase.
[In my environment running functests sequential takes 15+ minutes,
running them parallel takes less than 6 minutes.]
Change-Id: Id9ea5131f0026c292ca6453ba2c80fe12c47f808
Signed-off-by: Andreas Florath <andreas@florath.net>
This patch solves three issues with Debian packaging / apt:
o When building 'testing' only default apt sources is
included - backports, updates and security are skipped because they
do not exists.
o The default release for Debian was `unstable`: this is now fixed to
`stable`.
o Starting a Debian Stretch VM that was build with diskimage-builder
does not work, because some mandatory packages are missing.
This patch fixes this problem: it adds the mandatory packages and
the test case.
Change-Id: If49b5b162c4da1e074e9b19324839bc59d87dc57
Signed-off-by: Andreas Florath <andreas@florath.net>
We made this non-failing so tests could pass before
I92299b2f59aa8e7de8995a6294bf6c88bbd4cdc5 merges. We should make this a
test failure.
Change-Id: I4dd38c835d6fb397bbd0475120a4b0f2ab0290a8
Depends-On: I92299b2f59aa8e7de8995a6294bf6c88bbd4cdc5
We have some test cases which attempt to build docker images, therefore
we need docker.
Fix a few bugs that showed up when we run docker tests - we need to
docker rm with sudo and docker images don't always have a /tmp so check
before unmounting it.
Change-Id: I147d0ef3f2ea83f35bac568214573a6bde0b1967
lib/img-functions:finalise_base() does a recursive unmount of /tmp to
cleanup any old mounts that might be in there (we do things like bind
mount caches in there). unmount_dir fails if /tmp isn't a
directory.
The 'run_output_format_test.sh' uses the 'fake-os' element for
testing. Because 'fake-os' did not create the '/tmp' directory, the
test script fails.
This patch fixes the problem: it adds '/tmp' to the list of
directories that are created.
It was implemented replacing the existing five 'mkdir'
command with a for loop over now six directories.
Change-Id: Ic7aca155492967b9819f129d3843cf95e136410e
Signed-off-by: Andreas Florath <andreas@florath.net>
Here I add test deps for Gentoo, the main ones to call out are as follows.
* pyyaml: not installed by default and needed for the package element
* parted: not installed by default and needed for the VM element
* multipath-tools: not installed by default and provides kpartx
Change-Id: I07ce871fb3e684bfd9d06268d5c5bd118314f321
This was not well tested. Build the argument into a variable which
can be eval()ed to produce the final output.
Add the flag so we test this during functional tests. Add "-x" to dib
invocations so we can more easily debug failures.
Change-Id: Ifdc82627c520379b4124ccb9a4c2fe806c52c75c
Add qcow2 generation for better test coverage. Add "-x" to the dib
invocations so we can better debug failure cases.
Change-Id: Idd0e33c70fcd7737e6dc43e26b054fbc2982c022
The idea was to put this in non-voting, but we never added it to the
skip list so it has been running by default.
Change-Id: I67f3453607077146ceb9430d12b4b9bfcd34437f
This commit adds testing for the gentoo element based off of the debian
element. In adding testing support a couple of additional errors were
found and corrected, namely the following.
* shm is not in /proc/mounts though it is valid for use based off of
the sticky flag.
* The path in ELEMENTS_PATH needs to be pruned to be usable.
* Added uuidgen-runtime to the list of packages installed for Ubuntu
as the new ubuntu-trusty image does not ship with it.
Change-Id: I0d2768a912b350e1b2cf40d2fd227fdc767b1bcb
This reverts commit 5184d02a7c.
The decision was made to go with fstrim because it is faster and more
universal that zerofree. The related-id has the patchset that implements
fstrim.
Related-Id: I269b4416be450369616f9b8e030f84c30e329804
Change-Id: If40cf2fc0ecd8686768cbfeac9ecee90907674e7
If the image has an ext filesystem and the zerofree utility is present
on the build system then run zerofree. This should make images as
compressable as possible which is a nice feature when building
compressed qcow2 images.
Change-Id: Ia6062c291f7a3f58b85a4f408ecb3d0574c65d53
This simplifies and enhances the functional-test runner script for
much better interactive behaviour and to give us the ability to better
choose what is running in CI.
Firstly, I have split the image-output testing into a separate script.
This is not actually part of the functional testing of elements and is
both logically and functionally different. It currently does not run
in upstream CI because we don't have docker in the images. I have
nothing against it, but it can be it's own thing.
run_functests.sh is overhauled to have a useful interactive interface,
e.g.
---
$ ./run_functests.sh -h
run_functests.sh [-h] [-l] <test> <test> ...
-h : show this help
-l : list available tests
<test> : functional test to run
Special test 'all' will run all tests
$ ./run_functests.sh -l
The available functional tests are:
apt-sources/test-sources
debian/build-succeeds
fedora/build-succeeds
fedora/build-succeeds-f21
ironic-agent/build-succeeds-fedora
---
As described there, you can run a single test, a number of tests, the
default tests (as CI will do) or all tests. Running all tests is too
much for regular CI, but currently the only way to stop a low priority
test running, or temporarily pause is to remove it completely --
clearly sub-optimal (see I93c2990472e88ab3e5ff14db56b4ff1b4dd965ef).
There is nothing complicated about this, and to further simplify I
have merged the runner functions back into run_functests.sh which
remains a very modest ~150 lines, with most of that being argument
sanity. With that and the image-format cleanup, we can remove the
indirection of the 3 small library files.
For consistency, I have renamed the "dib_functions_test" (that tests
things from the dib functions library) with a run_* prefix.
Because the default list is the same as the current functional tests
run, this does not modify the status-quo. I plan to modify this,
however, to run fedora-minimal & centos-minimal tests in a future
change, as these are required to be stable for openstack ci.
Documentation is updated, and a README.rst is added in the tests
directory for discoverability.
Change-Id: I86d208bd34ff09a29fdb916a4e7ef740c7f65af8
I noticed when running functional tests via a interactive shell they
would end up hanging; I tracked it down to waiting for input on this
tar command. We need to specify "-f" so it opens the file ...
---
| + tar -t /tmp/tmp.qid2Ygfu7v/image.tar
| + grep -q /tmp/dib-test-should-fail
| tar: This does not look like a tar archive
| tar: /tmp/tmp.qid2Ygfu7v/image.tar: Not found in archive
| tar: Exiting with failure status due to previous errors
---
I guess we never noticed because it's always running in CI where stdin
is closed and it just get that nothing read above. This probably was
reversing the logic of the "should have failed" tests ... but I don't
think we take that path currently for any tests.
Change-Id: Idad2c172797395d45c0d750ec687011cc1fbf52a
We don't need the follow the pattern of checking wc -l against
/proc/mounts is 0, we care about any match, so we can just use
grep and it's exit code inside the if.
Change-Id: Idd0d78a613f8b43f169a320527c763d0bff74368
Cleanup this function to work with a symlinked directory. Document
it's behaviour more exactly, and add a simple unit-test for it (not
run by default, due to doing things like mounting and unmounting
system dirs on a live system, which doesn't seem safe for CI. But it
is useful for developers ensuring sanity).
Change-Id: I335316019ef948758392b03e91f9869102a472b9
We perform a check for this file after an image build fails to see if it
was intentional. This outputs a message which looks like the failure is
a result of the check when the file is missing which is very confusing
to users. Lets be more quiet about it.
Change-Id: I9b1f6c7c57021d11b4f91cd3c33846fae692b928
Now 'tox -efunc' can be invoked to run all functional tests in
the 'venv' tox environment. Also `tox -efunc element-name` can be
used to run function tests for one element (e.g. ironic-agent).
Change-Id: Ia685d1b2a7deef2f8b98876ac09792134dd30f2f
I am not sure why we didn't hit it before, but
1. The command is 'docker images', not 'docker image'
2. Calling 'docker rmi' requires a check on image presence
Change-Id: I127359043f975a25bb1bfad7469dfe01be67e2d9
Added support for ramdisk-type elements in tests/test_functions.bash
Elements are distinguished by element-type file in a test element.
Note that ironic-agent ramdisk is built with disk-image-create.
Change-Id: I4759859e7f3c004c2d00e7318729602e6c3c4d95
We have a bug where we are overriding the tmpdir cleanup with the docker
image rm cleanup. We should be doing both.
Change-Id: I678c6248c1902095352a454618e3bdb9ed73630a
App containers are a format used by rocket and are specified at the
following url:
https://github.com/appc/spec/
Change-Id: I8ac24f0194c4bf53dffd6c47e0587bc413101698
We can already produce tarballs, which is the input format docker import
expects. This makes it trivial to add docker as an output format.
Co-Authored-By: Dan Prince <dprince@redhat.com>
Change-Id: Ib60db3b717d33d4cf3181d70fe0ffbfa86fd5d02
Weve had some regressions recently with the changes in the debian
element. Lets tests that we can build debian images.
Change-Id: I048e7a32ecb4088ec1b1e3b1efdf146187b093db
Adding a test function which allows us to use elements to perform
element-specific tests. In order for this to work sanely, also adding
some configuration to our break system so we can assert on negative
tests.
Also adding a test for apt-sources to verify this code actually works.
Change-Id: I378a74255010eca192f5766b653f8a42404be5ea
This is useful for users and for CI so we do not have to chase infra for
changes when dependency issues arise.
Change-Id: I7c8bdaddfe316ae171b34164f99bc3d568eafd92
We do not have any testing inside DIB for testing disk-image-create
logic. Lets do some smoke testing for all our supported image formats.
Also adding a run_functests.sh so we can extend this later without editing
the jenkins job.
Change-Id: Ie491e27f00bde54f73af6b47c9696ec04d973b14
