Commit Graph

43 Commits

Author SHA1 Message Date
Jenkins
bbe81c30be Merge "Switch to openSUSE Leap 42.2 release by default" 2016-12-19 08:40:32 +00:00
Dirk Mueller
54f4e12765 Switch to openSUSE Leap 42.2 release by default
openSUSE 13.1 was discontinued on Feb 3rd, 2016, so defaulting
to it doesn't make sense (see https://en.opensuse.org/Lifetime).

Leap 42.2 is the most current release that is supported by
disk-image-builder and being tested in a 3rd party ci.

Enable functests for it to ensure we're not regressing again.
Moved to non-voting gate first.

Depends-On: Iff495b3cd0b6c3558c44cf4883651eca67b572d6
Change-Id: Iae6cd34a5853f1e309861c554d94d8595cbd9993
2016-12-17 10:46:17 +01:00
Gregory Haynes
c3293b3f7e Disable centos6 testing
Centos6 is no longer reasonable to expect to function with openstack,
and infra does not host a mirror for it.

Change-Id: I95ccf5840807fee73d6e78d596c82709e476bb3a
2016-12-14 11:10:54 +11:00
Ian Wienand
6fb658a5f1 Don't log datestamp by default in functional tests
We're getting double time-stamps in the console log of upstream jobs.
Move the logging of a prefix datestamp into a "-t" option to retain
the status quo prior to Id9ea5131f0026c292ca6453ba2c80fe12c47f808 (we
could, of course, do it the other way and turn if off in the jobs, but
since we didn't have it before...)

While poking, make the time-stamp consistent and always prefixed if -t
is turned on.

Also, it seems the parallel options got a bit of sync with what got
merged.  Add "-j" documentation and remove unused "p" option.

Change-Id: Ic7c2ebeca3f9d5784cac59505b6e6181151f5805
2016-10-24 11:21:43 +11:00
Jenkins
2e0f812efa Merge "Add option to be able to run_functests.sh in parallel" 2016-10-18 21:03:05 +00:00
Andreas Florath
20bb6a0255 Add option to be able to run_functests.sh in parallel
Running the functional tests is time consuming.  This patch adds the
option `-j <job count>` to the tests/run_functests.sh: when given the
test run in parallel up the <job count> jobs.
When using this, be sure to have enough resources (CPUs, RAM and HD
space) on the host.

In addition there was the need to change two things:
o Global /tmp/dib-test-should-fail was move to temporary build
  directory of each execution.
o Because the logs might now interleave, each log line has now a
  prefix of the name of the testcase.

[In my environment running functests sequential takes 15+ minutes,
running them parallel takes less than 6 minutes.]

Change-Id: Id9ea5131f0026c292ca6453ba2c80fe12c47f808
Signed-off-by: Andreas Florath <andreas@florath.net>
2016-09-03 16:51:55 +02:00
Gregory Haynes
a1bfca6022 Add tests for building *-minimal images
Lets make sure these images can be built.

Change-Id: Idbd07b98c0181738d002a53373425e056390beea
2016-08-22 16:53:32 +00:00
Andreas Florath
a8c8c61711 Fix packaging problems for Debian
This patch solves three issues with Debian packaging / apt:

o When building 'testing' only default apt sources is
  included - backports, updates and security are skipped because they
  do not exists.

o The default release for Debian was `unstable`: this is now fixed to
  `stable`.

o Starting a Debian Stretch VM that was build with diskimage-builder
  does not work, because some mandatory packages are missing.
  This patch fixes this problem: it adds the mandatory packages and
  the test case.

Change-Id: If49b5b162c4da1e074e9b19324839bc59d87dc57
Signed-off-by: Andreas Florath <andreas@florath.net>
2016-07-19 07:11:57 +02:00
Jenkins
8175c26774 Merge "Fail functests if refusing to run tests" 2016-06-29 21:38:30 +00:00
Jenkins
867bfaa44d Merge "Install docker for tests" 2016-06-29 21:23:41 +00:00
Gregory Haynes
feb4eda17a Fail functests if refusing to run tests
We made this non-failing so tests could pass before
I92299b2f59aa8e7de8995a6294bf6c88bbd4cdc5 merges. We should make this a
test failure.

Change-Id: I4dd38c835d6fb397bbd0475120a4b0f2ab0290a8
Depends-On: I92299b2f59aa8e7de8995a6294bf6c88bbd4cdc5
2016-05-13 17:07:32 +00:00
Gregory Haynes
edc06a20e5 Install docker for tests
We have some test cases which attempt to build docker images, therefore
we need docker.

Fix a few bugs that showed up when we run docker tests - we need to
docker rm with sudo and docker images don't always have a /tmp so check
before unmounting it.

Change-Id: I147d0ef3f2ea83f35bac568214573a6bde0b1967
2016-05-13 17:07:16 +00:00
Jenkins
4ad40f5387 Merge "Add test dependency installation on Gentoo" 2016-05-13 03:18:55 +00:00
Andreas Florath
0aa7f38a7f tests/elements/fake-os: add '/tmp' as top level dir
lib/img-functions:finalise_base() does a recursive unmount of /tmp to
cleanup any old mounts that might be in there (we do things like bind
mount caches in there).  unmount_dir fails if /tmp isn't a
directory.

The 'run_output_format_test.sh' uses the 'fake-os' element for
testing.  Because 'fake-os' did not create the '/tmp' directory, the
test script fails.

This patch fixes the problem: it adds '/tmp' to the list of
directories that are created.

It was implemented replacing the existing five 'mkdir'
command with a for loop over now six directories.

Change-Id: Ic7aca155492967b9819f129d3843cf95e136410e
Signed-off-by: Andreas Florath <andreas@florath.net>
2016-05-13 11:17:42 +10:00
Matthew Thode
d1b0fc16aa
Add test dependency installation on Gentoo
Here I add test deps for Gentoo, the main ones to call out are as follows.

* pyyaml: not installed by default and needed for the package element
* parted: not installed by default and needed for the VM element
* multipath-tools: not installed by default and provides kpartx

Change-Id: I07ce871fb3e684bfd9d06268d5c5bd118314f321
2016-04-30 20:56:01 -05:00
Ian Wienand
a7afe652d6 Fix disk usage report
This was not well tested.  Build the argument into a variable which
can be eval()ed to produce the final output.

Add the flag so we test this during functional tests.  Add "-x" to dib
invocations so we can more easily debug failures.

Change-Id: Ifdc82627c520379b4124ccb9a4c2fe806c52c75c
2016-04-08 07:07:00 +10:00
Ian Wienand
43e1e36cc6 Add qcow2 generation for better test coverage
Add qcow2 generation for better test coverage.  Add "-x" to the dib
invocations so we can better debug failure cases.

Change-Id: Idd0e33c70fcd7737e6dc43e26b054fbc2982c022
2016-04-07 15:25:34 +10:00
Ian Wienand
2764f2a659 Skip gentoo test
The idea was to put this in non-voting, but we never added it to the
skip list so it has been running by default.

Change-Id: I67f3453607077146ceb9430d12b4b9bfcd34437f
2016-04-07 15:13:40 +10:00
Matthew Thode
9707cf5282
Add testing for the Gentoo element
This commit adds testing for the gentoo element based off of the debian
element.  In adding testing support a couple of additional errors were
found and corrected, namely the following.

* shm is not in /proc/mounts though it is valid for use based off of
  the sticky flag.
* The path in ELEMENTS_PATH needs to be pruned to be usable.
* Added uuidgen-runtime to the list of packages installed for Ubuntu
  as the new ubuntu-trusty image does not ship with it.

Change-Id: I0d2768a912b350e1b2cf40d2fd227fdc767b1bcb
2016-03-17 00:13:32 -05:00
SamYaple
0c32784663 Revert "Zerofree the image if possible"
This reverts commit 5184d02a7c.

The decision was made to go with fstrim because it is faster and more
universal that zerofree. The related-id has the patchset that implements
fstrim.

Related-Id: I269b4416be450369616f9b8e030f84c30e329804
Change-Id: If40cf2fc0ecd8686768cbfeac9ecee90907674e7
2016-03-10 20:38:01 +00:00
Clark Boylan
5184d02a7c Zerofree the image if possible
If the image has an ext filesystem and the zerofree utility is present
on the build system then run zerofree. This should make images as
compressable as possible which is a nice feature when building
compressed qcow2 images.

Change-Id: Ia6062c291f7a3f58b85a4f408ecb3d0574c65d53
2016-03-07 17:45:14 -08:00
Ian Wienand
b18f71f781 Rework functional test runner
This simplifies and enhances the functional-test runner script for
much better interactive behaviour and to give us the ability to better
choose what is running in CI.

Firstly, I have split the image-output testing into a separate script.
This is not actually part of the functional testing of elements and is
both logically and functionally different.  It currently does not run
in upstream CI because we don't have docker in the images.  I have
nothing against it, but it can be it's own thing.

run_functests.sh is overhauled to have a useful interactive interface,
e.g.

---
 $ ./run_functests.sh -h
 run_functests.sh [-h] [-l] <test> <test> ...
   -h : show this help
   -l : list available tests
   <test> : functional test to run
            Special test 'all' will run all tests

 $ ./run_functests.sh -l
 The available functional tests are:

  apt-sources/test-sources
  debian/build-succeeds
  fedora/build-succeeds
  fedora/build-succeeds-f21
  ironic-agent/build-succeeds-fedora
---

As described there, you can run a single test, a number of tests, the
default tests (as CI will do) or all tests.  Running all tests is too
much for regular CI, but currently the only way to stop a low priority
test running, or temporarily pause is to remove it completely --
clearly sub-optimal (see I93c2990472e88ab3e5ff14db56b4ff1b4dd965ef).

There is nothing complicated about this, and to further simplify I
have merged the runner functions back into run_functests.sh which
remains a very modest ~150 lines, with most of that being argument
sanity.  With that and the image-format cleanup, we can remove the
indirection of the 3 small library files.

For consistency, I have renamed the "dib_functions_test" (that tests
things from the dib functions library) with a run_* prefix.

Because the default list is the same as the current functional tests
run, this does not modify the status-quo.  I plan to modify this,
however, to run fedora-minimal & centos-minimal tests in a future
change, as these are required to be stable for openstack ci.

Documentation is updated, and a README.rst is added in the tests
directory for discoverability.

Change-Id: I86d208bd34ff09a29fdb916a4e7ef740c7f65af8
2016-02-19 13:50:09 +11:00
Ian Wienand
f82cfb156a Fix tar listing in functional tests
I noticed when running functional tests via a interactive shell they
would end up hanging; I tracked it down to waiting for input on this
tar command.  We need to specify "-f" so it opens the file ...

---
 | + tar -t /tmp/tmp.qid2Ygfu7v/image.tar
 | + grep -q /tmp/dib-test-should-fail
 | tar: This does not look like a tar archive
 | tar: /tmp/tmp.qid2Ygfu7v/image.tar: Not found in archive
 | tar: Exiting with failure status due to previous errors
---

I guess we never noticed because it's always running in CI where stdin
is closed and it just get that nothing read above.  This probably was
reversing the logic of the "should have failed" tests ... but I don't
think we take that path currently for any tests.

Change-Id: Idad2c172797395d45c0d750ec687011cc1fbf52a
2016-02-04 15:50:25 +11:00
Jenkins
1733c5fb92 Merge "Don't use wc -l for the umount check" 2016-01-29 04:35:28 +00:00
Jenkins
fbd437fea6 Merge "Add inetutils-ping to test-deps" 2016-01-28 10:17:27 +00:00
Steve Kowalik
556535caaa Don't use wc -l for the umount check
We don't need the follow the pattern of checking wc -l against
/proc/mounts is 0, we care about any match, so we can just use
grep and it's exit code inside the if.

Change-Id: Idd0d78a613f8b43f169a320527c763d0bff74368
2016-01-28 19:24:10 +11:00
Ian Wienand
53c57d75c7 Cleanup unmount_dir function
Cleanup this function to work with a symlinked directory.  Document
it's behaviour more exactly, and add a simple unit-test for it (not
run by default, due to doing things like mounting and unmounting
system dirs on a live system, which doesn't seem safe for CI.  But it
is useful for developers ensuring sanity).

Change-Id: I335316019ef948758392b03e91f9869102a472b9
2016-01-28 14:47:43 +11:00
Gregory Haynes
c50e9cf26c Add inetutils-ping to test-deps
We require ping, and (amazingly) some hosts don't have ping - such as
ubuntu-minimal images.

Change-Id: I82b4ce15bffe6b164523a47151c01ef4ad190ab0
2016-01-19 02:04:34 +00:00
Jenkins
af5d3bd0c4 Merge "Make check for image-should-fail quiet" 2016-01-08 01:22:37 +00:00
Jenkins
c8c14edfe3 Merge "Support building ACIs" 2015-12-18 13:53:38 +00:00
Gregory Haynes
0537563c27 Make check for image-should-fail quiet
We perform a check for this file after an image build fails to see if it
was intentional. This outputs a message which looks like the failure is
a result of the check when the file is missing which is very confusing
to users. Lets be more quiet about it.

Change-Id: I9b1f6c7c57021d11b4f91cd3c33846fae692b928
2015-12-17 02:31:41 +00:00
Dmitry Tantsur
0e122e8e35 Add a tox target to run functional tests locally
Now 'tox -efunc' can be invoked to run all functional tests in
the 'venv' tox environment. Also `tox -efunc element-name` can be
used to run function tests for one element (e.g. ironic-agent).

Change-Id: Ia685d1b2a7deef2f8b98876ac09792134dd30f2f
2015-10-23 12:05:23 +02:00
Dmitry Tantsur
c49b733fef Fix tests/test_functions.bash
I am not sure why we didn't hit it before, but
1. The command is 'docker images', not 'docker image'
2. Calling 'docker rmi' requires a check on image presence

Change-Id: I127359043f975a25bb1bfad7469dfe01be67e2d9
2015-10-14 13:41:01 +02:00
Dmitry Tantsur
46dcaaedfc Add functional test for ironic-agent on Fedora
Added support for ramdisk-type elements in tests/test_functions.bash
Elements are distinguished by element-type file in a test element.
Note that ironic-agent ramdisk is built with disk-image-create.

Change-Id: I4759859e7f3c004c2d00e7318729602e6c3c4d95
2015-09-22 14:50:17 +02:00
Gregory Haynes
9e1cef01ec Fix test cleanup trap to cleanup tmpdir
We have a bug where we are overriding the tmpdir cleanup with the docker
image rm cleanup. We should be doing both.

Change-Id: I678c6248c1902095352a454618e3bdb9ed73630a
2015-07-22 03:57:09 -07:00
Gregory Haynes
eb6a260a5a Support building ACIs
App containers are a format used by rocket and are specified at the
following url:
https://github.com/appc/spec/

Change-Id: I8ac24f0194c4bf53dffd6c47e0587bc413101698
2015-07-22 09:30:09 +00:00
Monty Taylor
1187f9d5df Add docker output support
We can already produce tarballs, which is the input format docker import
expects. This makes it trivial to add docker as an output format.

Co-Authored-By: Dan Prince <dprince@redhat.com>

Change-Id: Ib60db3b717d33d4cf3181d70fe0ffbfa86fd5d02
2015-07-19 10:20:39 +00:00
Jenkins
6bf4631cfa Merge "Add debian build test case" 2015-06-17 21:10:23 +00:00
Gregory Haynes
2d6e977b2e Use tar -t instead of -l because centos 6
Centos 6 tar appears to only support unix style tar args. Use tar -t
when we inspect test builds.

Change-Id: I91bf321cf6700d59956779a60f613eddd2856024
2015-05-27 17:58:24 +00:00
Gregory Haynes
9a6cf7ee6f Add debian build test case
Weve had some regressions recently with the changes in the debian
element. Lets tests that we can build debian images.

Change-Id: I048e7a32ecb4088ec1b1e3b1efdf146187b093db
2015-05-19 19:23:53 +00:00
Gregory Haynes
b9b6640fa7 Initial element tests
Adding a test function which allows us to use elements to perform
element-specific tests. In order for this to work sanely, also adding
some configuration to our break system so we can assert on negative
tests.

Also adding a test for apt-sources to verify this code actually works.

Change-Id: I378a74255010eca192f5766b653f8a42404be5ea
2015-05-17 02:07:40 +00:00
Gregory Haynes
152975d258 Add install_test_deps script
This is useful for users and for CI so we do not have to chase infra for
changes when dependency issues arise.

Change-Id: I7c8bdaddfe316ae171b34164f99bc3d568eafd92
2015-05-01 21:33:55 +00:00
Gregory Haynes
8266e14adc Add functional smoke test for disk-image-create
We do not have any testing inside DIB for testing disk-image-create
logic. Lets do some smoke testing for all our supported image formats.

Also adding a run_functests.sh so we can extend this later without editing
the jenkins job.

Change-Id: Ie491e27f00bde54f73af6b47c9696ec04d973b14
2015-04-24 16:37:29 +00:00