Summary:
we have a long-standing problem with all the tests that hit
the repositories. The tests are triggered as soon as a compose
completes. At this point in time, the compose is not synced to
the mirrors, where the default 'fedora' repo definition looks;
the sync happens after the compose completes, and there is also
a metadata sync step that must happen after *that* before any
operation that uses the 'fedora' repository definition will
actually use the packages from the new compose. Thus all net
install tests and tests that installed packages have been
effectively testing the previous compose, not the current one.
We have some thoughts about how to fix this 'properly' (such
that the openQA tests wouldn't have to do anything special,
but their 'fedora' repository would somehow reflect the compose
under test), but none of them is in place right now or likely
to happen in the short term, so in the mean time this should
deal with most of the issues. With this change, everything but
the default_install tests for the netinst images should use
the compose-under-test's Everything tree instead of the 'fedora'
repository, and thus should install and test the correct
packages.
This relies on a corresponding change to openqa_fedora_tools
to set the LOCATION openQA setting (which is simply the base
location of the compose under test).
Test Plan:
Do a full test run, check (as far as you can) tests run sensibly
and use appropriate repositories.
Reviewers: jskladan, garretraziel
Reviewed By: garretraziel
Subscribers: tflink
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D989
Summary:
Set up the support server to provide DHCP/DNS functionality and
an NFS server, providing a kickstart. Add a kickstart test just
like the other root-user-crypted-net kickstart tests except it
gets the kickstart from the support server via NFS. Also add NFS
repository tests and a second support server for Server-dvd-iso
flavor: this test must run on that flavor to ensure that packages
are actually available. The support server just mounts the
attached 'DVD' and exports it via NFS.
Note we don't need to do anything clever to avoid IP conflicts
between the two support servers, because os-autoinst-openvswitch
ensures each worker group is on its own VLAN.
As part of adding the NFS repo tests, I did a bit of cleanup,
moving little things we were repeating a lot into anacondatest,
and sharing the 'check if the repo was used' logic between all
the tests (by making it into a test step that's loaded for all
of them). I also simplified the 'was repo used' checks a bit,
it seems silly to run a 'grep' command inside the VM then have
os-autoinst do a grep on the output (which is effectively what
we were doing before), instead we'll just use a single grep
within the VM, and clean up the messy quoting/escaping a bit.
Test Plan:
Run all tests - at least all repository tests - and
check they work (make sure the tests are actually still sane,
not just that they pass). I've done runs of all the repo tests
and they look good to me, but please double-check. I'm currently
re-running the whole 24-20160609.n.0 test on staging with these
changes.
Reviewers: jskladan, garretraziel
Reviewed By: garretraziel
Subscribers: tflink
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D888
Summary:
this is following a SUSE model for tests where we need a server
end but don't want setting up the server to constitute a real
test in itself, we want it to be stable. The 'support_server'
test just boots a pre-built (by createhdds) disk image, sets up
networking, and runs the iSCSI server.
To run the iSCSI test we need to handle networking config in
anaconda (or we would need to set the support server up as a
DHCP server, which may be worth considering), so this adds that.
We also need to be able to specify the target device for a
volume in custom partitioning, so this adds that too.
Test Plan:
Build the necessary support server disk image (use
D883), then run the test and make sure it works. Also make sure
all other tests continue to work.
Reviewers: jskladan, garretraziel
Reviewed By: garretraziel
Subscribers: tflink
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D884
Summary:
the Server DVD now just has 'Fedora Server' and 'Custom
Operating System' environments. Custom is basically minimal.
So we can use the DVD for 'universal' testing again, these
needles match the anaconda_minimal tags.
Test Plan:
Run the 'universal' tests on a DVD ISO with these
needles added, test that they work OK and use the 'Custom' env.
Reviewers: jskladan, garretraziel
Reviewed By: garretraziel
Subscribers: tflink
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D844
Summary:
Requires new needles and test suite and job template, plus a
few tweaks to handle 'switched' keyboard layouts (so we use the
switched layout in the username and password).
Test Plan:
Run the test and see that it...fails. But that's OK!
It's a genuine bug: RHBZ #1333998 . At least make sure it gets
to that point and no other tests have broken and all the needles
look sane.
Reviewers: garretraziel
Reviewed By: garretraziel
Subscribers: tflink
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D846
Summary:
This requires a PR I sent upstream:
https://github.com/os-autoinst/os-autoinst/pull/490
This change is in os-autoinst -10. However, with older packages
it won't crash or anything, it'll just behave as before.
With the change, this allows log upload to fail, so if one of
the logs is missing, the hook doesn't immediately die and fail
to upload the rest of the logs. Various anaconda logs are not
always present: the DNF logs are not present for Atomic or live
installs, and the X.log and syslog are not present for live
installs. Adding a fail-tolerant mode to upstream upload_logs
seemed a better option than testing for the existence of each
log file prior to uploading it, or adding a bunch of GET_VAR
calls to try and figure out which log files 'should' exist.
Test Plan:
Run an Atomic or live install test that fails, and
check what logs get uploaded. You can just test a current
Rawhide Atomic installer ISO, as they're crashing right now.
Without this patch (and the os-autoinst update) the hook dies
when it tries to upload dnf.log, so the traceback and /var/tmp
archive don't get uploaded; with this patch all the present
logs should get uploaded. Compare:
https://openqa.fedoraproject.org/tests/14834https://openqa.stg.fedoraproject.org/tests/15371
(I tested this out on staging).
Reviewers: jskladan, garretraziel
Reviewed By: garretraziel
Subscribers: tflink
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D834
Summary:
First off, this revises the anaconda crash handling needles a
bit. We ditch gtk3195 and update anaconda_error to reflect
current F24/Rawhide. We keep the old anaconda_error around for
now as anaconda_error-23, to handle crashes in the F23 two-week
Atomic nightlies. We also add an 'early' variant, which is for
when (I think) the installer crashes very early, before it's
loaded in GTK+ settings; when that happens, the dialog uses a
different font. The screenshot comes from a recent Rawhide test
that crashed.
We also restore the anaconda `post_fail_hook` code to click
the Report button when a crash happens. This was erroneously
removed in D637. Before the Report button is clicked, the
`anaconda-tb` file exists but the libreport stuff in `/var/tmp`
does not. By removing this, we lost the libreport bits from
the uploaded files, which makes it harder to report crashes. So
let's add it back.
Finally we fix the actual tarring and uploading of `/var/tmp`;
also in D637 this got broken because it was being tarred up in
whatever directory the commands happened to be running in, but
we were still trying to upload it from `/var/tmp`.
https://openqa.stg.fedoraproject.org/tests/8444 was run with
these changes, and has `/var/tmp` correctly uploaded.
Test Plan:
Run some test that crashes, make sure the crash
handling all works correctly.
Reviewers: jskladan, garretraziel
Reviewed By: garretraziel
Subscribers: tflink
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D768
Summary:
Apparently it'll be something like anaconda.core.(PID). This
should result in that getting compressed and uploaded only if it
exists.
Test Plan:
Try and hit the Mystery Crasher and see if we get the
damn core file this time.
Reviewers: jskladan, garretraziel
Reviewed By: garretraziel
Subscribers: tflink
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D688
Summary:
When anaconda manages to actually crash the python interpreter,
there should be a /tmp/anaconda.core containing the core dump
(per clumens, see calls to 'gcore' in isys.c). Let's upload it.
This might help us track down the mysterious occasional crashes
openQA seems to trigger (RHBZ #1289704)
Test Plan:
Try and trigger a python crash and see if the
file got uploaded. Of course, I did an entire freaking run on
staging and for *ONCE* not one test hit the mysterious crash,
thanks Murphy. I did at least check that this doesn't break
a 'normal' failure, if the file isn't there things don't
explode.
Reviewers: jskladan, garretraziel
Reviewed By: garretraziel
Subscribers: tflink
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D686
Summary:
since we did this live at Flock today, I figured I'd tidy it
up and submit it. This is an 'optional' test, but some people
do run this way so it'd be nice to have it. This adds another
little helper method in anacondatest.pm, for deleting partitions,
which works much like the others added in previous commits.
Test Plan: Schedule a test run, see if the test runs and works.
Reviewers: jskladan, garretraziel
Reviewed By: jskladan, garretraziel
Subscribers: tflink
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D503
Summary:
This adds three new custom storage tests and some needles to
support them, and tweaks the custom storage methods a bit to
address some things that cropped up in writing the tests. A
new method is added for changing the filesystem, as that's
a distinct operation from changing the device type.
This also restores the previous behaviour of select_disks()
where it handled selecting custom partitioning when needed.
Turns out it's pretty common to use regex'es in perl! Who'd'a
thought.
A corresponding commit to add the tests to openqa_fedora_tools
is coming.
There's no post-install step for the tests yet; I'll try and
write those up and add them soon.
Test Plan:
Do a full run, including the new tests, on Alpha RC2 and check
all are scheduled correctly and run correctly. The LVM thinp
test is expected to fail as it catches a genuine bug.
Reviewers: jskladan, garretraziel
Reviewed By: garretraziel
Subscribers: tflink
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D490
