Commit Graph

1591 Commits

Author SHA1 Message Date
Jan Sedlák
ea5296b306 add postinstalls to custom partitioning tests
Differential Revision: https://phab.qa.fedoraproject.org/D1192
2017-05-09 09:11:34 +02:00
Jan Sedlák
5e2a76efde fix blivet so that correct unit for size is selected
Differential Revision: https://phab.qa.fedoraproject.org/D1193
2017-05-05 09:29:55 +02:00
Adam Williamson
c2b1ea4e8c Use test3 not test1 for FreeIPA password change test module
Previously this module changed test1's password, so that it
would still be able to work even if the webui test module
failed (so test3/test4 didn't get created). But this means
that, for about 30 seconds, test1's password is 'loremipsum'
not 'batterystaple', and if one of the *other* client test
jobs happens to hit a point where it has to auth as test1
during the 30 seconds test1's password is different, it will
fail. This looks to be what happened to the join_kickstart
test the last few days - it failed because it tried to login
as test1 during the password change window.

By using the test3 user instead (which is only used by the
join_cockpit test, currently) we avoid this problem, at the
cost that the password_change module will always fail if the
webui module fails.
2017-05-04 17:29:35 -07:00
Adam Williamson
7d07147b7e Work around F26 base image bug: g-i-s failing to run
With the latest F26 base images, it seems like g-i-s fails to
run at first login. This is clearly some kind of bug somewhere
and I'll investigate it, but it shouldn't be causing the update
tests to fail - we can still validly run the tests with g-i-s
not running. So for now, adjust the _graphical_wait_login test
to tolerate this behaviour when running update tests.
2017-04-24 18:49:25 -07:00
Adam Williamson
1ac1977c2d Firefox addon interface changed again, add variant needles
We can probably throw the other needles away once Firefox 52
makes it to all release's stable repos.
2017-04-24 17:54:04 -07:00
Jan Sedlák
140c5f0a42 Add custom partitioning tests for blivet
Differential Revision: https://phab.qa.fedoraproject.org/D1188
2017-04-24 14:23:35 +02:00
Adam Williamson
22e210aa21 Run Atomic Host installer test on UEFI as well as BIOS
Atomic team asked for this, so hey, why not.
2017-04-18 16:48:37 -07:00
Adam Williamson
e6076e14b8 Workaround issues with loading same module more than once
Summary:
Loading the same module more than once *kinda* works, but it
shows up all kinds of funky in the openQA web interface. There's
a POO for this:
https://progress.opensuse.org/issues/10514
But it doesn't seem like it's going to be resolved immediately,
so in the mean time maybe we should avoid doing it so we don't
have to deal with the weirdness it produces in the web UI. So
here's a kinda icky hack that uses symlinks and stuff to load
multiple instances of 'the same' test module.

Test Plan:
Run an update test, look at how it looks in the web
UI and confirm it's a lot clearer and less buggy. Check there
aren't any bugs in the loading approach. This is deployed on stg
so you can look at it there.

Reviewers: jsedlak, jskladan

Reviewed By: jsedlak

Subscribers: tflink

Differential Revision: https://phab.qa.fedoraproject.org/D1186
2017-04-18 10:38:16 -07:00
Adam Williamson
9da72de4ed Hide mouse on login screen (BGO #781294)
GDM 3.24.1 dumps a cursor in the middle of the login screen even
if you didn't move the mouse yet, so let's workaround that.
2017-04-13 15:28:37 -07:00
Adam Williamson
ee283b1411 Tweak _console_avc_crash so we get soft failures
We do want soft failures (but not hard) when _console_avc_crash
fails. So let's tweak it to ensure that's what happens.
2017-04-10 15:44:13 -07:00
Adam Williamson
e68e113f76 Remove test_flags comments, add ignore_failure flag
It's not really a good idea to have the comments that explain
the test_flags in *every* test, because they can go stale and
then we either have to live with them being old or update them
all. Like, now. So let's just take 'em all out. There's always
a reference in the openQA and os-autoinst docs, and those get
updated faster.

More importantly, add the new `ignore_failure` flag to relevant
tests - all the tests that don't have the 'important' or
'fatal' flag at present. Upstream killed the 'important' flag
(making all tests 'important' by default), I got it replaced
with the 'ignore_failure' flag, we now need to explicitly mark
all modules we want the 'ignore_failure' behaviour for.
2017-04-10 15:00:10 -07:00
Adam Williamson
5b4ce80487 Tweak needle matches at start of _do_install_and_reboot
The way this was set up before, if `anaconda_main_hub` matched
immediately but some spoke was still in a 'processing' state,
it only had 30 seconds (default `assert_and_click` timeout) to
complete and allow the 'Begin Installation' button to appear.
It seems unnecessary to match on *both* needles, really, so
let's just give 300 seconds for the `begin_installation` needle
to appear. It's not going to appear on any other screen.

This problem caused a couple of spurious failures today -
https://openqa.fedoraproject.org/tests/77839 and
https://openqa.fedoraproject.org/tests/77858 - because they
took a bit too long for the INSTALLATION DESTINATION spoke to
clear.
2017-04-07 11:31:34 -07:00
Adam Williamson
bbeb6b1d9a Japanese language select needle changed a bit (Rawhide) 2017-04-05 19:59:23 -07:00
Adam Williamson
56364bb119 Firefox 'Save File' text changed in Rawhide KDE
Dunno, different font or something. New needle.
2017-04-05 19:50:20 -07:00
Adam Williamson
654534e694 Work around RHBZ #1439429 when running Firefox on X 2017-04-05 18:22:09 -07:00
Adam Williamson
e79d00500d Kicker launcher icon in KDE also larger in Rawhide 2017-04-05 17:34:58 -07:00
Adam Williamson
16517c4a6c Add yet another KDE login screen variant needle
Seems like the generic user avatars got a bit bigger in Rawhide?
Whatever.
2017-04-05 14:10:36 -07:00
Adam Williamson
1444c5030b Add tests with no user created during install
Summary:
This adds a new test suite, run for Workstation and KDE live
images, which does not create a user during install. It then
expects initial-setup (KDE) or gnome-initial-setup (Workstation)
to appear after install, creates a user, and proceeds with
normal boot.

Note the ARM image test already covers the initial-setup text
mode, and the ARM minimal image is the only case where that
actually matters (it's not included in Server).

Test Plan:
Run the new tests, check they work. Run all old
tests, check the changes didn't break them.

Reviewers: jsedlak, jskladan

Reviewed By: jsedlak

Subscribers: tflink

Differential Revision: https://phab.qa.fedoraproject.org/D1185
2017-04-05 09:43:26 -07:00
Adam Williamson
638adda97a Add new variant KDE update notification only needle
Dunno what changed here, different color tray I think.
2017-03-29 21:50:08 -07:00
Adam Williamson
42a0efc15c Add variant KDE update icon needle for new background 2017-03-29 21:32:29 -07:00
Adam Williamson
0b9006f1e1 Disambiguate another needle name 2017-03-29 21:24:47 -07:00
Adam Williamson
4ea5129312 Goddamnit, perl, and your goddamn brackets 2017-03-29 15:09:25 -07:00
Adam Williamson
4f014d4f11 Use tty2 not tty3 for freeipa_client login tests
There's an issue where the follow-on _advisory_post test tries
to log in before the 'login failed' error has cleared. We can
easily avoid this by using tty2 for the login tests, then
_advisory_post will switch to tty3 for its stuff.
2017-03-29 14:33:25 -07:00
Adam Williamson
b6d4fd7d4c Don't create user when USER_LOGIN is false, but for KDE install
Summary:
For some reason, we have `USER_LOGIN` set to 'false' for the KDE
package set install test. I really don't know / remember why
that would be; I'd think we should create a user and log in as
that user to make sure it works properly when installing KDE
from the traditional installer. It's not strictly part of the
package set test, true, but still, seems worth doing.

Also, when `USER_LOGIN` is set to 'false' and the installer runs,
we create a user called 'false'. This doesn't seem like what we
wanted, so let's not do that. I dunno if there are any other
cases besides the KDE one that this commit changes, but still.

Test Plan:
Run the full test suite and look for weirdness, check
KDE package set test works as intended (now creates a user called
'test' and logs in as that user).

Reviewers: jsedlak, jskladan

Reviewed By: jsedlak

Subscribers: tflink

Differential Revision: https://phab.qa.fedoraproject.org/D1182
2017-03-29 09:30:16 -07:00
Adam Williamson
9090485519 Add another live anaconda icon needle variant
With the new desktop background on KDE, the other needle only
matches 95%...
2017-03-28 17:37:14 -07:00
Adam Williamson
7c28a05083 Drop update desktop_notifications tests for now
These tests don't work right at all at present: they don't test
the update at all, they just boot the base image and run the
test, which is stupid.

I looked into various ways of fixing this but it's messy and I
don't think it can work properly without a lot of hacking. Even
if we get the test to do 'the right thing' - boot, set up the
update repo, update, reboot, do the test prep, reboot again, do
the actual test - I don't think it'll be quite a valid test,
because I think any AVCs or crashes that happen *before* the
update is installed will still appear as notifications when the
test finally does log into the desktop. So the test can fail
even if there are no post-update crashes or AVCs, I think.

I decided to give up on trying to make this test work properly
for now and just disable it. We can come back to it later if we
have great ideas and/or lots of time...
2017-03-28 12:26:14 -07:00
Adam Williamson
b17cf91c2d Add new GNOME update notification only needle
GNOME notification design changed a bit in F26 and Rawhide.
2017-03-27 15:35:50 -07:00
Adam Williamson
106972f35a Add login_sddm variant needles for new f26 backgrounds 2017-03-27 15:01:55 -07:00
Adam Williamson
7977b41804 Give update download yet more time (we have some big updates) 2017-03-24 14:10:29 -07:00
Jan Sedlák
d140e08463 test target on ARM is HDD_2
Differential Revision: https://phab.qa.fedoraproject.org/D1173
2017-03-22 10:38:12 +01:00
Adam Williamson
ac546e786a Add cockpit 135 variant needles
The cockpit sidebar changed color, so we need new needles.
2017-03-21 16:29:43 -07:00
Adam Williamson
1925606a57 Use std not qxl graphics to workaround #1403343 2017-03-20 11:36:20 -07:00
Adam Williamson
05ed04f04f Add manual partitioning needle variant for Rawhide text change
Label on the button changed in Rawhide.
2017-03-20 11:35:24 -07:00
Adam Williamson
cc2d53b4ed Wait a bit after input switch before doing Japanese typing 2017-03-20 11:32:50 -07:00
Adam Williamson
df32aa869a Add new 'noprofile user logged in' needle for sh 4.4 2017-03-17 08:27:42 -07:00
Adam Williamson
ba3a5152c1 Improve FreeIPA debug logging a bit
Committing without review as this is pretty trivial and I've
had it on staging for the last few days without issue. Just gets
us somewhat better info for debugging FreeIPA issues.
2017-03-16 12:36:33 -07:00
Adam Williamson
e4a8929465 Delete openh264 repo definition during repo_setup
This repo is causing problems for Branched update tests. The
repo is not available for 26 at all yet. This shouldn't be a
problem as the repo is disabled by default, but it seems that
some things - at least realmd, as used in the FreeIPA enrolment
tests - still try to update the repo's metadata when installing
packages, and fail because it 404s.

Since none of our tests actually needs this repo AFAIK, let's
just delete it in repo_setup.
2017-03-15 10:02:25 -07:00
Adam Williamson
bb1be2f0b1 Disable updates-testing for updates tests on Branched 2017-03-13 14:49:27 -07:00
Adam Williamson
da01d15406 Fix repo setup for Branched update tests (I hope)
Branched update tests are all failing because the baseurl in
fedora.repo is incorrect for Branched. This is a rather hacky
fix for this problem. It relies on the scheduler setting the
DEVELOPMENT variable when the update is for Branched (I named
the variable DEVELOPMENT rather than BRANCHED to be more
future-proof).

Alternative options I rejected were:

i) stick with MM links
ii) do something 'clever' to retrieve the URLs from MM

Rejected i) because the timing problem where the infra repo gets
updated before MM has the updated repodata checksums is just too
much of a problem; whenever that happens, dnf will refuse to use
the metadata from the infra repo and go pull it from an external
mirror, which can wind up timing out.

Rejected ii) because it seemed too fancy and not really any more
robust than just doing this and adapting it if Things Change In
Future (TM).
2017-03-13 12:43:54 -07:00
Adam Williamson
0e393f1512 Add variant 'Firefox download save file' needle for current F25 2017-03-13 08:29:10 -07:00
Adam Williamson
1b26f7c04c Add missing 'use utils' to disk_guided_encrypted_postinstall 2017-03-09 13:57:06 -08:00
Adam Williamson
115956f4ec Set a bus for the HDD in install_sata so it works with IDE CD
Explicitly specify the ahci0.0 bus for the HDD in install_sata.
This is needed to work if we are using CDMODEL=ide-cd (which we
need at present to work around a bug with SCSI CDs), and is a
good idea anyway to ensure the drive is actually connected to
the SATA bus (I dunno if it was before or not).
2017-03-09 13:50:19 -08:00
Adam Williamson
8eea2a5d1f Bump the timeout on the initial 'dnf -y update' for update tests 2017-03-03 15:55:31 -08:00
Adam Williamson
a581d20f8e Desktop update: Wait before clicking update apply button
We used to do this only for KDE, but I've seen the new update
tests sometimes fail at this point for no apparent reason, and
I'm thinking a wait may help (in case they're clicking the
button before it's really 'ready').
2017-03-02 15:33:23 -08:00
Adam Williamson
8d55935b41 Bump up the ARM boot timeout
Let's see if it can get a bit further, at least...
2017-03-02 15:16:02 -08:00
Adam Williamson
e9fda18016 Explicit timeout for the 'top' in _collect_data
This keeps failing because the default `assert_script_run`
timeout changed from 90 to 30 in the last os-autoinst update
(an unintended consequence of a change I made). This has been
fixed upstream, but in the mean time, let's just set an
explicit timeout on the call.
2017-03-02 14:47:44 -08:00
Adam Williamson
602a1ee101 Call boot_decrypt properly (function not method)
Noticed in e.g. https://openqa.fedoraproject.org/tests/58798
we're doing this wrong, `boot_decrypt` was moved into utils as
a function, but we were still calling it as a method...
2017-03-02 14:32:17 -08:00
Adam Williamson
672dd40738 Move update tests into a separate job group
This makes them more visible in the web UI.
2017-03-01 21:10:41 -08:00
Adam Williamson
9aed11a77b Work around RHBZ #1400293 (breaks IPA tests on Fedora 24) 2017-02-28 11:46:21 -08:00
Adam Williamson
effd7a6e40 Add F24 variant needle for cockpit log entry
Needed for update tests.
2017-02-28 09:52:41 -08:00