Commit Graph

848 Commits

Author SHA1 Message Date
Adam Williamson
df2c3cd906 Test upgrade of FreeIPA server and client deployment
Summary:
This adds an upgrade variant of the FreeIPA tests, with only
the simplest client enrolment (sssd) for now. The server test
starts from the N-1 release and deploys the domain controller
role. The client test similarly starts from the N-1 release
and, when the server is deployed, enrols as a domain client.
Then the server upgrades itself, while the client waits (as the
server is its name server). Then the client upgrades itself,
while the server does some self-checks. The server then waits
for the client to do its checks before decommissioning itself,
as usual. So, summary: *deployment* of both server and client
occurs on N-1, then both are upgraded, then the actual *checks*
occur on N.

In my testing, this all more or less works, except the role
decommission step fails. This failure seems to be a genuine one
so far as I can tell; I intend to file a bug for it soon.

Test Plan:
Run the new tests, check they work. Run the existing
FreeIPA tests (both the compose and the update variants), check
they both behave the same.

Reviewers: jsedlak, jskladan

Reviewed By: jsedlak

Subscribers: tflink

Differential Revision: https://phab.qa.fedoraproject.org/D1204
2017-06-02 12:17:07 -07:00
Jan Sedlák
48b99a2291 add base system logging test
Differential Revision: https://phab.qa.fedoraproject.org/D1202
2017-06-01 11:06:04 +02:00
Jan Sedlák
4114406668 add UEFI for blivet tests
Differential Revision: https://phab.qa.fedoraproject.org/D1201
2017-05-22 09:26:58 +02:00
Jan Sedlák
0b5f865c8f add custom btrfs partitioning test for blivet-gui
Differential Revision: https://phab.qa.fedoraproject.org/D1194
2017-05-19 13:58:16 +02:00
Adam Williamson
3638adda81 Tweak kernel patch needle to avoid raw patch files
Seems like the top '[patch]' link on kernel.org may now be a
raw patch file which the browser just displays, not a compressed
patch it offers to download. So tweak the needle so we should
click on the *bottom* link instead. openQA looks for matches
as close as possible to the location in the needle.
2017-05-15 18:00:51 -07:00
Adam Williamson
b3d8c4a811 Add variant blivet 'part_fs' needle, rename the other
When running universal tests with a non-Server image, which is
not common but *does* happen (e.g. manually built test image),
the pre-selected filesystem in blivet-gui will be ext4 not xfs
(just as with custom partitioning). So we need xfs and ext4
variants of this needle. Renamed the XFS one for consistency.
2017-05-11 22:21:38 -07:00
Jan Sedlák
ed983600fa add new blivet vdb selected needle for raid (shows up on i686 and sometimes x86_64) 2017-05-11 12:07:06 +02:00
Jan Sedlák
ea5296b306 add postinstalls to custom partitioning tests
Differential Revision: https://phab.qa.fedoraproject.org/D1192
2017-05-09 09:11:34 +02:00
Jan Sedlák
5e2a76efde fix blivet so that correct unit for size is selected
Differential Revision: https://phab.qa.fedoraproject.org/D1193
2017-05-05 09:29:55 +02:00
Adam Williamson
c2b1ea4e8c Use test3 not test1 for FreeIPA password change test module
Previously this module changed test1's password, so that it
would still be able to work even if the webui test module
failed (so test3/test4 didn't get created). But this means
that, for about 30 seconds, test1's password is 'loremipsum'
not 'batterystaple', and if one of the *other* client test
jobs happens to hit a point where it has to auth as test1
during the 30 seconds test1's password is different, it will
fail. This looks to be what happened to the join_kickstart
test the last few days - it failed because it tried to login
as test1 during the password change window.

By using the test3 user instead (which is only used by the
join_cockpit test, currently) we avoid this problem, at the
cost that the password_change module will always fail if the
webui module fails.
2017-05-04 17:29:35 -07:00
Adam Williamson
7d07147b7e Work around F26 base image bug: g-i-s failing to run
With the latest F26 base images, it seems like g-i-s fails to
run at first login. This is clearly some kind of bug somewhere
and I'll investigate it, but it shouldn't be causing the update
tests to fail - we can still validly run the tests with g-i-s
not running. So for now, adjust the _graphical_wait_login test
to tolerate this behaviour when running update tests.
2017-04-24 18:49:25 -07:00
Adam Williamson
1ac1977c2d Firefox addon interface changed again, add variant needles
We can probably throw the other needles away once Firefox 52
makes it to all release's stable repos.
2017-04-24 17:54:04 -07:00
Jan Sedlák
140c5f0a42 Add custom partitioning tests for blivet
Differential Revision: https://phab.qa.fedoraproject.org/D1188
2017-04-24 14:23:35 +02:00
Adam Williamson
22e210aa21 Run Atomic Host installer test on UEFI as well as BIOS
Atomic team asked for this, so hey, why not.
2017-04-18 16:48:37 -07:00
Adam Williamson
e6076e14b8 Workaround issues with loading same module more than once
Summary:
Loading the same module more than once *kinda* works, but it
shows up all kinds of funky in the openQA web interface. There's
a POO for this:
https://progress.opensuse.org/issues/10514
But it doesn't seem like it's going to be resolved immediately,
so in the mean time maybe we should avoid doing it so we don't
have to deal with the weirdness it produces in the web UI. So
here's a kinda icky hack that uses symlinks and stuff to load
multiple instances of 'the same' test module.

Test Plan:
Run an update test, look at how it looks in the web
UI and confirm it's a lot clearer and less buggy. Check there
aren't any bugs in the loading approach. This is deployed on stg
so you can look at it there.

Reviewers: jsedlak, jskladan

Reviewed By: jsedlak

Subscribers: tflink

Differential Revision: https://phab.qa.fedoraproject.org/D1186
2017-04-18 10:38:16 -07:00
Adam Williamson
9da72de4ed Hide mouse on login screen (BGO #781294)
GDM 3.24.1 dumps a cursor in the middle of the login screen even
if you didn't move the mouse yet, so let's workaround that.
2017-04-13 15:28:37 -07:00
Adam Williamson
ee283b1411 Tweak _console_avc_crash so we get soft failures
We do want soft failures (but not hard) when _console_avc_crash
fails. So let's tweak it to ensure that's what happens.
2017-04-10 15:44:13 -07:00
Adam Williamson
e68e113f76 Remove test_flags comments, add ignore_failure flag
It's not really a good idea to have the comments that explain
the test_flags in *every* test, because they can go stale and
then we either have to live with them being old or update them
all. Like, now. So let's just take 'em all out. There's always
a reference in the openQA and os-autoinst docs, and those get
updated faster.

More importantly, add the new `ignore_failure` flag to relevant
tests - all the tests that don't have the 'important' or
'fatal' flag at present. Upstream killed the 'important' flag
(making all tests 'important' by default), I got it replaced
with the 'ignore_failure' flag, we now need to explicitly mark
all modules we want the 'ignore_failure' behaviour for.
2017-04-10 15:00:10 -07:00
Adam Williamson
5b4ce80487 Tweak needle matches at start of _do_install_and_reboot
The way this was set up before, if `anaconda_main_hub` matched
immediately but some spoke was still in a 'processing' state,
it only had 30 seconds (default `assert_and_click` timeout) to
complete and allow the 'Begin Installation' button to appear.
It seems unnecessary to match on *both* needles, really, so
let's just give 300 seconds for the `begin_installation` needle
to appear. It's not going to appear on any other screen.

This problem caused a couple of spurious failures today -
https://openqa.fedoraproject.org/tests/77839 and
https://openqa.fedoraproject.org/tests/77858 - because they
took a bit too long for the INSTALLATION DESTINATION spoke to
clear.
2017-04-07 11:31:34 -07:00
Adam Williamson
bbeb6b1d9a Japanese language select needle changed a bit (Rawhide) 2017-04-05 19:59:23 -07:00
Adam Williamson
56364bb119 Firefox 'Save File' text changed in Rawhide KDE
Dunno, different font or something. New needle.
2017-04-05 19:50:20 -07:00
Adam Williamson
654534e694 Work around RHBZ #1439429 when running Firefox on X 2017-04-05 18:22:09 -07:00
Adam Williamson
e79d00500d Kicker launcher icon in KDE also larger in Rawhide 2017-04-05 17:34:58 -07:00
Adam Williamson
16517c4a6c Add yet another KDE login screen variant needle
Seems like the generic user avatars got a bit bigger in Rawhide?
Whatever.
2017-04-05 14:10:36 -07:00
Adam Williamson
1444c5030b Add tests with no user created during install
Summary:
This adds a new test suite, run for Workstation and KDE live
images, which does not create a user during install. It then
expects initial-setup (KDE) or gnome-initial-setup (Workstation)
to appear after install, creates a user, and proceeds with
normal boot.

Note the ARM image test already covers the initial-setup text
mode, and the ARM minimal image is the only case where that
actually matters (it's not included in Server).

Test Plan:
Run the new tests, check they work. Run all old
tests, check the changes didn't break them.

Reviewers: jsedlak, jskladan

Reviewed By: jsedlak

Subscribers: tflink

Differential Revision: https://phab.qa.fedoraproject.org/D1185
2017-04-05 09:43:26 -07:00
Adam Williamson
638adda97a Add new variant KDE update notification only needle
Dunno what changed here, different color tray I think.
2017-03-29 21:50:08 -07:00
Adam Williamson
42a0efc15c Add variant KDE update icon needle for new background 2017-03-29 21:32:29 -07:00
Adam Williamson
0b9006f1e1 Disambiguate another needle name 2017-03-29 21:24:47 -07:00
Adam Williamson
4ea5129312 Goddamnit, perl, and your goddamn brackets 2017-03-29 15:09:25 -07:00
Adam Williamson
4f014d4f11 Use tty2 not tty3 for freeipa_client login tests
There's an issue where the follow-on _advisory_post test tries
to log in before the 'login failed' error has cleared. We can
easily avoid this by using tty2 for the login tests, then
_advisory_post will switch to tty3 for its stuff.
2017-03-29 14:33:25 -07:00
Adam Williamson
b6d4fd7d4c Don't create user when USER_LOGIN is false, but for KDE install
Summary:
For some reason, we have `USER_LOGIN` set to 'false' for the KDE
package set install test. I really don't know / remember why
that would be; I'd think we should create a user and log in as
that user to make sure it works properly when installing KDE
from the traditional installer. It's not strictly part of the
package set test, true, but still, seems worth doing.

Also, when `USER_LOGIN` is set to 'false' and the installer runs,
we create a user called 'false'. This doesn't seem like what we
wanted, so let's not do that. I dunno if there are any other
cases besides the KDE one that this commit changes, but still.

Test Plan:
Run the full test suite and look for weirdness, check
KDE package set test works as intended (now creates a user called
'test' and logs in as that user).

Reviewers: jsedlak, jskladan

Reviewed By: jsedlak

Subscribers: tflink

Differential Revision: https://phab.qa.fedoraproject.org/D1182
2017-03-29 09:30:16 -07:00
Adam Williamson
9090485519 Add another live anaconda icon needle variant
With the new desktop background on KDE, the other needle only
matches 95%...
2017-03-28 17:37:14 -07:00
Adam Williamson
7c28a05083 Drop update desktop_notifications tests for now
These tests don't work right at all at present: they don't test
the update at all, they just boot the base image and run the
test, which is stupid.

I looked into various ways of fixing this but it's messy and I
don't think it can work properly without a lot of hacking. Even
if we get the test to do 'the right thing' - boot, set up the
update repo, update, reboot, do the test prep, reboot again, do
the actual test - I don't think it'll be quite a valid test,
because I think any AVCs or crashes that happen *before* the
update is installed will still appear as notifications when the
test finally does log into the desktop. So the test can fail
even if there are no post-update crashes or AVCs, I think.

I decided to give up on trying to make this test work properly
for now and just disable it. We can come back to it later if we
have great ideas and/or lots of time...
2017-03-28 12:26:14 -07:00
Adam Williamson
b17cf91c2d Add new GNOME update notification only needle
GNOME notification design changed a bit in F26 and Rawhide.
2017-03-27 15:35:50 -07:00
Adam Williamson
106972f35a Add login_sddm variant needles for new f26 backgrounds 2017-03-27 15:01:55 -07:00
Adam Williamson
7977b41804 Give update download yet more time (we have some big updates) 2017-03-24 14:10:29 -07:00
Jan Sedlák
d140e08463 test target on ARM is HDD_2
Differential Revision: https://phab.qa.fedoraproject.org/D1173
2017-03-22 10:38:12 +01:00
Adam Williamson
ac546e786a Add cockpit 135 variant needles
The cockpit sidebar changed color, so we need new needles.
2017-03-21 16:29:43 -07:00
Adam Williamson
1925606a57 Use std not qxl graphics to workaround #1403343 2017-03-20 11:36:20 -07:00
Adam Williamson
05ed04f04f Add manual partitioning needle variant for Rawhide text change
Label on the button changed in Rawhide.
2017-03-20 11:35:24 -07:00
Adam Williamson
cc2d53b4ed Wait a bit after input switch before doing Japanese typing 2017-03-20 11:32:50 -07:00
Adam Williamson
df32aa869a Add new 'noprofile user logged in' needle for sh 4.4 2017-03-17 08:27:42 -07:00
Adam Williamson
ba3a5152c1 Improve FreeIPA debug logging a bit
Committing without review as this is pretty trivial and I've
had it on staging for the last few days without issue. Just gets
us somewhat better info for debugging FreeIPA issues.
2017-03-16 12:36:33 -07:00
Adam Williamson
e4a8929465 Delete openh264 repo definition during repo_setup
This repo is causing problems for Branched update tests. The
repo is not available for 26 at all yet. This shouldn't be a
problem as the repo is disabled by default, but it seems that
some things - at least realmd, as used in the FreeIPA enrolment
tests - still try to update the repo's metadata when installing
packages, and fail because it 404s.

Since none of our tests actually needs this repo AFAIK, let's
just delete it in repo_setup.
2017-03-15 10:02:25 -07:00
Adam Williamson
bb1be2f0b1 Disable updates-testing for updates tests on Branched 2017-03-13 14:49:27 -07:00
Adam Williamson
da01d15406 Fix repo setup for Branched update tests (I hope)
Branched update tests are all failing because the baseurl in
fedora.repo is incorrect for Branched. This is a rather hacky
fix for this problem. It relies on the scheduler setting the
DEVELOPMENT variable when the update is for Branched (I named
the variable DEVELOPMENT rather than BRANCHED to be more
future-proof).

Alternative options I rejected were:

i) stick with MM links
ii) do something 'clever' to retrieve the URLs from MM

Rejected i) because the timing problem where the infra repo gets
updated before MM has the updated repodata checksums is just too
much of a problem; whenever that happens, dnf will refuse to use
the metadata from the infra repo and go pull it from an external
mirror, which can wind up timing out.

Rejected ii) because it seemed too fancy and not really any more
robust than just doing this and adapting it if Things Change In
Future (TM).
2017-03-13 12:43:54 -07:00
Adam Williamson
0e393f1512 Add variant 'Firefox download save file' needle for current F25 2017-03-13 08:29:10 -07:00
Adam Williamson
1b26f7c04c Add missing 'use utils' to disk_guided_encrypted_postinstall 2017-03-09 13:57:06 -08:00
Adam Williamson
115956f4ec Set a bus for the HDD in install_sata so it works with IDE CD
Explicitly specify the ahci0.0 bus for the HDD in install_sata.
This is needed to work if we are using CDMODEL=ide-cd (which we
need at present to work around a bug with SCSI CDs), and is a
good idea anyway to ensure the drive is actually connected to
the SATA bus (I dunno if it was before or not).
2017-03-09 13:50:19 -08:00
Adam Williamson
8eea2a5d1f Bump the timeout on the initial 'dnf -y update' for update tests 2017-03-03 15:55:31 -08:00