Commit Graph

211 Commits

Author SHA1 Message Date
Jan Sedlák
ed983600fa add new blivet vdb selected needle for raid (shows up on i686 and sometimes x86_64) 2017-05-11 12:07:06 +02:00
Jan Sedlák
5e2a76efde fix blivet so that correct unit for size is selected
Differential Revision: https://phab.qa.fedoraproject.org/D1193
2017-05-05 09:29:55 +02:00
Adam Williamson
1ac1977c2d Firefox addon interface changed again, add variant needles
We can probably throw the other needles away once Firefox 52
makes it to all release's stable repos.
2017-04-24 17:54:04 -07:00
Jan Sedlák
140c5f0a42 Add custom partitioning tests for blivet
Differential Revision: https://phab.qa.fedoraproject.org/D1188
2017-04-24 14:23:35 +02:00
Adam Williamson
bbeb6b1d9a Japanese language select needle changed a bit (Rawhide) 2017-04-05 19:59:23 -07:00
Adam Williamson
56364bb119 Firefox 'Save File' text changed in Rawhide KDE
Dunno, different font or something. New needle.
2017-04-05 19:50:20 -07:00
Adam Williamson
e79d00500d Kicker launcher icon in KDE also larger in Rawhide 2017-04-05 17:34:58 -07:00
Adam Williamson
16517c4a6c Add yet another KDE login screen variant needle
Seems like the generic user avatars got a bit bigger in Rawhide?
Whatever.
2017-04-05 14:10:36 -07:00
Adam Williamson
1444c5030b Add tests with no user created during install
Summary:
This adds a new test suite, run for Workstation and KDE live
images, which does not create a user during install. It then
expects initial-setup (KDE) or gnome-initial-setup (Workstation)
to appear after install, creates a user, and proceeds with
normal boot.

Note the ARM image test already covers the initial-setup text
mode, and the ARM minimal image is the only case where that
actually matters (it's not included in Server).

Test Plan:
Run the new tests, check they work. Run all old
tests, check the changes didn't break them.

Reviewers: jsedlak, jskladan

Reviewed By: jsedlak

Subscribers: tflink

Differential Revision: https://phab.qa.fedoraproject.org/D1185
2017-04-05 09:43:26 -07:00
Adam Williamson
638adda97a Add new variant KDE update notification only needle
Dunno what changed here, different color tray I think.
2017-03-29 21:50:08 -07:00
Adam Williamson
42a0efc15c Add variant KDE update icon needle for new background 2017-03-29 21:32:29 -07:00
Adam Williamson
0b9006f1e1 Disambiguate another needle name 2017-03-29 21:24:47 -07:00
Adam Williamson
9090485519 Add another live anaconda icon needle variant
With the new desktop background on KDE, the other needle only
matches 95%...
2017-03-28 17:37:14 -07:00
Adam Williamson
b17cf91c2d Add new GNOME update notification only needle
GNOME notification design changed a bit in F26 and Rawhide.
2017-03-27 15:35:50 -07:00
Adam Williamson
106972f35a Add login_sddm variant needles for new f26 backgrounds 2017-03-27 15:01:55 -07:00
Adam Williamson
ac546e786a Add cockpit 135 variant needles
The cockpit sidebar changed color, so we need new needles.
2017-03-21 16:29:43 -07:00
Adam Williamson
05ed04f04f Add manual partitioning needle variant for Rawhide text change
Label on the button changed in Rawhide.
2017-03-20 11:35:24 -07:00
Adam Williamson
df32aa869a Add new 'noprofile user logged in' needle for sh 4.4 2017-03-17 08:27:42 -07:00
Adam Williamson
0e393f1512 Add variant 'Firefox download save file' needle for current F25 2017-03-13 08:29:10 -07:00
Adam Williamson
effd7a6e40 Add F24 variant needle for cockpit log entry
Needed for update tests.
2017-02-28 09:52:41 -08:00
Adam Williamson
8712977ea4 Add new needle for FreeIPA web UI user menu
The silhouette icon seems to have changed a bit. Not sure if we
need the old needle any more, but let's keep it around for a bit
and see.
2017-02-21 14:39:06 -08:00
Adam Williamson
78c4741e00 Replace cockpit 'main screen' needle
They got rid of the 'Dashboard' text we were matching on, so
let's change this needle. This 'Hardware' text should show up
in all cockpit versions, I think.
2017-02-21 11:32:00 -08:00
Adam Williamson
58ff0ecdbd Add updated add_network_disk needle
Icon changed a bit.
2017-02-08 12:17:05 -08:00
Adam Williamson
186678e98b Make log upload work when installed system hits emergency mode
Summary:
This is to handle cases like #1414904 , where the system boots
to emergency mode. We really need logs to try and debug this.

Test Plan:
Force a test to hit emergency mode somehow (right now
you can just run base_services_start on Rawhide over and over
until you hit #1414904, but there's probably an easier way to
do it, I think there's a systemd boot arg to tell it which target
to boot for e.g.) and check logs get uploaded. Also check this
doesn't break log upload for a 'normal' failure.

Reviewers: garretraziel_but_actually_jsedlak_who_uses_stupid_nicknames

Reviewed By: garretraziel_but_actually_jsedlak_who_uses_stupid_nicknames

Subscribers: tflink

Differential Revision: https://phab.qa.fedoraproject.org/D1103
2017-02-01 12:30:21 +01:00
Adam Williamson
cf9fa890fa Avoid identical needle names in subdirectories
openQA has some problems handling needles with identical names
in different subdirectories. We haven't had the cycles to send
fixes for this yet, so for now, let's just rename all such
needles we have.
2017-01-25 10:32:14 +01:00
Adam Williamson
1e41b9db90 update KDE update icon for color change
May be other needles in the same tray that need changing, but
the test fails at the next step I think because of #1412750, so
can't be sure.
2017-01-25 10:20:46 +01:00
Adam Williamson
ff66be6ef9 Needle update for Firefox addon popover color change
Not sure if the color change will occur in Fedora 25 nightlies
or not, so not sure if we can remove the old needles. But we
*can* remove the F24 needles now I think.
2017-01-20 16:07:54 -08:00
Adam Williamson
a091185af1 Handle 'non-ASCII in password' warning for Russian installs
Summary:
Since 26.17, anaconda shows a warning when the user password
contains non-ASCII characters, and requires a second Done click
to confirm. This change should handle that.

On the 'catch cases where password typing went wrong and re-try'
bit: to keep that, but not re-type the password *every single
time* on the Russian install test, we'd have to make the needle
match the text of the warning. This is problematic because then
that needle will be able to break without us easily noticing;
that's why I wanted to keep the 'warning bar' needle text-free.
Unfortunately, that means we have to skip the protection for
switched-layout installs.

Note the protection was actually not working for any non-English
install anyhow, because the needle had `LANGUAGE-english` as a
tag. We never noticed that. Failed password typing is pretty
rare now, so we can live without the protection - it's just nice
to have it for the English install tests because there's so many
of them.

Test Plan:
Run the Russian install with a recent Rawhide image,
check it clicks 'Done' twice. Note, it will still fail, because
of RHBZ #1413813.

Reviewers: jskladan, garretraziel_but_actually_jsedlak_who_uses_stupid_nicknames

Reviewed By: garretraziel_but_actually_jsedlak_who_uses_stupid_nicknames

Subscribers: tflink

Differential Revision: https://phab.qa.fedoraproject.org/D1084
2017-01-17 23:13:47 -08:00
Adam Williamson
641b30ac09 Add another boot passphrase entry needle variant for F25
The F25 'desktopencrypt' images built by createhdds boot to a
console passphrase entry screen, not graphical. F24 images
built the same way boot to a graphical passphrase entry screen.
I'm not sure why, but it's not really worth spending a lot of
energy on, I don't think - let's just add a needle to cover
this case.
2017-01-11 14:56:50 -08:00
Adam Williamson
c4f32ab5ad add an Asian (Japanese) language install test
Summary:
Include some basic testing of Japanese input, and split the
input testing (including Russian) into a separate module, since
it's not really part of 'login' testing.

Test Plan:
Run the test, and the Russian and French tests too to
make sure they didn't break. Tested on staging. Note the Japanese
test soft fails, intentionally, at present, as I discovered a bug
while working on it:
https://bugzilla.gnome.org/show_bug.cgi?id=776189

Reviewers: jskladan, garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D1072
2016-12-21 08:41:00 -08:00
Adam Williamson
7135a4e05d iSCSI: test user authentication
Summary:
This isn't in the criteria, but it's commonly used, so we ought
to test this way. Require authentication for the iSCSI target
and have the test provide the appropriate auth info.

Test Plan:
Run the iscsi test and check it works (you need the
recent fixes for support_server to make *that* work). Nothing
else should be affected.

Reviewers: jskladan, garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D1070
2016-12-20 08:34:30 -08:00
Adam Williamson
0fa6138448 Have non-English tests do graphical install and login
Summary:
The non-English tests so far did not test that graphical login
worked as expected, which is a fairly large hole. With this
change, they should do a Workstation install and test login to
both GNOME and the console works as expected. KDE is not yet
tested.

As part of this we tweak the implementation of keyboard layout
switching in graphical environments to use a generic function
in main_common which can handle both anaconda and desktops
(just GNOME at present, but should extend easily to any desktop
with a known switcher key and a visible layout indicator),
replacing the anacondatest class method. I kinda don't like that
the test has to specifically tell the function when it's in
anaconda, but I don't think I want to start experimenting with
a global 'test phase' openQA variable or anything like that at
present.

Fixes T842.

Test Plan:
Run the French and Russian install tests and check
they work as expected. Also run an English Workstation install
if you like, and make sure that didn't break. This change is
live on staging ATM, seems to work fine.

Reviewers: jskladan, garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Maniphest Tasks: T842

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D1071
2016-12-16 09:40:29 -08:00
Adam Williamson
fbe5f0f1ba Add SDDM variant for updated F24
Looks like F24 sddm appearance changed again.
2016-12-12 15:06:19 -08:00
Adam Williamson
85d9fa4bf6 Add one more F24 needle variant 2016-12-07 17:29:15 -08:00
Adam Williamson
601815af93 Add several more F24 variants for live-respins tests 2016-12-07 17:10:45 -08:00
Adam Williamson
8510fbc008 Add updated variants of a few more needles 2016-12-07 16:11:27 -08:00
Adam Williamson
f11d5265b7 add F24 variants of some needles
To fix tests of the live-respins (which are still F24 ATM).
2016-12-07 16:01:22 -08:00
Adam Williamson
5da52acc75 tweak firefox 'download complete' needle
Rawhide changed the color of the popover outline, so exclude it
from the needle so the needle continues to work for 24/25/RH.
2016-12-07 11:51:48 -08:00
Adam Williamson
3889000907 update text install main hub needle for text change in Rawhide 2016-12-05 14:41:20 -08:00
Adam Williamson
b76da0452f update anaconda_rescue_mounted needle
changed a bit in Rawhide
2016-11-29 21:45:46 -08:00
Adam Williamson
7b31b8263e Force GNOME to notify updates, re-enable test on Workstation
Summary:
GNOME's update notification criteria are pretty braindead: it
fires the update check timer once on login then once every hour
thereafter, but only actually checks for and notifies of updates
once a day if it's after 6am(?!?!?!). So we have to do a bunch
of fiddling around to ensure we reliably get a notification.
Move the clock to 6am if it's earlier than that, and reset the
'last update check' timer to 48 hours ago, then log in to GNOME
after that.

Note: I thought this still wasn't fully reliable, but I've looked
into all the recent failures of either test on staging and
there's only one which was really 'no update notification came
up', and the logs clearly indicate PK did run an update check,
so I don't think that was a test bug (I think something went
wrong with the update check). The other failures are all 'GNOME
did something wacky', plus one case where the needle didn't quite
match because I think the match area is slightly too tall; I'll
fix that in a second.

Test Plan:
Run the tests on both KDE and GNOME and check they
work properly now (assuming nothing unrelated breaks, like KDE
crashing...)

Reviewers: jskladan, garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D1039
2016-10-27 16:23:59 -07:00
Adam Williamson
638aa01de2 update some iscsi needles for current f25 (and rawhide) 2016-10-27 00:35:42 -07:00
Adam Williamson
2a97d137ea tweak kde installer icon a bit more 2016-10-10 09:34:36 -07:00
Adam Williamson
e56b228f21 add yet another sddm variant
goddamn background change
2016-10-10 09:29:17 -07:00
Adam Williamson
0d51cdf5c1 tweak KDE 'anaconda icon' needle to not include background 2016-10-10 09:23:44 -07:00
Adam Williamson
b1f42f068d add new SDDM needle variants for Rawhide 2016-10-08 10:58:07 -07:00
Adam Williamson
e317244d1e revise install_done needles to include button text
OK, we once again hit an annoying case where the button match
that doesn't include text got matched in the sidebar:

https://openqa.fedoraproject.org/tests/39576#step/_do_install_and_reboot/13

so I think it's time to give up on that approach and just go
ahead and deal with the button text varying between live and
traditional installs. So let's change the existing needles to
include the button text and add one new needle to cover the
English live F25/F26 case. I don't think we need a 'quit'
needle for F24 (as the only thing we test for F24 is the Atomic
installer image, which says 'Reboot') or for French or Russian
(as we only test a traditional installer image for those, not
a live image).
2016-10-07 10:10:10 -07:00
Adam Williamson
ab5c97907e add 'finish configuration' needle variants for f24
the Atomic installer test is hitting 'finish configuration' now
(thanks, slow typing!) so we need an F24 needle variant (font
rendering changed).
2016-09-29 15:23:40 -07:00
Adam Williamson
92cea9f8a8 tweak initial_setup_done match for change in Rawhide
The time spoke is now named 'Time settings' (not 'Timezone
settings') so the needle didn't match. Tweak the match so it
matches both versions.
2016-09-28 13:41:33 -07:00
Adam Williamson
e9ce14a891 consolidate login waits, use postinstall not entrypoint for base
Summary:
I started out wanting to fix an issue I noticed today where
graphical upgrade tests were failing because they didn't wait
for the graphical login screen properly; the test was sitting
at the 'full Fedora logo' state of plymouth for a long time,
so the current boot_to_login_screen's wait_still_screen was
triggered by it and the function wound up failing on the
assert_screen, because it was still some time before the real
login screen appeared.

So I tweaked the boot_to_login_screen implementation to work
slightly differently (look for a login screen match, *then* -
if we're dealing with a graphical login - wait_still_screen
to defeat the 'old GPU buffer showing login screen' problem
and assert the login screen again). But while working on it,
I figured we really should consolidate all the various places
that handle the bootloader -> login, we were doing it quite
differently in all sorts of different places. And as part of
that, I converted the base tests to use POSTINSTALL (and thus
go through the shared _wait_login tests) instead of handling
boot themselves. As part of *that*, I tweaked main.pm to not
require all POSTINSTALL tests have the _postinstall suffix on
their names, as it really doesn't make sense, and renamed the
tests.

Test Plan: Run all tests, see if they work.

Reviewers: jskladan, garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D1015
2016-09-27 11:48:15 -07:00