Commit Graph

779 Commits

Author SHA1 Message Date
Adam Williamson
417b876ead Add F31 variant Terminal app run needle
Needed for F31 respin tests.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-04-14 12:12:57 -07:00
Adam Williamson
19677de20a Add variant anaconda keyboard layout needles
There's this weird thing where the vertical alignment of the
language name in the keyboard layout indicator is sometimes
different. I never can figure out why. It may be to do with
presence or absence of the pre-release indicator.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-04-14 12:07:06 -07:00
Adam Williamson
610572f8a0 Add new variant SDDM input needle
Looks like a kerning difference or something.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-04-14 12:05:33 -07:00
Adam Williamson
21eea18562 Add a needle check script, remove some unused needles
I call this...The @lruzicka Catcher!

It's a script that checks for needles that aren't actually used
anywhere. It also checks for cases where we have a needle JSON
file but no image, or an image file but no JSON file (and wipes
one case of the latter). It also adds a run of the script to tox
so we get it in CI.

You could make this script a lot more elaborate if you like, by
being fancier about parsing the test code and templates, but I
don't think it's really warranted, I think it just needs to be
'good enough'. It's not the end of the world if it misses the
odd thing or the whitelisting goes stale.

Quite a lot of the removed needles are remnants of different
approaches to app start/stop testing which weren't caught in the
initial PR review. The short-name partitioning ones are odd; they
were introduced in the commit that moved needles into subdirs,
but at least some of them don't actually appear to be moves. They
may have been non-tracked files Josef had lying around that got
into the commit by mistake, or they may just be old needles we
really used at some point but aren't using any more.
reclaim_space_second_partition was introduced as part of the
shrink test (along with reclaim_space_first_partition) but was
never actually used by that test - I guess, again, the test got
re-written during review but we forgot to remove the needle. We
rejigged user creation to use tab presses not a needle match a
while back, which made user_creation_password_input unnecessary.
The various cockpit_updates_* needles are I think remnants of
rewrites of the cockpit update tests that again were missed in
PR review, the tests as merged never used them.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-04-08 16:55:46 -07:00
Adam Williamson
8b5be61971 Update a few GNOME app needles that seem to have changed
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-04-02 20:27:01 -07:00
Adam Williamson
ea397783d5 Update French 'accept fate' needle
Verb changed.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-04-02 20:17:42 -07:00
Adam Williamson
6b5e933ff4 Update error_report needle
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-04-02 20:16:02 -07:00
Lukáš Růžička
a54f020118 Revert "Fix the Konqueror test - handle Locations dialogue."
This reverts commit d784bf54ca.

It turned out that Locations are not connected to Konqueror
at all. The reason why the test is failing is that the
application has been removed to limit the number of
web browsers.
2020-03-31 09:23:13 +02:00
Lukáš Růžička
d784bf54ca Fix the Konqueror test - handle Locations dialogue. 2020-03-26 10:57:00 +01:00
Adam Williamson
dc7b7a7241 Great Needle Cleanup 2020
Remove a bunch of needles that have not been used for some time,
plus a few workarounds that are similarly stale.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-03-20 14:02:10 -07:00
Adam Williamson
e2df109539 Update cockpit_updates_check needle
Button changed a bit (likely Patternfly 4 changes).

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-03-18 12:21:18 -07:00
Adam Williamson
c18a0d5008 Update cockpit 'updated' needle (check mark got thinner)
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-03-18 11:57:33 -07:00
Lukáš Růžička
d19cf9553e Reneedle the Gnome Application StartStop tests. 2020-03-18 18:06:34 +01:00
Adam Williamson
608209499d Add Fedora 33 IoT ident needle
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-03-16 14:17:31 -07:00
Adam Williamson
30f5eae3ac Add another kernelorg_patch needle
I think what happens here is the kerning and/or subpixel hinting
changes depending on the column position, and the column position
keeps changing as upstream releases new versions on different
dates and stuff. Hopefully eventually we'll have enough needles
to cover all possibilities...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-03-13 14:08:05 -07:00
Adam Williamson
ff1d75050e Update French pre-release note needle
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-03-12 09:19:04 -07:00
Adam Williamson
948e5f23ae Add missing tags to Russian prerelease note needle
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-03-12 08:50:05 -07:00
Lukáš Růžička
0557af9a2d Add needle to cover a changed prerelease note for Russian. 2020-03-12 16:17:28 +01:00
Adam Williamson
00f1241318 Update French 'install done' needle
Translation was updated.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-03-10 16:04:45 -07:00
Adam Williamson
513f0a23f4 Add F32 background needle
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-03-06 15:33:15 -08:00
Adam Williamson
e5feab11f7 Update iSCSI-related needles for font changes
This test has been failing forever, now the bug is fixed, we need
to update the needles for font changes...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-03-04 09:15:00 -08:00
Adam Williamson
884f71bb81 Revert "Workaround #1805553 (single disk not selected as install target)"
This reverts commit adf3f91818.
The bug has been fixed in anaconda, so we can drop this, which
is good as it has timing issues producing false positives on
Rawhide...
2020-02-26 14:30:08 -08:00
Adam Williamson
2c236ef16a Update GNOME Software 'Download' needle for changed kerning
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-26 11:34:21 -08:00
Adam Williamson
95e6ddc896 Update blivet vdb selected needle yet again
I do not know why this needle seems so troublesome...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-25 23:02:29 -08:00
Adam Williamson
da28938572 Update prerelease note needle for font rendering change
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-25 22:59:14 -08:00
Adam Williamson
adf3f91818 Workaround #1805553 (single disk not selected as install target)
Rawhide seems to have developed a bug where a single disk is no
longer automatically selected as the install target when you
enter the INSTALLATION DESTINATION spoke. We need to work around
this (but register it as a soft failure).

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-20 17:41:34 -08:00
Adam Williamson
54afe23572 Update an iSCSI needle for font changes
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-20 12:14:35 -08:00
Adam Williamson
18c818da7d Update overview app grid needle (more GNOME icon changes)
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-20 09:57:04 -08:00
Adam Williamson
5300c35106 Update desktop update notification only needle
More GNOME icon changes.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-20 09:53:30 -08:00
Adam Williamson
deab03a80d Update another partitioning needle for GTK+ widget changes
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-20 08:12:16 -08:00
Adam Williamson
2c33325595 Add IoT 32 identification needle
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-20 08:11:52 -08:00
Adam Williamson
7fd23dacea Update GDM input needles for non-English languages
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-19 10:37:51 -08:00
Adam Williamson
8d14addd18 Update two more anaconda needles for smaller radio buttons
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-19 10:36:59 -08:00
Adam Williamson
9f331dbbe3 Update authentication required GNOME needle
They took away the icon, so now we can only match on text :(

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-19 10:35:47 -08:00
Adam Williamson
2466cf53bd Update Firefox launcher needle
Looks like GNOME is rendering it a bit smaller now.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-19 10:35:06 -08:00
Adam Williamson
dde0113fb1 Update KDE 'no notifications' needle
Probably a transparency thing.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-19 09:47:36 -08:00
Adam Williamson
f7a9625b55 Update GDM input needle
GNOME reworked this widget a bit.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-19 09:47:13 -08:00
Adam Williamson
2fda57482b Update anaconda needles for icon/widget changes
Seems some changes to symbolic icons and widgets have happened
in GTK+ or something, this updates anaconda needles for them.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-19 09:46:35 -08:00
Adam Williamson
0f8f6082eb Enable IoT testing with install and base tests
IoT is becoming a release-blocking edition for F32, so we should
be testing it for sure. We may add specific tests, but for now
let's run the install and base tests on it.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-17 08:36:55 -08:00
Adam Williamson
b09936ba0a Update 'select_disk_1' needle
Some kinda font rendering change or something.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-17 08:33:57 -08:00
Adam Williamson
e5728beada Add version 32 identification needle
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-14 16:53:31 -08:00
Adam Williamson
80ddb89f91 Fix firefox addon_success needle
I updated this last month without looking closely enough; one of
the matches was bogus. We now don't seem to get the 'added to
Firefox' popup any more, so we can't match on it, but I guess we
can match on the 'remove' button as an indicator that the add
worked.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-07 12:10:21 -08:00
Adam Williamson
80c54d5491 Add cockpit_updates and remote_logging tests to updates
Again, no reason not to run these on updates. Includes adding
oldcantarell versions of several needles for current cockpit,
as they're needed for the tests to pass. Also tweak a couple of
needles to avoid false matches (add more empty space).

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-24 21:03:41 +01:00
Adam Williamson
12b0fb04dc Update a ppc64le-only blivet needle for Cantarell 0.201
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-24 15:20:54 +01:00
Adam Williamson
776d31bd5b Update various Cockpit needles for 211
Note we need two variants of most of these, one for Rawhide with
Cantarell 0.201, one for F30/F31 with Cantarell 0.111.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-24 15:18:27 +01:00
Adam Williamson
f11a57f01a Drop match level on cockpit main needle
New cockpit hits 95%, let's just drop the required %age rather
than make a new needle.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-24 12:53:14 +01:00
Adam Williamson
c922226d7c Add a variant Cockpit join_complete needle
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-20 13:36:58 +01:00
Adam Williamson
49a540cc3d Update addon_success needle for new Firefox
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-20 13:36:28 +01:00
Adam Williamson
f8292bfeeb More Cantarell: Russian and French
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-18 23:56:11 +00:00
Adam Williamson
a653ded877 Next batch of Cantarell needles: Arabic install
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-17 15:32:30 +00:00
Adam Williamson
28642f4002 Move a stray app needle to the right place
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-17 15:19:17 +00:00
Adam Williamson
8612858b1f Second batch of Cantarell needles: GNOME apps tests
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-17 15:18:50 +00:00
Adam Williamson
21febe3592 First batch of updates for new version of cantarell
Still need to do the non-English languages. I'll do those
tomorrow.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-17 01:18:52 +00:00
Adam Williamson
13da4f2d26 Add another new Cockpit join complete needle
This one from ppc64le test. The difference between all the
recent needles here is the amount of white space between the
'Domain' and 'domain.local'. I don't want to change it to two
match areas as there's a chance we may get a false positive
match when the join actually failed (it may match on 'Domain'
in the right place, and 'domain.local' somewhere else just as
part of the system's hostname). I suppose if this keeps happening
we could try two match areas but include some white space to the
left of 'domain.local' to avoid matching on it as part of the
hostname...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-15 15:22:34 +00:00
Adam Williamson
1e025c0b27 Tweak cockpit main needle to work on ppc64 too
We don't get a CPU model on ppc64, we get a machine ID everywhere
(I hope).

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-15 12:16:33 +00:00
Adam Williamson
35c863a312 Lower match level on another kernel.org patch needle
We're now getting 91-92% matches on this older needle.
¯\_(ツ)_/¯

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-10 08:25:55 -08:00
Adam Williamson
aa0edd263b Update Cockpit 'realm join complete' needle for new cockpit
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-08 17:28:49 -08:00
Adam Williamson
3998e9c86c Drop the 'cockpit join progress' needle match
The text we used to get has been replaced with a spinner, which
is difficult and unreliable to match on. This match was only
here to make the test fail a bit faster if it was broken, so
let's just live without it.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-08 15:32:29 -08:00
Adam Williamson
e3430e82ef Update Rhythmbox app needle
The icon changed, so the needle needs changing.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-06 12:32:44 -08:00
Adam Williamson
766d5f163e Update GDM input needles (an icon appeared in the text entry)
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-06 10:57:53 -08:00
Adam Williamson
acf27fcd4a Update the blivet 'bluebox-alt' needle used on alt arches
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-06 10:56:44 -08:00
Adam Williamson
7234729949 Drop use of logs_severity needle from Cockpit test entirely
This match keeps giving us problems, and now I look at the test
again, there is actually no need for it at all. Clicking it
doesn't do anything, and we already confirm that we're on the
right page at the next step, where we look for a log entry and
click on that - that will fail if we aren't actually on the
Logs page.

I don't remember what Cockpit used to look like when we first
put this line and needle in, presumably there's a reason we had
them, but they're clearly unnecessary now.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-03 09:27:48 -08:00
Adam Williamson
5be282c744 Drop cockpit_logs_severity match requirement again
I am so freaking pissed off with these bizarre 'not quite'
matches, what the hell is Firefox doing

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-12-20 09:04:52 -08:00
Adam Williamson
a88d7b701e Add another Cockpit realm join complete button
Whitespace changed, sigh...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-12-18 23:21:15 -08:00
Adam Williamson
f9f2d08263 Add new cockpit autoupdate on needle
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-12-18 12:36:57 -08:00
Adam Williamson
237cdee4b2 Tweak Cockpit log severity needles
Sigh, I don't know why we're having so much trouble with fonts
in browsers lately. Drop the match requirement on the needles
that have been matching or nearly matching the most, and remove
older ones that aren't matching.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-12-18 10:51:47 -08:00
Adam Williamson
beba6426b1 Update realmd_join_cockpit for Cockpit 209
We need a couple of new needles, plus the 'join domain' button
has disappeared from the front page due to the very inefficient
UI redesign, so we need to scroll down to find it.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-12-14 12:29:00 -08:00
Adam Williamson
5d59834a36 Drop kernel.org path needle match level again (to 90)
Man, I have had it with these needles.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-12-14 09:24:15 -08:00
Adam Williamson
5692692610 Add new cockpit_main needle for cockpit 209
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-12-13 17:55:06 -08:00
Adam Williamson
fa062bd9bb Tweak kernel.org patch needles again
This is getting annoying. We're getting, like, 94-95% matches
for the recent needles sometimes. Instead of continuing to make
more and more needles, let's try dropping the required match
threshold a bit. The kde-20191206 needle doesn't *seem* to be
needed so let's get rid of it.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-12-13 17:18:51 -08:00
Adam Williamson
30a2b68974 Add some variant printing needles for F30
Printer icon is different, final output looks a bit different.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-12-06 17:28:27 -08:00
Adam Williamson
485a24b3ee Yet another cockpit logs_severity needle
I don't know why this is causing so much trouble...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-12-06 11:54:29 -08:00
Adam Williamson
629c67e1f5 Whoops, add missing area to akonadi notification needle
Needed for the click to actually dismiss the notification.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-12-06 10:07:40 -08:00
Adam Williamson
c8672f3b48 Widen match area in FreeIPA add button needle
Otherwise we can get a problematic match on 'Add and Add Another'

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-12-06 09:40:51 -08:00
Adam Williamson
a9f1591c32 Another kernel.org patch needle
This is for KDE. KDE and GNOME have different font rendering,
and they both look slightly different between a few days ago
and today, I think because of a change to the width of the
columns on the page itself.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-12-06 09:37:15 -08:00
Adam Williamson
e684fed622 Add missing akonadi notification needle
I must've forgotten to include this in the commit.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-12-06 09:34:41 -08:00
Adam Williamson
44ad6a5068 Add yet another kernel.org patch browser needle
Slight font rendering changes from the last GNOME version of
this needle, not sure if there's actually a font library change
or it's just hinting changes caused by the column being a bit
further right than before...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-12-06 09:27:07 -08:00
Lukáš Růžička
77ae127cce Create an automated test to test printing on Desktop
Add functions to test on KDE, too.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-12-05 09:22:50 -08:00
Adam Williamson
f4ca51e508 Update a few Firefox needles again
Seems the latest Firefox builds in F30 and F31 updates render
fonts a little differently again, not sure why this is.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-12-04 09:42:02 -08:00
Adam Williamson
52b34c2be0 Add another Firefox close tabs needle
Not sure what this vs. the one from two days is all about, but.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-29 13:45:17 -08:00
Adam Williamson
ee140358b7 Update akonadi migration agent needle
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-29 13:04:51 -08:00
Adam Williamson
7fbaa426e8 Update a bunch of needles for font rendering change in Rawhide
Not sure what changed; it seems like mostly browser needles got
broken, but there's a few installer needles too.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-29 13:00:54 -08:00
Adam Williamson
5c03c435ca Stop having two 'firefox close tabs' needle tags, drop old ones
We still have a 'apps_run_firefox_stop' needle tag which is for
the same thing as 'firefox_close_tabs'. That's dumb. Get rid of
it and only have the firefox_close_tabs tag and needles. Also
clean up some old firefox_close_tabs needles that haven't matched
for months and all the 'apps_run_firefox_stop' needles.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-29 12:10:35 -08:00
Adam Williamson
e7a7c053bd Update firefox_close_tabs needle
Seems to look different in a recent Rawhide test.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-27 11:08:44 -08:00
Adam Williamson
e8802935f9 Tweak text_console_login match area
On Rawhide Cloud_Base boots, there's some SSH key and network
information printed above the 'login:' prompt, so we can't
expect empty space there. Also tweak console_login() to clear
the screen after logging in, so the login prompt is cleared and
doesn't confuse things on subsequent runs (like it did first
time we tried this). And add a new user logged in needle, as it
seems after we clear the screen the tilde appears in a slightly
different position and the existing needle doesn't match.

https://openqa.fedoraproject.org/tests/489003#step/_console_wait_login/7

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-27 10:39:29 -08:00
Adam Williamson
411f0d34e7 Revert "Tweak text_console_login match area"
This reverts commit f66f510832.
Turns out this breaks lots of stuff, so it needs to go back.
I'll have to figure another way to handle the cloud login.
2019-11-27 08:36:35 -08:00
Adam Williamson
f66f510832 Tweak text_console_login match area
On Rawhide Cloud_Base boots, there's some SSH key and network
information printed above the 'login:' prompt, so we can't
expect empty space there. Let's just hope not looking for the
empty space doesn't break anything else.

https://openqa.fedoraproject.org/tests/489003#step/_console_wait_login/7

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-26 08:26:27 -08:00
Adam Williamson
25cce2361c Update a KDE app test needle
Window title changed.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-22 09:13:27 -08:00
Adam Williamson
0722cc5bcf Update Cockpit 'join domain' needle for cockpit 207
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-14 08:43:03 -08:00
Adam Williamson
4f72d4ebd3 Add one missing needle for VNC test
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-05 14:48:25 -08:00
Adam Williamson
b82a1dec6f Add test for QA:Testcase_Anaconda_User_Interface_VNC
This adds a test for QA:Testcase_Anaconda_User_Interface_VNC -
the VNC install test case. It's implemented as a server/client
pair, with the server booting from the Server DVD image with
`inst.vnc` and the client booting from the desktop base disk
image, setting up networking, then running Boxes to connect to
the server and run the install.

There are various little tweaks to test loading and logic to
handle this, mostly pretty clear. We also move the workaround
for 'spurious auth prompt appears on desktop after you switch
away to a VT and back' out of the desktop update test and into
the `desktop_vt` helper function, since now this test can hit
it as well. We enhance _graphical_wait_login to handle the boot
loader if needed (it has never needed to until now).

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-05 14:46:10 -08:00
Adam Williamson
a0e191e097 Update cockpit login screen needle *again*
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-30 15:20:30 -07:00
Adam Williamson
efec7010cb Handle an 'akonadi did something' notification in KDE, etc.
This handles a case where KDE shows a notification saying
'PIM Maintenance (Finished)', like this:

https://openqa.fedoraproject.org/tests/477345#step/desktop_notifications/34

we need to click it away for the desktop_notification test to
pass. It also clarifies the difference between this notification
and the eternal 'akonadi_migration_agent is doing something'
popup in the needle names and comments. It also replaces the
'check_screen then assert_and_click if found' pattern in several
notifications-related places with the better 'check_screen then
click_lastmatch if found' pattern now available upstream.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-30 09:05:06 -07:00
Adam Williamson
81100e8a17 Add new Cockpit 'software updates' needle
Needed for the UI change in Cockpit 205.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-29 16:01:35 -07:00
Adam Williamson
4fb29f8bdb Add new Firefox GNOME app menu needle
Firefox icon changed, so this needs updating too.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-29 15:55:52 -07:00
Adam Williamson
3cdfe96a5c Add another Firefox launcher needle variant
Seems we need two as the launcher background is a different
color on different Fedora releases...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-24 13:53:52 -07:00
Adam Williamson
51d37aeeef Update Firefox launcher needles for new icon
Firefox icon changed a bit in 70.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-24 13:23:24 -07:00
Adam Williamson
e313e58cdd Add new variant boot_enter_passphrase graphical needle
New plymouth has tweaked this screen's appearance.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-24 13:20:51 -07:00
Adam Williamson
e1030118f9 Add new cockpit login needle
The login screen background changed in 205.1 (it's actually the
same image, but it's now zoomed rather than just a chunk of it
being shown on a smaller screen).

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-18 10:35:14 -07:00
Adam Williamson
b455e04188 Add new cockpit needles for 205
The cockpit UI went all GRIMDARK so we need a couple of new
needles.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-16 17:10:28 -07:00
Adam Williamson
5f38971a78 Add some new variant layout indicator needles
Once again, the text is at a slightly different height to the
needles we already have. I do not know why this sometimes
happens. It's a mystery.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-16 16:34:55 -07:00
Adam Williamson
b55782358d Don't check version or pre-release note on banners
This reduces the coverage of the identification test a bit but
also *substantially* simplifies it. We run into a ton of problems
when we try to check the version and prerelease text on screens
where it appears on banners:

* The banners differ between variants
* The pre-release text is translated
* The banners have gradients so for RTL languages, even if some
  text is untranslated (e.g. 'Fedora 31') it appears on a
  different background color than on LTR languages
* The prerelease text is dark red; if it appears on a dark blue
  area of the banner this can trigger an os-autoinst needle
  comparison bug: https://progress.opensuse.org/issues/56822

All of this together means we wind up continually fighting these
checks and we have a whole forest of needles just for them, and
it doesn't seem worthwhile. So let's drop all the places where
we were checking version and prerelease on banners, and only
check them in two places where they appear on a grey background,
which avoids most of the problems (we just need one version
needle per release, and one prerelease needle per language).

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-16 11:08:24 -07:00
Adam Williamson
205c0c3d97 Add new variant ABRT needle for KDE app test
Font rendering changed.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-10 12:09:41 -07:00
Adam Williamson
6636122f62 New needle for *partly* translated 'install done' Japanese
Now one of the strings we check for is translated, but the other
is not...handle this, again with a workaround needle so the
missing translation triggers a soft fail.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-07 11:26:54 -07:00
Lukas Ruzicka
d82f4ecd7f Develop a new test for system updates via Cockpit.
* server-cockpit-updates tests that Cockpit can be used to update the system.
* server-cockpit-autoupdate tests that users can use dnf-automatic for system
updates.
* cockpit functions were removed from utils.pm and put into an extra library
for cockpit - cockpit.pm which all cockpit tests are now using.

Review cockpit.pm

Review autoupdate test.

Review the update test.

Fix typo in cockpit.pm

Add sleep.

Add missing command.

Delete an unused needle.
2019-10-01 16:04:01 +02:00
Adam Williamson
0782f4faf2 Add new akonadi migration agent needle for F31 background
Light blue throwing us off again.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-09-26 10:24:37 -07:00
Adam Williamson
61f8adefdf Add some new update-related needles for KDE
The new background seems to be throwing us off here. I think
the menu is *very slightly* transparent so the needles usually
work for most Fedora backgrounds as they tend to have a similar
base color, but the 31 one is unusually light.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-09-19 10:23:21 -07:00
Adam Williamson
6e9a7a57c5 Add several more variant keyboard layout needles for new icon
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-09-16 12:26:26 -07:00
Adam Williamson
96cf14d555 Add new variant us_rtl layout needle
For RTL languages the keyboard icon and language indicator are
the other way around, so we have different needles even for the
US layout indicator which is otherwise the same in all langs.
This is a variant needle for the new keyboard icon that showed
up in Rawhide recently.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-09-16 11:32:58 -07:00
Adam Williamson
0f9fd1a76a Add alternative vdb selected needle for new blue checkboxes
This checkbox sometimes renders slightly differently, especially
on non-x86_64. We're not really sure why, but we just add more
needles to handle it.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-09-16 11:23:47 -07:00
Adam Williamson
ea11c8344f Shrink KDE Firefox launcher icon to avoid background issues
Existing one doesn't match if the background is too different,
shrink it to just the center of the icon.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-09-11 15:12:03 -07:00
Adam Williamson
a1d9b14383 Some more prerelease needle updates 2019-09-11 15:01:44 -07:00
Adam Williamson
dc8ed1c9ba Yet more pre-release check fixes
We need another needle for French installs, and we need to use
ISO_URL for checking the ISO file name if ISO isn't set.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-09-11 14:31:54 -07:00
Adam Williamson
29a5e0bda7 Add variant 'permission denied' needle
For some reason, occasionally the 'm' character looks slightly
different. I have no idea why.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-09-11 10:47:13 -07:00
Adam Williamson
be1baa90e5 Update main hub needles for changed keyboard icon
Someone is pretty OCD about this damn keyboard icon, it just
keeps getting changed...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-09-09 11:44:28 -07:00
Adam Williamson
a3c207a045 Add a few more 'blue button' needle variants
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-09-05 18:09:14 -07:00
Adam Williamson
23ad087ed3 Add blue button variants for workstation_selected needle too
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-09-05 17:28:41 -07:00
Adam Williamson
cf6f8db3e7 Add new variant 'minimal selected' needle for radio button change
Radio buttons look different with recent GTK, it seems.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-09-05 16:14:59 -07:00
Lukas Ruzicka
1d6e2aaa19 Add changed needle that failed for cyrillic test. 2019-09-05 17:10:50 +02:00
Adam Williamson
df88196357 Add F31 background needle
Now we have the backgrounds!

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-09-04 12:44:55 -07:00
Adam Williamson
6d68b2c6fa Add yet another prerelease note needle
Seriously starting to think about separating out this ident
stuff into a separate test case :/

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-29 17:49:27 -07:00
Adam Williamson
53778fe494 Add language tags to the new language prerelease note needles
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-28 14:46:23 -07:00
Adam Williamson
b19e6bbda7 Add some needles for pre-release note translations
These showed up because I did a manual universal flavor run on
an ISO today, but we'll need them as soon as we do a candidate
compose too.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-28 14:14:57 -07:00
Adam Williamson
2cb224a184 Make generic topbar needle narrower
We did this for the orange (Server) one already, but not the
blue generic one.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-28 14:07:33 -07:00
Adam Williamson
26308a6b47 Update rarely-used installer ext4/xfs partitioning needles
These only get used when we run universal tests on a non-Server
image, which is pretty rare (these days, as Server DVD is a
critical image and composes fail if it fails, it only really
happens when we do it manually for some reason, like I did
today). So they get stale and aren't updated for font rendering
changes and stuff. As I said I had to do a run like that today,
so I had to update all these needles...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-28 14:04:13 -07:00
Adam Williamson
4bb89046d3 Add one more updated GNOME app test needle
'Simple Scan' renamed itself.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-26 17:51:20 -07:00
Adam Williamson
28443abf04 Add new variants for GNOME app start/stop tests
Several apps changed icon.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-26 16:29:37 -07:00
Adam Williamson
77a6c0ed4d Add yet another variant 'vdb selected' needle
This screen often seems to show slight variances in color,
really not sure why.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-26 16:28:44 -07:00
Lukas Ruzicka
022865ab9f KDE app start/stop: launch apps with commands not menus
This is doing slightly less to exercise the launcher menus and see
whether icons appear or disappear, but it's much faster and more
reliable. We do still use menu launch for one app, just to check
the mechanism works in general.
2019-08-23 12:41:49 -07:00
Adam Williamson
06e26e9721 Narrow topbar and add new version needle to cover languages
The top bar differs in non-English languages - for RTL the
version ident is on the left (so background is different), and
the width of the area with no text differs depending on the
length of the translations.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-21 14:36:53 -07:00
Adam Williamson
23552c777d Add a new KDE desktop update needle variant
Little black bits on the icon are in a different place now. Why,
I do not know.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-21 11:55:07 -07:00
Adam Williamson
ecfc9a5234 Add new variant abrt_runs needle for KDE app tests
Text rendering changed.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-21 10:44:25 -07:00
Adam Williamson
2f51f3ef20 Add PXE boot test
This adds a whole wodge of stuff to support_server to make it
act as a PXE server, then adds a new test which boots from PXE
and so should hit the PXE server. We use the NFS install repo as
that can be relied on to work for a support_server install.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-20 18:17:57 -07:00
Adam Williamson
0d66b4bbcd Add various new non-English language needle variants
Seems like there's been a change to freetype or something, text
rendering changed in all of these. Note there's one workaround
needle for an untranslated message in Japanese.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-19 19:03:57 -07:00
Adam Williamson
bc26017d84 Add needles for new anaconda icon
It's new!

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-19 17:36:41 -07:00
Adam Williamson
ca0f772120 Add some new variant needles for anaconda text changes
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-16 12:35:52 -07:00
Adam Williamson
469a1395d0 Revise desktop background checks
Split this out of install_default, because it really is not a
part of that test and we do not want that test to fail because
the desktop background is wrong. Make it its own test module
and test suite instead. Don't do it on Rawhide, because we
really can't assert anything worthwhile about Rawhide at the
moment at least (this means the test runs but is a no-op and
will always pass on Rawhide, unfortunately). Move the needles
to a more appropriate location (this has nothing to do with
anaconda) and use 'background' not 'wallpaper' naming (that's
the name we use elsewhere in the project, e.g. package names).
Also, run the test on updates, and add an F29 needle for this
purpose.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-14 14:30:25 -07:00
Adam Williamson
d9c45454b2 Clean up wallpaper needles a bit
The backgrounds we have are F30 backgrounds, not F31: there are
no F31 backgrounds yet, F31 images are using F30 backgrounds
(which is a bug we should file). Also we really only need one
F30 background needle to match both KDE and GNOME if we pick a
sensible area of the screen to use, and let's use one that has
a bit more contrast for safer matching.

Note: F30 background is *meant* to be animated, but in fact
neither GNOME nor KDE seems to use the animated version by
default. Which makes our lives easier! Sucks for whoever put in
the work to animate it, though.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-14 13:17:35 -07:00
Lukas Ruzicka
d06b30e8dc Add new testcases and methods for release identification. 2019-07-30 15:45:02 -07:00
Adam Williamson
b31422d188 Add some new KDE app test needle variants
Underlines on button text seem to have moved.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-07-30 15:43:42 -07:00
Adam Williamson
c1393ec2de Add missing akonadi_migration_agent needle
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-07-30 15:43:04 -07:00
Adam Williamson
3fe3fc453c Add new Cockpit log detail variant needle
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-07-25 14:11:12 -07:00
Adam Williamson
be12f07900 GNOME apps: dismiss new LibreOffice 'tip of the day' if it shows
Also add a new needle for Impress startup.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-07-23 20:15:24 -07:00
Adam Williamson
17438386c1 Add new 'install done' needle
Text on the button was changed in Rawhide.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-07-23 13:16:04 -07:00
Adam Williamson
9078bca871 cockpit: update needles for 198, rejig Services page handling
Cockpit 198 comes with a UI overhaul, so almost all needles
need an update.

The 'auditd' service is no longer on the first page. To make
this less fragile (at the cost of not testing that clicking on
a service actually opens the detail page *for that service*,
tweak the needles to just look for *any* running service, click
on it, and check we got to a 'details' page. We also redo the
existing needles for this design.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-07-10 11:43:03 -07:00
Adam Williamson
0c650ce424 Add new anaconda warning bar needle (color changed)
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-07-09 22:32:26 -07:00
Adam Williamson
0160b4dd4c Add yet another keyboard layout needle variant
Different distance between the text and the arrow. I dunno why.
GNOME is a mystery sometimes.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-07-09 22:31:30 -07:00
Adam Williamson
de4a58438b Use existing needle for dismissing KDE update notification
@lruzicka added this to the KDE app tests in 71c4e273, but there
was no need for a new needle as I'd already done the same thing
in the desktop updates tests; let's just use the same needle.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-07-03 08:39:01 -07:00
Lukas Ruzicka
71c4e273ef Click off updates notification if it appears. 2019-07-03 10:52:47 +02:00
Adam Williamson
dcc8fb9370 Update Firefox new tab needle
This should make the desktop_browser test a bit more reliable.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-06-28 14:48:42 -07:00
Adam Williamson
1b43fc2055 Move layout_us_ltr_overview from arabic to GNOME too
This is really a 'generic' needle since it'd be needed for any
RTL language test, it's not specific to arabic.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-06-25 15:09:00 -07:00
Adam Williamson
fa2fd70b1f Update a bunch of needles that have changed in Rawhide
Mainly because the GDM background became a lighter shade of
grey, for some reason, but also some dialog and icon changes.

Also put all forms of layout_us_ltr-gdm in the same directory.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-06-25 15:07:46 -07:00
Adam Williamson
bc2add845e Add the settings needle that provides the 'workspace' tag back
Removing this broke the GNOME apps test entirely because there
was no GNOME 'workspace' needle any more.

I don't like this needle much, we should probably use
check_desktop_clean or something instead. But for now let's just
put it back.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-06-25 14:16:11 -07:00
Adam Williamson
6656f1659d apps_activities is a dupe with overview_app_grid, kill it
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-06-25 14:01:39 -07:00
Lukas Ruzicka
9e3b16bfcf Change Settings test due to change functionality. 2019-06-21 14:54:18 +02:00
Lukas Ruzicka
868fca87b3 Add new needle for Simple Scan Menu item. 2019-06-21 11:33:12 +02:00
Adam Williamson
47bf561712 Update some KDE app launch icons
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-06-11 17:14:46 -07:00
Adam Williamson
7953fb91d1 Try to handle spurious 'auth required' on KDE update test too
Sigh, now *this* is showing up in KDE too. Fun.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-05-30 17:54:27 -07:00
Adam Williamson
2e56facb68 Try to handle changes to KDE update notifications in Rawhide
The way KDE does update notifications has changed - it's now a
permanent pop-up notification. This is a bit awkward for our
logic; it's hard to define a needle that proves this pop-up is
the only notification. Instead, let's dismiss it, then open the
notification tray and assert that there aren't any others. But
we also retain the old behaviour (more or less) for testing old
releases.

The popup notification also blocks the 'refresh' needle in the
systray and so breaks the desktop update test, so we deal with
that too.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-05-30 17:34:09 -07:00
Adam Williamson
8b353a216a Update some KDE app test needles for icon and font changes
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-05-30 14:40:32 -07:00
Adam Williamson
bad3263160 Update ABRT app launcher needle
Icon was changed.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-05-27 17:51:27 -07:00
Adam Williamson
ce7f5f9853 Add another new SDDM login needle
I cannot see how this varies from the other one, but os-autoinst
thinks they're different...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-05-27 17:50:40 -07:00
Adam Williamson
8109c49d5a Update KDE 'no notifications' needle
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-05-23 17:40:10 -07:00
Adam Williamson
a9da421d35 Update KDE login screen needle
We now skip straight to the password input stage, it seems...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-05-23 15:13:52 -07:00
Adam Williamson
b37ee51bfb Update a Firefox needle again
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-05-22 14:31:17 -07:00
Lukas Ruzicka
cc650ed195 Add needle to fix Mediawriter test. 2019-05-03 11:09:56 +02:00
Lukas Ruzicka
ddddc9d8eb Recreate needles to make Games working. 2019-05-02 17:35:31 +02:00
Adam Williamson
ed17620110 Add the new variant needle for realz this time
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-04-29 18:31:05 -07:00
Adam Williamson
c5e775a8d5 Whoops, wrong needle. REVERT! REVERT!
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-04-29 18:26:00 -07:00
Adam Williamson
aecd11d149 Add another Japanese needle variant
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-04-29 18:21:41 -07:00
Adam Williamson
9389acf4b0 Update a Japanese needle for slight font rendering change
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-04-29 11:14:03 -07:00
Lukas Ruzicka
8a321dec2c Fix broken needles for foreign installations. 2019-04-25 10:12:36 +02:00
Adam Williamson
d30714dbca Add a variant apps_stop needle for Firefox 66 without header
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-04-17 12:30:11 -07:00
Adam Williamson
382f815110 Update KDE menu_games needle for icon change
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-04-17 10:59:13 -07:00
Adam Williamson
e744cea358 Update GNOME apps needles for cheese and scan
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-04-16 18:46:31 -07:00
Guy Menanteau
9475fec56f New needle anaconda_blivet_vdb_selected-20190416
as previous needle only matched 96%
https://openqa.stg.fedoraproject.org/tests/520670#step/disk_custom_blivet_software_raid/36

Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2019-04-16 17:01:13 +01:00
Adam Williamson
c043fb6189 Add a Bash 5.0 variant of the noprofile shell needle...
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-04-12 16:05:12 -07:00
Adam Williamson
33f63c3d95 rescue_shell_prompt is really just a root console needle...
...so make it look like all the other ones we have already.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-04-12 15:54:26 -07:00
Guy Menanteau
29070a54f1 New install_repository_hd_variation
use a new PREINSTALL to execute a rescue script before install.

This is covering the  Hard_drive_variation test as per:
https://fedoraproject.org/wiki/QA:Testcase_install_repository_Hard_drive_variation

Signed-off-by: Guy Menanteau <menantea@linux.vnet.ibm.com>
2019-04-12 15:49:54 -07:00
Lukas Ruzicka
99f93945b3 Change KDE desktop background at start of KDE app test
Since the KDE menus have transparency set, any time the wallpaper
changes the menus will look different, and the app tests may
fail. This sets the desktop wallpaper to black at the start of
the test suite to avoid this problem.
2019-04-09 08:57:43 -07:00
Lukas Ruzicka
9a2d9ea190 Create needles needed for firefox test to really work. 2019-03-28 14:18:51 +01:00
Lukas Ruzicka
f9201fb7d8 Fix kmail and kontakt tests
Fix to pass kontakt tests properly.
2019-03-27 18:00:05 +01:00
Adam Williamson
e396341f31 Handle 'authentication required' appearing on Boxes startup
In the last few weeks Boxes is showing an auth dialog on start.
I've filed a bug on this; let's have the test handle it as a
soft failure, since this isn't a fatal problem. Do this by
making an existing needle for this dialog a bit more generic
in name and using it.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-03-26 12:30:47 -07:00
Adam Williamson
5a1a71c67f Use the existing 'oh hai this is firefox' needle for app tests
The KDE and GNOME 'apps' tests for Firefox both invented their
own 'oh look Firefox is running' needle, even though we already
had one. The GNOME one was broken by the removal of the app
title bar in Firefox 66.

Instead of having three needles for the same thing, let's just
throw the 'apps' ones out and use the pre-existing one from
needles/firefox for all cases.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-03-26 12:23:45 -07:00
Adam Williamson
846c2969a4 Update KDE 'addon install' Firefox needle
This looks different in KDE to GNOME for some reason, so we need
yet another new needle.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-03-25 13:16:23 -07:00
Adam Williamson
7ebe604135 Update Cockpit needle for upstream change
Looks like this label went from regular to bold font in a new
cockpit release.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-03-25 08:30:47 -07:00
Lukáš Růžička
6fc8c99515 Add needle to fix iscsi test. 2019-03-21 13:18:41 +01:00
Adam Williamson
26cdcb8d3c Add several new needles for changed GNOME app icons
Utilities came up different *again* in a Rawhide test today, so
this adds a needle which matches on the text. Usually we prefer
image matches to text, but this image seems to keep changing...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-03-20 12:21:55 -07:00
Adam Williamson
5195a20ab5 Add a variant needle for juk
This button can have the O underlined, apparently...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-03-20 12:10:13 -07:00
Lukas Ruzicka
9ae60de422 Reneedle menus and fix some issues in tests.
This update fixes the strange behaviour in KDE which causes wrong clicking.
With new logic, this is not an issue anymore.
2019-03-20 10:26:33 +01:00
Lukas Ruzicka
d4403962da Delete unused needles. 2019-03-19 12:15:03 -07:00
Lukas Ruzicka
c9a25cb2a6 Create an application start-stop test for KDE.
Add png files as a background for the needles.

Rewrite the test handling methods to make them ready for KDE

Add the ABRT startstop test.

Make changes and corrections to the abrt test.

Add test for Firewall.

Add test for DNF dragora.

Add wait time for dnfdragora test.

Correct syntax.

Add Language test.

Make some changes to the DNF dragora test.

Add Users test case.

Add needles for DBUS viewer test.

Add Dbusviewer test.

Add Mahjong test and needles for games.

Add Minesweeper tests.

Add Patience test.

Add test for Document Viewer.

add test for gwenview

Add test for koulourpaint.

Add test for Kruler

Add test for Kcolorchooser

Reneedle failing needle.

Add ktorrent tests.

Add tests for CPT editors.

Add test for Krfb

Fix names for those files.

Add test for Kget

Add Akregator test.

Add test for Konversation.

Make Konversation end really.

Add tests for Kmail

Add test for PIM exporter.

Add test for KTnef and Krdc.

Fix problems after test runs.

Make more tries.

Fix needle to be found better.

Fix more errors.

Add test for Falkon.

Add tests for browsers.

Add support for closing tabs into Firefox test.

Add tests for K3B

move needles to correct directory

Add Kaddressbook test.

Add Kontakt text.

Add test for korganizer.

Add menu office needle and correct konqueror needle.

Add test for calligra stage.

Add test for Calligra.

Add test for network connections.

Modify needle for kaddressbook to prevent failing.

Add test for system settings and fix others.

Add test for FMW.

Add test for Dolphin

Add test for Infocenter.

Add test for kparted.

Fix a wrong needle.

Test relnotes.

Fix some errors in tests.

Add test for Discover.

Add test for Ksysguard.

Add tests for Konsole.

Add tests for KDE wallet.

Add tests for several utilities.

Add Krusader test.

Finish utility tests.

Fix some errors.

Fix needle for spectacle.

Add wait time to let Dragora wait for network.
2019-03-19 12:14:47 -07:00
Adam Williamson
29b809376c Add yet more Firefox needle variants
I don't know why the ones I *just created* don't always match,
but they don't. Let's just have a bunch of needles and in a few
weeks I'll look and see if any are never matching, and get rid
of them.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-03-19 11:46:06 -07:00
Adam Williamson
fa7e674e6e Update a couple of Firefox needles
In latest Branched, 'Close Tabs' button has an upper-case T, and
the addon install button is a darker blue.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-03-19 10:16:11 -07:00
Adam Williamson
0365e2f18d Add another utilities needle variant
Dunno why we need yet *another*, but even the new one didn't
match on the most recent Branched compose. For now just add
another, we can look in a few weeks which can be thrown away.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-03-19 10:13:44 -07:00
Adam Williamson
1ed87382a9 abrt back under utilities, drop sundry
As of Fedora-30-20190316.n.1, gnome-abrt is back under the
Utilities menu and the 'Sundry' menu seems to have disappeared.
Change the test back to using utilities and drop the sundry
needle.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-03-19 10:11:32 -07:00
Lukas Ruzicka
bbb2ac06b8 Add new needle to cover changed utilities. 2019-03-18 15:27:44 +01:00
Adam Williamson
2e75cfe5db Add new variant Firefox certificate error needle
It looks different in Fedora 30 update tests for some reason.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-03-08 07:47:33 -08:00
Adam Williamson
b35492e8b5 Add variant GNOME Software refresh needle
desktop_update is failing on F30 update tests because this icon
looks a bit different...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-03-05 08:13:20 -08:00
Adam Williamson
86fe913b84 Soft fail workaround the 'upgrade hangs at end' bug (#1674045)
We want to get accurate results from the rest of the upgrade
test and we know about this bug, so let's make it a soft fail
so we can see how the test is functioning otherwise.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-03-04 18:07:04 -08:00
Adam Williamson
f89e08e16d Add some new needle variants that failed ppc64 tests
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-03-01 08:52:25 -08:00
Adam Williamson
072333a0f7 Update needle for apps 'System Monitor'
Icon changed.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-02-25 11:30:19 -08:00
Adam Williamson
855e54d8df Refresh workaround for occasional FreeIPA web UI cert issue
Investigation suggests that the Firefox certificate validation
failure we sometimes get when loading the FreeIPA web UI in the
freeipa_webui test module is a race issue in Firefox itself:

https://bugzilla.mozilla.org/show_bug.cgi?id=1530429

refreshing the page seems to work around the problem, so let's
do that.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-02-25 10:13:33 -08:00
Adam Williamson
965722e6b3 Add new variant 'Getting Started' needle
The 'largetext' version of the needle is needed when there's
some notification displaying over the window title, it seems
F28's current font rendering doesn't match the current version
of that needle...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-02-22 08:40:50 -08:00
Adam Williamson
945a93aa05 Get more info on failures caused by Firefox certificate issues
We're getting an intermittent case where FreeIPA tests fail
because of a web server certificate issue. Click 'Advanced' in
Firefox when this happens so we can get a bit more info on the
problem.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-02-21 12:16:42 -08:00
Lukas Ruzicka
690788f107 Reneedle the test suite
This is another take of needles to make the suite work again.
2019-02-21 12:29:36 +01:00
Lukas Ruzicka
1e8834825d Add new set of needles (first run)
These needles fix the test suite after the looks has been changed.
2019-02-20 15:52:38 +01:00
Adam Williamson
fc1c8185f0 Add updated graphical encryption passphrase needle
The screen appearance changed in Rawhide, new needle needed.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-02-15 12:10:36 -08:00
Adam Williamson
c1a85c0d72 Add updated grey KDE network notification needle
Same font change as blue.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-02-14 12:35:53 -08:00
Adam Williamson
965a1a6546 Click instead of using ctrl-t for new browser tab
ctrl-t seems to be failing often in Firefox tests, and I can't
figure out why. Let's try clicking the 'new tab' button instead
as a workaround.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-02-13 09:35:32 -08:00
Adam Williamson
1bbb62af32 Drop GDM no logo needle as the logo is back in 0212.n.1...
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-02-13 00:19:47 -08:00
Adam Williamson
a2a51ca41c Add various new needle variants for Rawhide changes
A bunch of stuff, probably mostly more GTK+ icon theme changes.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-02-13 00:15:50 -08:00
Adam Williamson
42fb8491a2 Re-do the GDM no logo needle
Using translated text was a poor choice...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-02-11 16:29:30 -08:00
Adam Williamson
9cae91cd89 Add updated needles for Adwaita icon theme changes
The Adwaita icon theme got some changes in a git snapshot build
for Rawhide recently. These changed icons break various needles,
we need new ones.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-02-11 13:12:13 -08:00
Adam Williamson
200755814f Add variant GDM needle for missing Fedora logo
Fedora logo is missing from GDM in today's Rawhide. Not going
all the way and making this a soft fail as the bug is known and
should be fixed soon, I'll take the needle out again then.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-02-11 13:10:46 -08:00
Adam Williamson
0b5dd623aa Add updated KDE blue network notification needle variant
Text display changed here.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-02-11 13:03:15 -08:00
Adam Williamson
809f291a1f Add a couple of needles needed for live image tests (F28)
Forgot to include these in the live image test commit.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-02-08 08:35:35 -08:00
Adam Williamson
578a254cca Add GTK+ 3.24.5 desktop update needle
Didn't notice it in the big retake.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-02-07 10:12:27 -08:00
Adam Williamson
5cbc7df810 Work around GGO gnome-software/582 till it's fixed
We're getting a spurious "Authentication required" for a
background refresh of the system repos sometimes, depending on
whether it happens while we're at a tty. This is now accepted
by upstream, so let's handle it and make it a soft failure for
now.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-02-07 09:58:10 -08:00
Adam Williamson
bd4edb0da7 GTK+ 3.24.5 needle retake
New GTK+ changed something so the background of many interface
elements is a slightly lighter grey, this broke a bunch of
needles. Here are the retakes. Includes one not-strictly-related
SDDM update and a rename of the one @lruzicka did to match the
others.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-02-06 13:27:30 -08:00
Lukas Ruzicka
b182edf697 Add new needles to make tests work
OpenQA tests were choking on _boot_to_anaconda, because
the rawhide_accept_fate was not recognized. I created
a new needle to make it work.
2019-02-05 15:46:22 +01:00
Lukas Ruzicka
be3aadc249 Change the target of the needle for Clocks in StartStop test
The needle could sometimes fail because the item it was looking
for only appeared in a limited set of situations. The new
location should be safer to check and the test should not
fail as a result.
2019-01-29 10:48:28 +01:00
Lukas Ruzicka
83ff7a0679 Delete Evolution needles.
Evolution has been removed from the default installation of Workstation
so the test is no more needed.
2019-01-28 14:54:58 +01:00
Lukas Ruzicka
4288041889 Use the current date instead f30 as a filename for new needles. 2019-01-28 14:53:59 +01:00
Lukas Ruzicka
e72a800fd1 Add new needles.
This commit adds new needles to reflect the changes which
have been made to the Desktop. It also now FIXES #87.
2019-01-28 11:11:36 +01:00
Lukas Ruzicka
c97c23d485 Recreate needles to make them check different area.
This commit adjusts the area that needles look for to avoid checking
based on a menu item that has disappeared from the system.
2019-01-28 11:11:36 +01:00
Adam Williamson
b6cf28715a Add new needles for Cockpit 186 UI redesign
Some bits of Cockpit's UI are changed in 186, we need a few new
needles.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-01-26 23:31:43 +01:00
Adam Williamson
12affb145f Add update tests to build and test a netinst image
This adds a test which builds a netinst image potentially with
the package(s) from the update, and uploads that image. It also
adds a test which runs a default install using that image. This
is intended to check whether the update breaks the creation or
use of install images; particularly this will let us test
anaconda etc. updates. We also update the minimal disk image
name, as we have to make it bigger to accommodate this test,
and making it bigger changes its name - the actual change to
the disk image itself is in createhdds. We also have to redo a
bunch of installer needles for F28 fonts, after I removed them
a month or so back...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-01-18 08:24:44 -08:00
Adam Williamson
536f699013 Post fail: handle landing in dracut shell, upload rdsosreport
If a test fails to the dracut shell, we currently don't do
anything useful. This should recognize when that happens, and
upload rdsosreport.txt.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-01-16 11:24:06 -08:00
Lukas Ruzicka
39d3427471 Create a test suite for application start stop testing.
Merges #86
Fixes #85
2019-01-04 15:23:03 -08:00
Adam Williamson
2f1570d46b Tweak firefox_close_tabs needle match
So it matches when the button has an 'active' indicator...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-01-02 20:07:37 -08:00
Adam Williamson
ea867ef431 Great Stale Needle Cleanup 2018: everything else
It's my second least favorite day of the year again: Stale
Needle Cleanup Day!

This should get rid of all non-anaconda needles that definitely
are not being used any more. Cleanup of all anaconda needles was
in the previous commit.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-17 17:07:15 -08:00
Adam Williamson
ad29a8fcde Great Stale Needle Cleanup 2018: anaconda
It's my second least favorite day of the year again: Stale
Needle Cleanup Day!

This should get rid of all anaconda needles that definitely are
not being used any more. A few borderline cases (where I'm not
100% sure if they may still be useful in odd corner cases, like
running universal tests on non-server images, and runs outside
of the US, and stuff) are kept around.

Cleanup of all non-anaconda needles will be in the next commit.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-17 16:30:57 -08:00
Adam Williamson
44c92e7c7e Remove a stray, no-longer-used needle
This was part of a workaround for a Fedora Atomic Workstation
(old name for Silverblue) issue that got resolved a while back;
we forgot to take the needle out.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-17 14:41:29 -08:00
Adam Williamson
ef3555aaa1 Drop another old workaround (KDE package signature check)
It seems Rawhide auto-signing is working fine now: openQA claims
this needle has 'never' been matched (which really means 'not
for a long time'), and I can't find any test fail in the last
year which looks like it landed on this 'authenticate' screen
but the needle failed to match, or anything like that.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-17 12:24:09 -08:00
Adam Williamson
666cac410a Add another variant KDE notification needle
More font size changes.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-13 16:01:17 -08:00
Adam Williamson
5f014de08a Add new KDE desktop notification needle variant
Font got bigger, or smaller, or something.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-13 14:22:59 -08:00
Adam Williamson
fe49095901 Add a couple of new Japanese needles
Translations changed.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-12 22:16:57 -08:00
Adam Williamson
65b4755358 Simplify anaconda error screen handling, add new needle
There's really no point having separate error and error_report
needles. Just match on error_report as well as clicking on it.
Also add a new error_report needle for latest Rawhide fonts.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-12 22:06:20 -08:00
Adam Williamson
0eaddd1215 Add some new variant needles for the FreeIPA web UI
Looks like they changed capitalization...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-11-15 18:22:35 -08:00
Adam Williamson
a10867d4fc Add new variant French 'install done' needle
Translation changed again, I think.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-11-15 12:47:50 -08:00
Adam Williamson
5a2e8f20a7 Add new variant French Rawhide accept fate needle
Translation changed.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-11-09 17:42:48 -08:00
Adam Williamson
d76003a6fb Add a new needle for the KDE desktop notifications test
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-11-09 17:41:49 -08:00
Lukas Ruzicka
b999ee93fe Add new needles to utilize Arabic tests (Fixes #83).
Add needles to utilize the Arabic test.

Add rest of the needles.
2018-11-01 15:30:03 +01:00
Lukas Ruzicka
8190c2a0eb Add needles to utilize the Cyrilic installation test. 2018-11-01 14:48:03 +01:00
Adam Williamson
fd753b2e3a Handle split of 'download' and 'apply' phases in gnome-software
GNOME Software 3.30.5 split the offline update process into two
separate 'download' and 'apply' phases. So we need to handle
clicking 'download' before 'apply', if that happens.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-10-31 11:50:14 -07:00
Adam Williamson
e6c8c5f0ff Work around Firefox 'close multiple tabs' warning
Somehow, recently, FreeIPA tests are running into Firefox not
quitting because it's showing a warning about closing multiple
tabs. (I think we didn't *get* multiple tabs before but now we
do, for some reason). So let's work around this by clicking
"Close tabs" if the warning appears.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-10-30 18:34:37 -07:00
Adam Williamson
86eaf0cd48 Remove no-longer-used 'uptodate' needle
Since we tweaked the g-s workaround not to use this any more, we
don't need it.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-10-12 22:11:35 -07:00
Adam Williamson
0f1d0a1228 Add new Japanese needles
Seems the installer Japanese font changed.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-10-12 21:01:07 -07:00
Adam Williamson
17911efcce Add a new variant needle for GNOME reboot confirm screen
Seems none of the existing needles matches on current F28 update
tests.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-10-11 16:44:41 -07:00