Commit Graph

2219 Commits

Author SHA1 Message Date
Adam Williamson
ec762117d0 Wait an extra 2 minutes for the installer 'beta nag' screen
There seems to be an issue in Rawhide ATM which can cause the
'beta nag' screen to take a very long time to appear. Bump the
timeout to avoid tests failing on this.

https://bugzilla.redhat.com/show_bug.cgi?id=1666112

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-01-14 15:22:01 -08:00
Adam Williamson
9df8be32cb Upload dbus.log on anaconda failure
This log file was quite recently added to anaconda, let's upload
it with all the others.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-01-14 12:57:38 -08:00
Adam Williamson
a1bb5bd1f7 Drop an update test hotfix (no longer needed, breaks F28)
The packages from this update seem to be breaking F28 update
tests for some reason; a later update has gone stable anyway, so
this is no longer needed.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-01-07 18:13:21 -08:00
Lukas Ruzicka
39d3427471 Create a test suite for application start stop testing.
Merges #86
Fixes #85
2019-01-04 15:23:03 -08:00
Adam Williamson
c6fac00698 More fixing of the workaround...
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-01-04 13:55:31 -08:00
Adam Williamson
42a38009eb Fix workaround (have to wait for desktop to appear)
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-01-04 13:27:01 -08:00
Adam Williamson
4e537684b1 Work around #1663040 in Workstation live installs
We don't want the tests to fail on this now we know what the
bug is, really - we want to find if there are any subsequent
fails, and allow the post-install tests to run also. So, let's
make it a soft failure.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-01-04 13:01:36 -08:00
Adam Williamson
141f29c7cc fix syntax of advisory_check_nonmatching_packages in post-fail
You...can't do this like that.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-01-03 11:35:40 -08:00
Adam Williamson
2f1570d46b Tweak firefox_close_tabs needle match
So it matches when the button has an 'active' indicator...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-01-02 20:07:37 -08:00
Adam Williamson
ca6e1105c1 Work around RHBZ #1663050 (tty quitting on exit from startx)
A bug showed up in Rawhide where, when you run startx in a tty,
when you exit that X session, the tty quits and returns to the
login prompt. This is a slightly sloppy workaround for that
problem.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-01-02 13:53:31 -08:00
Adam Williamson
2b29ecbc18 FreeIPA client: set longer timeout for login(1) (#1661273)
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-20 09:02:36 -08:00
Adam Williamson
2d63bd9702 Bump HDDSIZEGB to 13GiB for language tests
12GiB seems not to be enough for aarch64.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-18 11:05:10 -08:00
Adam Williamson
e9642eace5 Give console login even longer on aarch64 for #1644919
Even an extra 30 secs doesn't seem to be long enough for the
aarch64 tests.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-18 11:01:35 -08:00
Adam Williamson
ea867ef431 Great Stale Needle Cleanup 2018: everything else
It's my second least favorite day of the year again: Stale
Needle Cleanup Day!

This should get rid of all non-anaconda needles that definitely
are not being used any more. Cleanup of all anaconda needles was
in the previous commit.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-17 17:07:15 -08:00
Adam Williamson
ad29a8fcde Great Stale Needle Cleanup 2018: anaconda
It's my second least favorite day of the year again: Stale
Needle Cleanup Day!

This should get rid of all anaconda needles that definitely are
not being used any more. A few borderline cases (where I'm not
100% sure if they may still be useful in odd corner cases, like
running universal tests on non-server images, and runs outside
of the US, and stuff) are kept around.

Cleanup of all non-anaconda needles will be in the next commit.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-17 16:30:57 -08:00
Adam Williamson
44c92e7c7e Remove a stray, no-longer-used needle
This was part of a workaround for a Fedora Atomic Workstation
(old name for Silverblue) issue that got resolved a while back;
we forgot to take the needle out.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-17 14:41:29 -08:00
Adam Williamson
3d35ec9670 Update a comment to not mention a removed workaround
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-17 14:36:10 -08:00
Adam Williamson
00d1fdbd2c Drop #1600823 workaround
Bug was fixed months ago.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-17 14:30:09 -08:00
Adam Williamson
f40599ee15 Drop all #1594402 workarounds
I'm pretty sure we got all the bugs this was working around
fixed. Again, if not, we can put this back!

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-17 14:27:18 -08:00
Adam Williamson
01fd92cb34 Drop #1444225 workaround
The bug never got explicitly addressed, but it's very old and
anaconda has been substantially changed since. The workaround
sometimes triggers erroneously now (because the icon sometimes
goes black while the spoke is still in 'Checking storage
configuration...' state), which is awkward. I can't be 100% sure
the bug doesn't sometimes still happen, but if it does, we'll
notice fairly soon, and we can tweak this and put it back.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-17 14:18:05 -08:00
Adam Williamson
d6de57c6de Drop RHBZ#1618928 workaround
Bug was fixed back in August.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-17 14:10:58 -08:00
Adam Williamson
451f35feec Remove another case of RHBZ#1622254 workaround
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-17 14:04:53 -08:00
Adam Williamson
777de30179 Remove RHBZ#1622254 workaround
Fixed since 2018-09-12.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-17 13:47:46 -08:00
Adam Williamson
7cbd859d81 Remove another old workaround (for #1606541)
This was fixed long ago as well...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-17 13:36:13 -08:00
Adam Williamson
e04ee00e19 Drop various old workarounds from _support_server
I tested a run with these commented out, it worked fine. Seems
the bugs are all fixed now.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-17 13:06:19 -08:00
Adam Williamson
ef3555aaa1 Drop another old workaround (KDE package signature check)
It seems Rawhide auto-signing is working fine now: openQA claims
this needle has 'never' been matched (which really means 'not
for a long time'), and I can't find any test fail in the last
year which looks like it landed on this 'authenticate' screen
but the needle failed to match, or anything like that.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-17 12:24:09 -08:00
Adam Williamson
1fd0097d1d Simplify an 'if >F26' thing
We don't care about anything older than 27 any more.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-17 12:19:44 -08:00
Adam Williamson
d1e7b89efd Fix a potential race in desktop update test
https://openqa.stg.fedoraproject.org/tests/424393 is a failure
where the 'Download' [updates] button was already visible when
we went to the tab. We already checked whether an 'apply' button
is visible and skipped the 'refresh' click if so, but because
the 'download' button is a new thing, we weren't skipping the
'refresh' click if 'download' was already visible.

So in this case, even though we could already see 'download', we
went ahead and clicked 'refresh'...then *immediately* started
looking for 'download'. It seems that Software did not refresh
and remove the 'Download' button *immediately* when we pressed
'refresh' - it left the 'Download' button visible briefly, and
*in this brief window*, we clicked it. *Then* Software kinda
'noticed' we'd clicked 'Update', and it seems it just sort of
throws away our click on 'Download' at that point and does the
refresh.

So at that point, the test thinks it's clicked 'Download' and
expects to see 'Apply', but actually the 'Download' click got
more or less thrown away, so the test fails, sitting at the
'Download' button.

To solve this, let's just extend the existing check to skip the
'refresh' click if 'download' *or* 'apply' are already visible.

There is a sort of possibility here that we could wind up
downloading and installing some updates that existed and were
noticed *before* we did our python3-kickstart trick, but not
install the python3-kickstart update, and cause the test to fail
because of that, but that doesn't seem to have happened before
when we were seeing the 'update' button, so I think I'm not
going to borrow trouble. If it happens, we'll deal with it I
guess.

The comment talks only about KDE, but clearly it can be the case
that an automatic check makes the button visible on GNOME too,
so let's rewrite the comment too.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-17 12:10:06 -08:00
Adam Williamson
517750443e Remove RHBZ #1638563 workaround
The fix for this bug was sent to all releases now, so we should
not need the workaround any more. Let's kill it.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-17 12:02:57 -08:00
Adam Williamson
b171c0a1c4 Try to fix the 'matching update package' check for kernel
That clever-clever 'check the packages from the update were
installed' thing from yesterday breaks on kernel updates, as
they're installonly; after the update, the new version of the
package is installed, but the *old* version is too, and the way
I implemented the check, it treats that as a failure. Let's try
and handle this a somewhat-clever way (if this fails, I'm just
going to grep out lines with 'kernel' in them, as a *dumb* way).

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-13 18:52:51 -08:00
Adam Williamson
4629e5b740 Fix console keyboard layout in installedtest post_fail_hook
This should fix log collection when a French or Japanese test
fails before the test itself would have done this.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-13 16:29:53 -08:00
Adam Williamson
666cac410a Add another variant KDE notification needle
More font size changes.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-13 16:01:17 -08:00
Adam Williamson
0498f4db94 Use 30s timeout on the check_screen in the workaround
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-13 15:29:35 -08:00
Adam Williamson
b3f51e6108 Gah come back missing bracket
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-13 15:23:50 -08:00
Adam Williamson
1f7e5ebe20 Attempt a workaround for RHBZ#1659266
This is breaking the memory_check tests. I just reproduced it
manually and the UI *does* come back to life if you wait some
time; let's see if we can work around the bug this way.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-13 15:15:49 -08:00
Adam Williamson
5f014de08a Add new KDE desktop notification needle variant
Font got bigger, or smaller, or something.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-13 14:22:59 -08:00
Adam Williamson
12e103e3da Factor meat out of advisory_post and do it in postfail too
If an update test fails before reaching advisory_post, we don't
generate the 'what update packages were installed' and 'were
any update packages *not* installed when they should have been'
logs, but these may well be useful for diagnosing the failure -
so let's also do the same stuff there. Only let's not do it all
twice.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-12 22:17:29 -08:00
Adam Williamson
764c6dbd95 Notice when update package should have been installed but wasn't
We hit an interesting case in update testing recently:

https://bodhi.fedoraproject.org/updates/FEDORA-2018-115068f60e

An earlier version of that update failed testing. When we dug
into it a bit, we found that the test was failing because an
earlier version of the `pki-server` package was installed than
the version that was in the update; when asked (as part of
FreeIPA deployment) to install it, dnf had noticed that there
were dependency issues with the version of the package from the
update, but it happened to be able to install the version from
the frozen 'stable' repo...so it just went ahead and did that.

In this case, the 'missed' package resulted in a test failure,
but it'd actually be possible for this to happen and the test
to complete; we really ought to notice when this happens, and
treat it as a test failure.

So what this attempts to do is: at the end of all update tests,
check for all installed packages with the same name as a package
from the update, and compare their full NEVR to the one of the
package from the update. If a package with the same name as one
of the update packages is installed, but does not appear to be
the *same NEVR*, we fail, and upload the lists of packages for
manual investigation as to what the heck's going on.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-12 22:17:29 -08:00
Adam Williamson
fe49095901 Add a couple of new Japanese needles
Translations changed.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-12 22:16:57 -08:00
Adam Williamson
59ec361700 Temporarily disable modularity_update_norepo subtest
This is known to not work at present as it's not implemented in
DNF yet; failing on it every day isn't proving much. Once it's
implemented (expected by end Jan) we can turn it back on again.
See https://bugzilla.redhat.com/show_bug.cgi?id=1616167

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-12 22:13:20 -08:00
Adam Williamson
65b4755358 Simplify anaconda error screen handling, add new needle
There's really no point having separate error and error_report
needles. Just match on error_report as well as clicking on it.
Also add a new error_report needle for latest Rawhide fonts.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-12 22:06:20 -08:00
Adam Williamson
0639468de6 Use -l for systemctl status (to avoid ellipsization)
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-12 12:36:21 -08:00
Adam Williamson
cb035c7737 Still fixing up this serial logging stuff
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-12 12:28:23 -08:00
Adam Williamson
95b227b97a Sigh, fix a syntax error in previous commit
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-12 11:57:53 -08:00
Adam Williamson
0f5281f389 In post_fail_hooks, try using serial line if no network
Sometimes we get a test failing because the SUT isn't connecting
to the network for some reason. In this case we never get any
logs, because `upload_logs` relies on being able to reacht at
least the worker host system via the network.

This attempts to detect when we can't ping the worker host, and
in that case, send some info out over the serial line instead.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-12 11:40:58 -08:00
Adam Williamson
2f2f0973ec Update shrink disk image names in templates
We weren't using the gpt-labelled one, so I took it out of
createhdds; that makes the name of the mbr-labelled one lose
its label tag, so we need to update that name here.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-01 09:05:48 -08:00
Adam Williamson
e76ad4bbc1 Crank sss debuglevel up to 9, and also do it for cockpit test
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-11-30 14:58:53 -08:00
Adam Williamson
25d8b56fe9 Drop 'get_milestone' from utils (it's unimplemented and unused)
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-11-30 14:22:08 -08:00
Adam Williamson
56060ff8bd Simplify and improve how we get to 'linux' line in grub
That whole creaky edifice of conditionals that figured out how
many times to press 'down' was a mess I always hated, and I just
found out that the fix for BLS wasn't complete - I'd assumed in
writing it that systems weren't being migrated to BLS on upgrade
to F30, but actually they are. This makes that design very hard
as we'd have had to find a way to change the number of 'down'
presses part-way through update tests, and all the ways I can
think of to do that would've made this even sillier.

Happily I managed to come up with what looks like a much simpler
approach: just go from the bottom. It seems that in every setup
I can think of to check - all three arches, BLS, no BLS, pre-
install, post-install - the linux line is two lines up from the
bottom of the config stanza (the last line is blank, and the
last line but one is the initramfs line). So we can just press
down 50 times (to make damn sure we're at the bottom) then press
up twice and we should be in the right place, no matter the arch,
the release, or if BLS is in use or not. Whew.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-11-30 14:17:01 -08:00
Adam Williamson
0d54e222f7 Pull in the update that fixes the FreeIPA / authselect bug
This bug is breaking all update FreeIPA tests; until the updates
go stable, let's pull them in to update tests so the results
are useful.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-11-28 17:42:58 -08:00