Commit Graph

288 Commits

Author SHA1 Message Date
Adam Williamson
8832c5b81e Add FEDORA-2020-5f05c3ec46 as a workaround to fix FreeIPA upgrade
FreeIPA F31 -> F32 upgrade test is currently failing because
a new pki-core hit F31 stable but not F32 stable yet. It can't go
backwards on upgrade, that breaks stuff. The F32 update has been
pushed stable but just hasn't made mirrors yet as the last F32
nightly compose failed, so let's add it to the workarounds for
now.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-03-17 08:18:11 -07:00
Adam Williamson
d9438e8042 Drop workaround updates that have been pushed stable
Except one that's pushed stable but hasn't made repos yet (as the
last F32 nightly compose failed).

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-03-17 08:17:00 -07:00
Adam Williamson
5fdf0041cf Bump another blivet timeout
Seems 5 seconds isn't long enough to wait here on aarch64, the
previous dialog hasn't always cleared by then. See e.g.
https://openqa.stg.fedoraproject.org/tests/753802

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-03-11 17:43:23 -07:00
Adam Williamson
3444d2f959 Try and fix blivet test fails caused by too-early OK click
We seem to quite often get a failure in the blivet_lvmthin test
here which seems to be caused by trying to click 'OK' while the
'Device type' menu is still changing state or something. Let's
throw in a little delay.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-03-11 17:22:26 -07:00
Adam Williamson
5c6ee14ed3 Add F32 backgrounds update as an update test workaround
So at least GNOME background test passes.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-03-06 21:51:32 -08:00
Adam Williamson
e7c6501737 Extend update test workaround mechanism to upgrade tests
To cover cases like #1767351 where we need to apply a workaround
to the pre-upgrade environment.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-03-06 19:21:44 -08:00
Adam Williamson
9df8fdb00e Drop workaround package that's gone stable
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-03-06 12:54:16 -08:00
Adam Williamson
884f71bb81 Revert "Workaround #1805553 (single disk not selected as install target)"
This reverts commit adf3f91818.
The bug has been fixed in anaconda, so we can drop this, which
is good as it has timing issues producing false positives on
Rawhide...
2020-02-26 14:30:08 -08:00
Adam Williamson
3d29d7b746 Add efivar update for F32 as workaround to fix UEFI update tests
This is pending stable, but looks like the update push won't
happen for a few hours, so I'm adding it as a workaround so we
can re-run the tests and get them to pass.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-26 12:48:17 -08:00
Adam Williamson
8e43dfa304 Drop F31 workaround that's gone stable, add F32 lmod workaround
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-25 17:40:22 -08:00
Adam Williamson
adf3f91818 Workaround #1805553 (single disk not selected as install target)
Rawhide seems to have developed a bug where a single disk is no
longer automatically selected as the install target when you
enter the INSTALLATION DESTINATION spoke. We need to work around
this (but register it as a soft failure).

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-20 17:41:34 -08:00
Adam Williamson
1eb4e3dca5 Add perl syntax check test, add it to CI
Inspired by openQA's 01-compile-check-all.t, this adds a perl
test which checks the syntax of main.pm and all lib and test
files, and hooks it up to CI. Requires os-autoinst and
perl-Test-Strict.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-13 15:28:09 -08:00
Adam Williamson
66e53feb4d Update workarounds (drop update now stable, add F31 libdnf fix)
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-02-05 02:51:37 -08:00
Adam Williamson
0a4a787c03 Give Firefox longer to start
It seems to take a long time sometimes for some reason. Can't
pin it down but it's causing test flakes, so let's just let it
be. It *may* happen when chrony adjusts the system clock just as
Firefox is starting, for some reason.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-22 10:40:46 +01:00
Adam Williamson
d236c48152 Doh, fix stray quote mark from last commit
Where did I leave my brown paper bag...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-16 18:57:17 +00:00
Adam Williamson
1098450fd6 Pull in F30 Cockpit 210 update as a workaround
Otherwise the new cockpit main screen needle match fails, because
'machine' is not on the top part of the screen.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-16 18:51:40 +00:00
Adam Williamson
ba40ba3e4a Drop #1663050 workaround (fixed a while ago)
This block is kinda weird, but I don't want to fiddle with it
any more right now. I don't know why we do this check_screen
exactly like this.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-10 14:41:41 -08:00
Adam Williamson
0714143673 Drop a stray old comment and move a debug block up
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-10 14:34:58 -08:00
Adam Williamson
fc93309b41 Drop _assert_and_click shim
We're on newer code on both deployments now, no longer needed.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-10 14:22:20 -08:00
Adam Williamson
dba4e8c57d Drop old version conditionals
We're well past 27 now...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-10 14:20:44 -08:00
Adam Williamson
fcb06a874c Drop no-longer-needed update workarounds
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-10 14:18:54 -08:00
Adam Williamson
0926be0520 Drop similarity level on a flashing cursor wait_still_screen
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-08 15:31:32 -08:00
Adam Williamson
478a55f057 Drop one more mention of wait_idle
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-03 14:20:18 -08:00
Adam Williamson
34dc2751a5 Replace remaining wait_idles with sleeps
wait_idle is finally removed upstream in recent git os-autoinst.
This replaces all remaining wait_idles with sleeps, except for
one which is removed (I'm hoping improvements to typing in the
last few years should mean it isn't necessary any more, if it
turns out to be, I'll put it back as a sleep).

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-01-02 17:44:45 -08:00
Adam Williamson
501b4bccc8 Fix 'clear' after login in French
...because it comes out as 'cleqr'. Note, this may be fragile if
we start doing more stuff post-install, but for now I think it's
safe, I don't *think* we should ever hit this after running
`loadkeys us`.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-12-14 09:17:34 -08:00
Adam Williamson
de2c44e7ae Work around Lmod bug in KDE update packages, with new mechanism
It seems like the situation where we need to pull an update from
updates-testing into all update tests to work around some known
issue is going to keep happening. So instead of constantly
adding and then entirely removing bespoke lines for each specific
workaround, let's have a permanent mechanism for doing this: a
hash with release numbers as keys, and arrayrefs of update IDs
as values, and a block to call `bodhi updates download` on the
appropriate array for the release under test. This way, to add
or remove a workaround you just update the hash. If we're at a
point where *no* workarounds are needed the %workarounds hash
can be made entirely empty (it must exist, though) and the code
will be a clean no-op.

The actual workaround here pulls in Lmod updates I just sent out
to work around this issue in one of the KDE update tests:

https://openqa.fedoraproject.org/tests/497160#step/base_update_cli/11

there's some code in Lmod that gets sourced in bash profiles
which breaks openQA's `validate_script_output` by blurping two
lines of informational output into the output of the script.
The update backports a change from upstream Lmod master that
sends that informational output to stderr instead of stdout.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-12-11 12:02:21 -08:00
Adam Williamson
c9ae9c4d67 Drop SELinux and NSS update workarounds (went stable)
All these updates have gone stable now, so we no longer need
these workarounds.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-12-10 18:12:13 -08:00
Adam Williamson
3511a5b57b Replace jss workaround with selinux and nss workarounds
The jss updates all went stable already. Now we have a problem
with SELinux, upower and container-selinux (we need a newer
selinux-policy to avoid upower failures in the services_start
test, but the first attempt to fix it caused the desktop_updates
test to start failing because container-selinux needed adapting
to changes in selinux-policy...let's just pull in the updates
with the latest versions of both to be safe), and one with NSS
that causes Firefox to give false certificate errors sometimes
(this is particularly affecting the FreeIPA browser test). As
usual these should be dropped once the updates go stable.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-12-06 11:51:07 -08:00
Adam Williamson
e8802935f9 Tweak text_console_login match area
On Rawhide Cloud_Base boots, there's some SSH key and network
information printed above the 'login:' prompt, so we can't
expect empty space there. Also tweak console_login() to clear
the screen after logging in, so the login prompt is cleared and
doesn't confuse things on subsequent runs (like it did first
time we tried this). And add a new user logged in needle, as it
seems after we clear the screen the tilde appears in a slightly
different position and the existing needle doesn't match.

https://openqa.fedoraproject.org/tests/489003#step/_console_wait_login/7

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-27 10:39:29 -08:00
Adam Williamson
defab3bfcb Correct renaming of the virtio serial consoles
That second string didn't work how I thought it did.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-20 09:15:58 -08:00
Adam Williamson
48b6c9d3e9 Change name we use for virtio serial consoles
There is nothing inherently 'root'-y about these so it makes no
sense to prefix their names with 'root-'. And why change from
'console' to 'terminal' compared to the naming used in the
actual qemu command and the log files? It's just confusing.
Let's be consistent (except for using - instead of _ here...
but - is easier to type!)

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-20 08:39:28 -08:00
Lukáš Růžička
b93a197c22 Enable Anaconda Text install via serial console.
This adds the Anaconda text installation test over
serial console and FIXES #115.
2019-11-19 22:54:55 -08:00
Adam Williamson
9bec7ee797 Bump timeout on exiting Firefox again...
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-07 07:35:32 -08:00
Adam Williamson
b82a1dec6f Add test for QA:Testcase_Anaconda_User_Interface_VNC
This adds a test for QA:Testcase_Anaconda_User_Interface_VNC -
the VNC install test case. It's implemented as a server/client
pair, with the server booting from the Server DVD image with
`inst.vnc` and the client booting from the desktop base disk
image, setting up networking, then running Boxes to connect to
the server and run the install.

There are various little tweaks to test loading and logic to
handle this, mostly pretty clear. We also move the workaround
for 'spurious auth prompt appears on desktop after you switch
away to a VT and back' out of the desktop update test and into
the `desktop_vt` helper function, since now this test can hit
it as well. We enhance _graphical_wait_login to handle the boot
loader if needed (it has never needed to until now).

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-05 14:46:10 -08:00
Adam Williamson
efec7010cb Handle an 'akonadi did something' notification in KDE, etc.
This handles a case where KDE shows a notification saying
'PIM Maintenance (Finished)', like this:

https://openqa.fedoraproject.org/tests/477345#step/desktop_notifications/34

we need to click it away for the desktop_notification test to
pass. It also clarifies the difference between this notification
and the eternal 'akonadi_migration_agent is doing something'
popup in the needle names and comments. It also replaces the
'check_screen then assert_and_click if found' pattern in several
notifications-related places with the better 'check_screen then
click_lastmatch if found' pattern now available upstream.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-30 09:05:06 -07:00
Adam Williamson
8c07149ec7 Workaround #1766451 in update tests
Download the fixed jss to the advisory repo.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-29 18:03:59 -07:00
Adam Williamson
764a2bdc33 Drop #1757948 workaround, shouldn't be needed any more
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-29 17:59:33 -07:00
Adam Williamson
33e3224411 Drop stray start_cockpit export from utils.pm
It was moved to cockpit.pm, but this wasn't removed.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-29 15:12:04 -07:00
Lukas Ruzicka
22b185abf6 Enable tracking of executed applications in Application Start/Stop tests (FIXES #116)
This PR adds the following:
* moves out the presetting procedures, so that two long terminal tests do not have
to run twice
* add methods for application to register when successfully started
* adds a test that checks if all required applications have registered
2019-10-16 17:36:37 -07:00
Adam Williamson
079b969fab Workaround RHBZ#1757948 in F30 update tests
We need to pull in the fixed fwupd or else the desktop_update
test always fails.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-15 19:23:55 -07:00
Lukas Ruzicka
d82f4ecd7f Develop a new test for system updates via Cockpit.
* server-cockpit-updates tests that Cockpit can be used to update the system.
* server-cockpit-autoupdate tests that users can use dnf-automatic for system
updates.
* cockpit functions were removed from utils.pm and put into an extra library
for cockpit - cockpit.pm which all cockpit tests are now using.

Review cockpit.pm

Review autoupdate test.

Review the update test.

Fix typo in cockpit.pm

Add sleep.

Add missing command.

Delete an unused needle.
2019-10-01 16:04:01 +02:00
Adam Williamson
b32449f72a Add missing bracket
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-09-20 11:04:31 -07:00
Adam Williamson
6538a82096 Download by ADVISORY_NVRS with koji for updates, not ADVISORY
This goes with a corresponding fedora_openqa commit that passes
ADVISORY_NVRS when scheduling update jobs. This is to address
https://pagure.io/fedora-qa/fedora_openqa/issue/78 so we can
publish spec-compliant and correct CI Messages: we will decide
what NVRs are in the update at scheduling time and always get
and test those NVRs, rather than the test downloading whatever
is in the update when it is run.

Some consequences: manual restarting or cloning of an update
test scheduled before this change lands will result in failure;
to do this you'll have to add the ADVISORY_NVRS value manually
with clone_job.pl, or simply reschedule the update with
fedora-openqa.

Manual restarting or cloning of an update test scheduled with
this change will always test the same NVRs. If the update has
changed and you want to re-test with the new packages in the
update, you must change ADVISORY_NVRS manually with clone_job,
or reschedule with fedora-openqa.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-09-20 10:58:27 -07:00
Adam Williamson
7f96c3aa49 Remove a now-unneeded workaround
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-09-20 10:51:16 -07:00
Michel Normand
6455319ef5 typo in lib/utils.pm
reported in autoinst-log.txt
===
"my" variable $files masks earlier declaration in same scope at /var/lib/openqa/share/tests/fedora/lib/utils.pm line 352.
"my" variable $beta masks earlier declaration in same scope at /var/lib/openqa/share/tests/fedora/lib/utils.pm line 995.
===

Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2019-09-20 07:15:21 +02:00
Adam Williamson
dc8ed1c9ba Yet more pre-release check fixes
We need another needle for French installs, and we need to use
ISO_URL for checking the ISO file name if ISO isn't set.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-09-11 14:31:54 -07:00
Adam Williamson
243dfbdacd desktop_vt: hackaround 'tty?' problem on F31
On F31 update tests, desktop_vt is broken because the 'ps -C'
command is showing 'tty?' as the tty on which Xwayland/Xorg is
running. Let's try using loginctl as a workaround for this.
This ordering of commands should ensure the ps -C output takes
precedence when it's correct.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-09-03 16:19:08 -07:00
Adam Williamson
567b2f1a3f Wait longer for console to appear after quitting Firefox
A couple of tests failed in F31/Rawhide lately because quitting
Firefox (running on a bare X server) took more than 10 seconds
to get back to a console...not sure why, but it's not something
we ought to waste a lot of time on, let's just bump the timeout.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-28 11:07:13 -07:00
Adam Williamson
b9934c9816 Fix prerelease check for Branched update tests
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-27 08:36:40 -07:00
Adam Williamson
bf5691e69d Fix and simplify fedora_release test
Also simplify os_release a bit thanks to the improved spell_
version_number.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-21 13:09:40 -07:00