Commit graph

1788 commits

Author SHA1 Message Date
Lukáš Růžička
5a28c0c0f2 Add uefi variations of the minimal upgrade test case. 2019-11-20 19:00:13 -08:00
Adam Williamson
5c784e22cc Skip post-install actions for VNC client installs
...otherwise the VNC client tests fail on aarch64 because we try
to apply the 'console=tty0' workaround for #1661288. Fortunately
we don't really need that for the VNC install test to work, so
let's just skip it. We can make this more sophisticated later if
it turns out to be necessary.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-20 18:56:06 -08:00
Michel Normand
bbb5d76849 install_vnc_client & install_vncconnect_client to be ARCH specific
Use  ARCH specific desktop_4 images supposed to be
generated by createhdds tool.

Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2019-11-20 14:05:28 -08:00
Adam Williamson
00c81f9a30 Drop 'anaconda' from serial console test name
It's really not useful.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-20 09:35:23 -08:00
Adam Williamson
82db1770d5 Hack around a serial console prompt check failing during install
os-autoinst has this code when doing `script_run` on a serial
terminal that does a `wait_serial` for '# ' before running the
command. For some reason when we switch to the anaconda root
console and run a couple of commands after finishing install on
ppc64, the first of these prompt checks times out, which means
the test sits there doing nothing for 90 seconds unnecessarily.
Let's try and avoid that by hacking the prompt check regex to
be empty.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-20 09:15:58 -08:00
Adam Williamson
67e7415f2f Fix a bug with stashing original serial_term_prompt setting
Whoops.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-20 09:15:58 -08:00
Adam Williamson
defab3bfcb Correct renaming of the virtio serial consoles
That second string didn't work how I thought it did.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-20 09:15:58 -08:00
Adam Williamson
48b6c9d3e9 Change name we use for virtio serial consoles
There is nothing inherently 'root'-y about these so it makes no
sense to prefix their names with 'root-'. And why change from
'console' to 'terminal' compared to the naming used in the
actual qemu command and the log files? It's just confusing.
Let's be consistent (except for using - instead of _ here...
but - is easier to type!)

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-20 08:39:28 -08:00
Michel Normand
73ca0f2d3f Add missing groupe_name for PowerPC and AArch64 added tests
That concerns:
  test_suite => { name => "install_vnc_server" },
  test_suite => { name => "install_vnc_client" },
  test_suite => { name => "install_vncconnect_server" },
  test_suite => { name => "install_vncconnect_client" },

Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2019-11-20 08:46:43 +01:00
Lukáš Růžička
b93a197c22 Enable Anaconda Text install via serial console.
This adds the Anaconda text installation test over
serial console and FIXES #115.
2019-11-19 22:54:55 -08:00
Adam Williamson
0722cc5bcf Update Cockpit 'join domain' needle for cockpit 207
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-14 08:43:03 -08:00
Adam Williamson
3e6764dbb5 Fix French install test on aarch64 by skipping uefi_postinstall
The French install/boot test fails on aarch64 due to a bit of an
ordering issue:

https://openqa.stg.fedoraproject.org/tests/665124

we run `uefi_postinstall`, which does `loadkeys us`, before we
run `_console_login`, which still expects the French layout, so
it breaks. The safest way to solve this I think is to add a new
variable that lets us skip `uefi_postinstall` - I don't want to
change the ordering so we load `uefi_postinstall` after we read
`POSTINSTALL` in case that breaks things somewhere else.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-08 11:21:37 -08:00
Adam Williamson
13eb4ac2e6 Disable snapshotting on all language tests
There's no point at which the snapshot might be restored, and
this works around a difficult-to-debug crash on ppc64le (it seems
the crash only happens when resuming after the snapshot).

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-08 11:00:14 -08:00
Adam Williamson
9bec7ee797 Bump timeout on exiting Firefox again...
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-07 07:35:32 -08:00
Adam Williamson
a8dc6c425c Use comma not space for QEMUMACHINE value
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-06 19:48:15 -08:00
Adam Williamson
f2ef1f1a37 Workaround RHBZ #1769600 by using pseries-4.0 machine type
ppc64le tests are failing to boot at all with stock qemu in F31.
Using pseries-4.0 machine type (the default is pseries-4.1)
seems to work around the problem.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-06 19:41:15 -08:00
Adam Williamson
1c75256f9d Add new VNC tests to aarch64 and ppc64le too
Forgot to do this.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-06 18:14:23 -08:00
Adam Williamson
4c05f19a9f Add QA:Testcase_Anaconda_User_Interface_VNC_Vncconnect tests
This adds tests for vncconnect, a variation on VNC install using
reverse VNC (where the client waits for the server to connect to
it). Very similar to the regular-VNC test, but we have to use
tigervnc as Boxes doesn't do reverse VNC.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-06 17:42:29 -08:00
Adam Williamson
4f72d4ebd3 Add one missing needle for VNC test
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-05 14:48:25 -08:00
Adam Williamson
b82a1dec6f Add test for QA:Testcase_Anaconda_User_Interface_VNC
This adds a test for QA:Testcase_Anaconda_User_Interface_VNC -
the VNC install test case. It's implemented as a server/client
pair, with the server booting from the Server DVD image with
`inst.vnc` and the client booting from the desktop base disk
image, setting up networking, then running Boxes to connect to
the server and run the install.

There are various little tweaks to test loading and logic to
handle this, mostly pretty clear. We also move the workaround
for 'spurious auth prompt appears on desktop after you switch
away to a VT and back' out of the desktop update test and into
the `desktop_vt` helper function, since now this test can hit
it as well. We enhance _graphical_wait_login to handle the boot
loader if needed (it has never needed to until now).

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-05 14:46:10 -08:00
Adam Williamson
a0e191e097 Update cockpit login screen needle *again*
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-30 15:20:30 -07:00
Adam Williamson
efec7010cb Handle an 'akonadi did something' notification in KDE, etc.
This handles a case where KDE shows a notification saying
'PIM Maintenance (Finished)', like this:

https://openqa.fedoraproject.org/tests/477345#step/desktop_notifications/34

we need to click it away for the desktop_notification test to
pass. It also clarifies the difference between this notification
and the eternal 'akonadi_migration_agent is doing something'
popup in the needle names and comments. It also replaces the
'check_screen then assert_and_click if found' pattern in several
notifications-related places with the better 'check_screen then
click_lastmatch if found' pattern now available upstream.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-30 09:05:06 -07:00
Lukáš Růžička
907e27cd8f Change nodejs module version in the install test to make it pass. 2019-10-30 14:58:26 +01:00
Adam Williamson
47fe6acf73 Add a sleep to server_cockpit_autoupdate to work around #1765685
We don't know exactly what's going on with this bug yet, but as
a 15 second sleep seems to avoid the issue, let's do that for
now so we catch any future issues.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-29 18:29:24 -07:00
Adam Williamson
8c07149ec7 Workaround #1766451 in update tests
Download the fixed jss to the advisory repo.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-29 18:03:59 -07:00
Adam Williamson
764a2bdc33 Drop #1757948 workaround, shouldn't be needed any more
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-29 17:59:33 -07:00
Adam Williamson
81100e8a17 Add new Cockpit 'software updates' needle
Needed for the UI change in Cockpit 205.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-29 16:01:35 -07:00
Adam Williamson
4fb29f8bdb Add new Firefox GNOME app menu needle
Firefox icon changed, so this needs updating too.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-29 15:55:52 -07:00
Adam Williamson
33e3224411 Drop stray start_cockpit export from utils.pm
It was moved to cockpit.pm, but this wasn't removed.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-29 15:12:04 -07:00
Adam Williamson
3cdfe96a5c Add another Firefox launcher needle variant
Seems we need two as the launcher background is a different
color on different Fedora releases...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-24 13:53:52 -07:00
Adam Williamson
51d37aeeef Update Firefox launcher needles for new icon
Firefox icon changed a bit in 70.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-24 13:23:24 -07:00
Adam Williamson
e313e58cdd Add new variant boot_enter_passphrase graphical needle
New plymouth has tweaked this screen's appearance.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-24 13:20:51 -07:00
Adam Williamson
d4634e082c Use bypass_1691487 at the start of fedora_release
It gets bit by that problem on ppc64.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-22 14:32:09 -07:00
Adam Williamson
bf7d6d0149 Wait out animation in 'no user' 'not GNOME' flow
See https://openqa.fedoraproject.org/tests/473215 - it failed
because we tried to click FINISH CONFIGURATION while it was
still animating downwards.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-21 10:56:31 -07:00
Adam Williamson
b12c529e34 Try and reduce timing failures in desktop_browser new tab
The way this works at present, with a check_screen and then an
assert_and_click, there's a window where the check has passed
so we're committed to the assert, but it takes a half second or
so for the assert to actually complete (checking for a needle
is a somewhat heavy operation). During that half second the
'new update!' notification can...and quite often does...appear.
Changing the assert_and_click to a click_lastmatch should (I
hope) tighten this window; click_lastmatch should fire faster
than assert_and_click so there'll be less of a window for the
update notification to appear and break stuff.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-19 10:30:50 -07:00
Adam Williamson
e1030118f9 Add new cockpit login needle
The login screen background changed in 205.1 (it's actually the
same image, but it's now zoomed rather than just a chunk of it
being shown on a smaller screen).

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-18 10:35:14 -07:00
Lukas Ruzicka
22b185abf6 Enable tracking of executed applications in Application Start/Stop tests (FIXES #116)
This PR adds the following:
* moves out the presetting procedures, so that two long terminal tests do not have
to run twice
* add methods for application to register when successfully started
* adds a test that checks if all required applications have registered
2019-10-16 17:36:37 -07:00
Adam Williamson
b455e04188 Add new cockpit needles for 205
The cockpit UI went all GRIMDARK so we need a couple of new
needles.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-16 17:10:28 -07:00
Adam Williamson
5f38971a78 Add some new variant layout indicator needles
Once again, the text is at a slightly different height to the
needles we already have. I do not know why this sometimes
happens. It's a mystery.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-16 16:34:55 -07:00
Adam Williamson
b55782358d Don't check version or pre-release note on banners
This reduces the coverage of the identification test a bit but
also *substantially* simplifies it. We run into a ton of problems
when we try to check the version and prerelease text on screens
where it appears on banners:

* The banners differ between variants
* The pre-release text is translated
* The banners have gradients so for RTL languages, even if some
  text is untranslated (e.g. 'Fedora 31') it appears on a
  different background color than on LTR languages
* The prerelease text is dark red; if it appears on a dark blue
  area of the banner this can trigger an os-autoinst needle
  comparison bug: https://progress.opensuse.org/issues/56822

All of this together means we wind up continually fighting these
checks and we have a whole forest of needles just for them, and
it doesn't seem worthwhile. So let's drop all the places where
we were checking version and prerelease on banners, and only
check them in two places where they appear on a grey background,
which avoids most of the problems (we just need one version
needle per release, and one prerelease needle per language).

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-16 11:08:24 -07:00
Adam Williamson
079b969fab Workaround RHBZ#1757948 in F30 update tests
We need to pull in the fixed fwupd or else the desktop_update
test always fails.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-15 19:23:55 -07:00
Adam Williamson
205c0c3d97 Add new variant ABRT needle for KDE app test
Font rendering changed.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-10 12:09:41 -07:00
Lukas Ruzicka
519e6a8d20 Fix Silveblue Beta Values 2019-10-08 14:36:49 +02:00
Lukas Ruzicka
5aa9ca1cf3 Fix VERSION and PRETTY_NAME os-release checks for Silverblue.
This PR fixes #124.
It uses the BUILD variable to get the build string from the
tested compose and uses it to compare correct values for Silverblue.
Originally, it used Workstation values that did not match the
Silveblue ones.

Delete bad yank.
2019-10-08 11:08:58 +02:00
Adam Williamson
80dea7efe2 Tweak PXE boot test to also check kickstart file path
One of the test cases we didn't yet automate is:
https://fedoraproject.org/wiki/QA:Testcase_Kickstart_File_Path_Ks_Cfg
Now we have a PXE test, it's actually a good opportunity to test
that at the same time. I don't usually like combining tests like
this but in this case it sort of makes sense as otherwise we'd
have to have a whole parallel PXE install just to test this one
other detail. So, instead of doing an interactive PXE install as
we did at first, let's tweak the test to include a kickstart in
the initramfs and run the install from that.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-07 14:11:48 -07:00
Adam Williamson
6636122f62 New needle for *partly* translated 'install done' Japanese
Now one of the strings we check for is translated, but the other
is not...handle this, again with a workaround needle so the
missing translation triggers a soft fail.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-07 11:26:54 -07:00
Lukas Ruzicka
d82f4ecd7f Develop a new test for system updates via Cockpit.
* server-cockpit-updates tests that Cockpit can be used to update the system.
* server-cockpit-autoupdate tests that users can use dnf-automatic for system
updates.
* cockpit functions were removed from utils.pm and put into an extra library
for cockpit - cockpit.pm which all cockpit tests are now using.

Review cockpit.pm

Review autoupdate test.

Review the update test.

Fix typo in cockpit.pm

Add sleep.

Add missing command.

Delete an unused needle.
2019-10-01 16:04:01 +02:00
Adam Williamson
0782f4faf2 Add new akonadi migration agent needle for F31 background
Light blue throwing us off again.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-09-26 10:24:37 -07:00
Adam Williamson
8be0541ce7 Tweak live_build mock handling to work with templates
mock is changing the config files to be based on templates,
which breaks how we generate the live build mock config. This
should work with both the old and new config files.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-09-23 16:00:08 -07:00
Adam Williamson
b32449f72a Add missing bracket
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-09-20 11:04:31 -07:00