Commit Graph

1277 Commits

Author SHA1 Message Date
Adam Williamson
3e6764dbb5 Fix French install test on aarch64 by skipping uefi_postinstall
The French install/boot test fails on aarch64 due to a bit of an
ordering issue:

https://openqa.stg.fedoraproject.org/tests/665124

we run `uefi_postinstall`, which does `loadkeys us`, before we
run `_console_login`, which still expects the French layout, so
it breaks. The safest way to solve this I think is to add a new
variable that lets us skip `uefi_postinstall` - I don't want to
change the ordering so we load `uefi_postinstall` after we read
`POSTINSTALL` in case that breaks things somewhere else.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-08 11:21:37 -08:00
Adam Williamson
13eb4ac2e6 Disable snapshotting on all language tests
There's no point at which the snapshot might be restored, and
this works around a difficult-to-debug crash on ppc64le (it seems
the crash only happens when resuming after the snapshot).

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-08 11:00:14 -08:00
Adam Williamson
9bec7ee797 Bump timeout on exiting Firefox again...
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-07 07:35:32 -08:00
Adam Williamson
a8dc6c425c Use comma not space for QEMUMACHINE value
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-06 19:48:15 -08:00
Adam Williamson
f2ef1f1a37 Workaround RHBZ #1769600 by using pseries-4.0 machine type
ppc64le tests are failing to boot at all with stock qemu in F31.
Using pseries-4.0 machine type (the default is pseries-4.1)
seems to work around the problem.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-06 19:41:15 -08:00
Adam Williamson
1c75256f9d Add new VNC tests to aarch64 and ppc64le too
Forgot to do this.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-06 18:14:23 -08:00
Adam Williamson
4c05f19a9f Add QA:Testcase_Anaconda_User_Interface_VNC_Vncconnect tests
This adds tests for vncconnect, a variation on VNC install using
reverse VNC (where the client waits for the server to connect to
it). Very similar to the regular-VNC test, but we have to use
tigervnc as Boxes doesn't do reverse VNC.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-06 17:42:29 -08:00
Adam Williamson
4f72d4ebd3 Add one missing needle for VNC test
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-05 14:48:25 -08:00
Adam Williamson
b82a1dec6f Add test for QA:Testcase_Anaconda_User_Interface_VNC
This adds a test for QA:Testcase_Anaconda_User_Interface_VNC -
the VNC install test case. It's implemented as a server/client
pair, with the server booting from the Server DVD image with
`inst.vnc` and the client booting from the desktop base disk
image, setting up networking, then running Boxes to connect to
the server and run the install.

There are various little tweaks to test loading and logic to
handle this, mostly pretty clear. We also move the workaround
for 'spurious auth prompt appears on desktop after you switch
away to a VT and back' out of the desktop update test and into
the `desktop_vt` helper function, since now this test can hit
it as well. We enhance _graphical_wait_login to handle the boot
loader if needed (it has never needed to until now).

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-05 14:46:10 -08:00
Adam Williamson
a0e191e097 Update cockpit login screen needle *again*
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-30 15:20:30 -07:00
Adam Williamson
efec7010cb Handle an 'akonadi did something' notification in KDE, etc.
This handles a case where KDE shows a notification saying
'PIM Maintenance (Finished)', like this:

https://openqa.fedoraproject.org/tests/477345#step/desktop_notifications/34

we need to click it away for the desktop_notification test to
pass. It also clarifies the difference between this notification
and the eternal 'akonadi_migration_agent is doing something'
popup in the needle names and comments. It also replaces the
'check_screen then assert_and_click if found' pattern in several
notifications-related places with the better 'check_screen then
click_lastmatch if found' pattern now available upstream.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-30 09:05:06 -07:00
Lukáš Růžička
907e27cd8f Change nodejs module version in the install test to make it pass. 2019-10-30 14:58:26 +01:00
Adam Williamson
47fe6acf73 Add a sleep to server_cockpit_autoupdate to work around #1765685
We don't know exactly what's going on with this bug yet, but as
a 15 second sleep seems to avoid the issue, let's do that for
now so we catch any future issues.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-29 18:29:24 -07:00
Adam Williamson
8c07149ec7 Workaround #1766451 in update tests
Download the fixed jss to the advisory repo.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-29 18:03:59 -07:00
Adam Williamson
764a2bdc33 Drop #1757948 workaround, shouldn't be needed any more
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-29 17:59:33 -07:00
Adam Williamson
81100e8a17 Add new Cockpit 'software updates' needle
Needed for the UI change in Cockpit 205.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-29 16:01:35 -07:00
Adam Williamson
4fb29f8bdb Add new Firefox GNOME app menu needle
Firefox icon changed, so this needs updating too.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-29 15:55:52 -07:00
Adam Williamson
33e3224411 Drop stray start_cockpit export from utils.pm
It was moved to cockpit.pm, but this wasn't removed.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-29 15:12:04 -07:00
Adam Williamson
3cdfe96a5c Add another Firefox launcher needle variant
Seems we need two as the launcher background is a different
color on different Fedora releases...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-24 13:53:52 -07:00
Adam Williamson
51d37aeeef Update Firefox launcher needles for new icon
Firefox icon changed a bit in 70.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-24 13:23:24 -07:00
Adam Williamson
e313e58cdd Add new variant boot_enter_passphrase graphical needle
New plymouth has tweaked this screen's appearance.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-24 13:20:51 -07:00
Adam Williamson
d4634e082c Use bypass_1691487 at the start of fedora_release
It gets bit by that problem on ppc64.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-22 14:32:09 -07:00
Adam Williamson
bf7d6d0149 Wait out animation in 'no user' 'not GNOME' flow
See https://openqa.fedoraproject.org/tests/473215 - it failed
because we tried to click FINISH CONFIGURATION while it was
still animating downwards.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-21 10:56:31 -07:00
Adam Williamson
b12c529e34 Try and reduce timing failures in desktop_browser new tab
The way this works at present, with a check_screen and then an
assert_and_click, there's a window where the check has passed
so we're committed to the assert, but it takes a half second or
so for the assert to actually complete (checking for a needle
is a somewhat heavy operation). During that half second the
'new update!' notification can...and quite often does...appear.
Changing the assert_and_click to a click_lastmatch should (I
hope) tighten this window; click_lastmatch should fire faster
than assert_and_click so there'll be less of a window for the
update notification to appear and break stuff.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-19 10:30:50 -07:00
Adam Williamson
e1030118f9 Add new cockpit login needle
The login screen background changed in 205.1 (it's actually the
same image, but it's now zoomed rather than just a chunk of it
being shown on a smaller screen).

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-18 10:35:14 -07:00
Lukas Ruzicka
22b185abf6 Enable tracking of executed applications in Application Start/Stop tests (FIXES #116)
This PR adds the following:
* moves out the presetting procedures, so that two long terminal tests do not have
to run twice
* add methods for application to register when successfully started
* adds a test that checks if all required applications have registered
2019-10-16 17:36:37 -07:00
Adam Williamson
b455e04188 Add new cockpit needles for 205
The cockpit UI went all GRIMDARK so we need a couple of new
needles.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-16 17:10:28 -07:00
Adam Williamson
5f38971a78 Add some new variant layout indicator needles
Once again, the text is at a slightly different height to the
needles we already have. I do not know why this sometimes
happens. It's a mystery.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-16 16:34:55 -07:00
Adam Williamson
b55782358d Don't check version or pre-release note on banners
This reduces the coverage of the identification test a bit but
also *substantially* simplifies it. We run into a ton of problems
when we try to check the version and prerelease text on screens
where it appears on banners:

* The banners differ between variants
* The pre-release text is translated
* The banners have gradients so for RTL languages, even if some
  text is untranslated (e.g. 'Fedora 31') it appears on a
  different background color than on LTR languages
* The prerelease text is dark red; if it appears on a dark blue
  area of the banner this can trigger an os-autoinst needle
  comparison bug: https://progress.opensuse.org/issues/56822

All of this together means we wind up continually fighting these
checks and we have a whole forest of needles just for them, and
it doesn't seem worthwhile. So let's drop all the places where
we were checking version and prerelease on banners, and only
check them in two places where they appear on a grey background,
which avoids most of the problems (we just need one version
needle per release, and one prerelease needle per language).

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-16 11:08:24 -07:00
Adam Williamson
079b969fab Workaround RHBZ#1757948 in F30 update tests
We need to pull in the fixed fwupd or else the desktop_update
test always fails.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-15 19:23:55 -07:00
Adam Williamson
205c0c3d97 Add new variant ABRT needle for KDE app test
Font rendering changed.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-10 12:09:41 -07:00
Lukas Ruzicka
519e6a8d20 Fix Silveblue Beta Values 2019-10-08 14:36:49 +02:00
Lukas Ruzicka
5aa9ca1cf3 Fix VERSION and PRETTY_NAME os-release checks for Silverblue.
This PR fixes #124.
It uses the BUILD variable to get the build string from the
tested compose and uses it to compare correct values for Silverblue.
Originally, it used Workstation values that did not match the
Silveblue ones.

Delete bad yank.
2019-10-08 11:08:58 +02:00
Adam Williamson
80dea7efe2 Tweak PXE boot test to also check kickstart file path
One of the test cases we didn't yet automate is:
https://fedoraproject.org/wiki/QA:Testcase_Kickstart_File_Path_Ks_Cfg
Now we have a PXE test, it's actually a good opportunity to test
that at the same time. I don't usually like combining tests like
this but in this case it sort of makes sense as otherwise we'd
have to have a whole parallel PXE install just to test this one
other detail. So, instead of doing an interactive PXE install as
we did at first, let's tweak the test to include a kickstart in
the initramfs and run the install from that.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-07 14:11:48 -07:00
Adam Williamson
6636122f62 New needle for *partly* translated 'install done' Japanese
Now one of the strings we check for is translated, but the other
is not...handle this, again with a workaround needle so the
missing translation triggers a soft fail.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-07 11:26:54 -07:00
Lukas Ruzicka
d82f4ecd7f Develop a new test for system updates via Cockpit.
* server-cockpit-updates tests that Cockpit can be used to update the system.
* server-cockpit-autoupdate tests that users can use dnf-automatic for system
updates.
* cockpit functions were removed from utils.pm and put into an extra library
for cockpit - cockpit.pm which all cockpit tests are now using.

Review cockpit.pm

Review autoupdate test.

Review the update test.

Fix typo in cockpit.pm

Add sleep.

Add missing command.

Delete an unused needle.
2019-10-01 16:04:01 +02:00
Adam Williamson
0782f4faf2 Add new akonadi migration agent needle for F31 background
Light blue throwing us off again.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-09-26 10:24:37 -07:00
Adam Williamson
8be0541ce7 Tweak live_build mock handling to work with templates
mock is changing the config files to be based on templates,
which breaks how we generate the live build mock config. This
should work with both the old and new config files.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-09-23 16:00:08 -07:00
Adam Williamson
b32449f72a Add missing bracket
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-09-20 11:04:31 -07:00
Adam Williamson
6538a82096 Download by ADVISORY_NVRS with koji for updates, not ADVISORY
This goes with a corresponding fedora_openqa commit that passes
ADVISORY_NVRS when scheduling update jobs. This is to address
https://pagure.io/fedora-qa/fedora_openqa/issue/78 so we can
publish spec-compliant and correct CI Messages: we will decide
what NVRs are in the update at scheduling time and always get
and test those NVRs, rather than the test downloading whatever
is in the update when it is run.

Some consequences: manual restarting or cloning of an update
test scheduled before this change lands will result in failure;
to do this you'll have to add the ADVISORY_NVRS value manually
with clone_job.pl, or simply reschedule the update with
fedora-openqa.

Manual restarting or cloning of an update test scheduled with
this change will always test the same NVRs. If the update has
changed and you want to re-test with the new packages in the
update, you must change ADVISORY_NVRS manually with clone_job,
or reschedule with fedora-openqa.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-09-20 10:58:27 -07:00
Adam Williamson
7f96c3aa49 Remove a now-unneeded workaround
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-09-20 10:51:16 -07:00
Michel Normand
a631030698 Add Workstation-live-iso all tests for ppc64le
Add HDDSIZEGB for Workstation-live-iso ppc64le
as install_default_upload needs at least 11.07GB

Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2019-09-20 13:11:38 +02:00
Michel Normand
200913547c Add Everything-boot-iso install_default for ppc64le
Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2019-09-20 13:08:13 +02:00
Michel Normand
f247ea7511 Remove install_default when install_default_upload
two tests only differ by saving of qcow2 on test end for last one.

That concerns ppc64le and aarch64 flavors:
    AtomicHost-dvd_ostree-iso
    Server-dvd-iso

Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2019-09-20 13:08:13 +02:00
Michel Normand
c2a59b8f01 Add more missing tests for ppc64le
server-dvd-iso:
    release_identification
    server_freeipa_replication_master
    server_freeipa_replication_replica
    server_freeipa_replication_client

Silverblue-dvd_ostree-iso:
    release_identification

universal:
    install_package_set_kde
    install_updates_img_local

Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2019-09-20 13:08:13 +02:00
Guy Menanteau
3e6568c7df Add base_system_logging test for ppc64le
Signed-off-by: Guy Menanteau <menantea@linux.vnet.ibm.com>
2019-09-20 13:08:13 +02:00
Michel Normand
6455319ef5 typo in lib/utils.pm
reported in autoinst-log.txt
===
"my" variable $files masks earlier declaration in same scope at /var/lib/openqa/share/tests/fedora/lib/utils.pm line 352.
"my" variable $beta masks earlier declaration in same scope at /var/lib/openqa/share/tests/fedora/lib/utils.pm line 995.
===

Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2019-09-20 07:15:21 +02:00
Adam Williamson
61f8adefdf Add some new update-related needles for KDE
The new background seems to be throwing us off here. I think
the menu is *very slightly* transparent so the needles usually
work for most Fedora backgrounds as they tend to have a similar
base color, but the 31 one is unusually light.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-09-19 10:23:21 -07:00
Adam Williamson
6e9a7a57c5 Add several more variant keyboard layout needles for new icon
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-09-16 12:26:26 -07:00
Adam Williamson
96cf14d555 Add new variant us_rtl layout needle
For RTL languages the keyboard icon and language indicator are
the other way around, so we have different needles even for the
US layout indicator which is otherwise the same in all langs.
This is a variant needle for the new keyboard icon that showed
up in Rawhide recently.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-09-16 11:32:58 -07:00