Commit Graph

445 Commits

Author SHA1 Message Date
Adam Williamson
d4634e082c Use bypass_1691487 at the start of fedora_release
It gets bit by that problem on ppc64.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-22 14:32:09 -07:00
Adam Williamson
bf7d6d0149 Wait out animation in 'no user' 'not GNOME' flow
See https://openqa.fedoraproject.org/tests/473215 - it failed
because we tried to click FINISH CONFIGURATION while it was
still animating downwards.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-21 10:56:31 -07:00
Adam Williamson
b12c529e34 Try and reduce timing failures in desktop_browser new tab
The way this works at present, with a check_screen and then an
assert_and_click, there's a window where the check has passed
so we're committed to the assert, but it takes a half second or
so for the assert to actually complete (checking for a needle
is a somewhat heavy operation). During that half second the
'new update!' notification can...and quite often does...appear.
Changing the assert_and_click to a click_lastmatch should (I
hope) tighten this window; click_lastmatch should fire faster
than assert_and_click so there'll be less of a window for the
update notification to appear and break stuff.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-19 10:30:50 -07:00
Lukas Ruzicka
22b185abf6 Enable tracking of executed applications in Application Start/Stop tests (FIXES #116)
This PR adds the following:
* moves out the presetting procedures, so that two long terminal tests do not have
to run twice
* add methods for application to register when successfully started
* adds a test that checks if all required applications have registered
2019-10-16 17:36:37 -07:00
Adam Williamson
b55782358d Don't check version or pre-release note on banners
This reduces the coverage of the identification test a bit but
also *substantially* simplifies it. We run into a ton of problems
when we try to check the version and prerelease text on screens
where it appears on banners:

* The banners differ between variants
* The pre-release text is translated
* The banners have gradients so for RTL languages, even if some
  text is untranslated (e.g. 'Fedora 31') it appears on a
  different background color than on LTR languages
* The prerelease text is dark red; if it appears on a dark blue
  area of the banner this can trigger an os-autoinst needle
  comparison bug: https://progress.opensuse.org/issues/56822

All of this together means we wind up continually fighting these
checks and we have a whole forest of needles just for them, and
it doesn't seem worthwhile. So let's drop all the places where
we were checking version and prerelease on banners, and only
check them in two places where they appear on a grey background,
which avoids most of the problems (we just need one version
needle per release, and one prerelease needle per language).

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-16 11:08:24 -07:00
Lukas Ruzicka
519e6a8d20 Fix Silveblue Beta Values 2019-10-08 14:36:49 +02:00
Lukas Ruzicka
5aa9ca1cf3 Fix VERSION and PRETTY_NAME os-release checks for Silverblue.
This PR fixes #124.
It uses the BUILD variable to get the build string from the
tested compose and uses it to compare correct values for Silverblue.
Originally, it used Workstation values that did not match the
Silveblue ones.

Delete bad yank.
2019-10-08 11:08:58 +02:00
Adam Williamson
80dea7efe2 Tweak PXE boot test to also check kickstart file path
One of the test cases we didn't yet automate is:
https://fedoraproject.org/wiki/QA:Testcase_Kickstart_File_Path_Ks_Cfg
Now we have a PXE test, it's actually a good opportunity to test
that at the same time. I don't usually like combining tests like
this but in this case it sort of makes sense as otherwise we'd
have to have a whole parallel PXE install just to test this one
other detail. So, instead of doing an interactive PXE install as
we did at first, let's tweak the test to include a kickstart in
the initramfs and run the install from that.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-07 14:11:48 -07:00
Lukas Ruzicka
d82f4ecd7f Develop a new test for system updates via Cockpit.
* server-cockpit-updates tests that Cockpit can be used to update the system.
* server-cockpit-autoupdate tests that users can use dnf-automatic for system
updates.
* cockpit functions were removed from utils.pm and put into an extra library
for cockpit - cockpit.pm which all cockpit tests are now using.

Review cockpit.pm

Review autoupdate test.

Review the update test.

Fix typo in cockpit.pm

Add sleep.

Add missing command.

Delete an unused needle.
2019-10-01 16:04:01 +02:00
Adam Williamson
8be0541ce7 Tweak live_build mock handling to work with templates
mock is changing the config files to be based on templates,
which breaks how we generate the live build mock config. This
should work with both the old and new config files.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-09-23 16:00:08 -07:00
Adam Williamson
c0f911067f Disable prerelease text check in disk_guided_empty
This is causing all kinds of trouble, because when the test is
run on the Server DVD - with the 'orange to blue' gradient - the
prerelease note is dark red text on a dark blue background.
os-autoinst actually reduces the color depth of images/needles
and greyscales them before performing the match...but for this
dark red text on dark blue background, the result seems to be
that the text and background come out *the same grey*, so *any*
text will match the needle (even if it's completely different
text), as will *no text at all*. I've tried finessing around
this a few times but it just keeps happening, so for now I'm
just disabling the pre-release text check at this point. We still
have the check during _boot_to_anaconda, when the text appears
on a *grey* background and so isn't a problem. I'm not removing
the needles yet, until we hear back from upstream:

https://progress.opensuse.org/issues/56822

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-09-11 15:01:54 -07:00
Lukas Ruzicka
46bc71afd6 Delete the KDE test for Release Notes due to retirement. 2019-09-05 11:41:18 +02:00
Adam Williamson
8b236592d9 Disable SELinux for live image build test
Building images with SELinux in permissive mode worked fine in
F29 and F30, but it seems to be broken in F31. Releng seems to
have it set up so the build environment has SELinux disabled,
so let's do the same. Will file a bug for this, as bcl says it
should be considered a bug.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-29 17:49:27 -07:00
Lukas Ruzicka
022865ab9f KDE app start/stop: launch apps with commands not menus
This is doing slightly less to exercise the launcher menus and see
whether icons appear or disappear, but it's much faster and more
reliable. We do still use menu launch for one app, just to check
the mechanism works in general.
2019-08-23 12:41:49 -07:00
Adam Williamson
bf5691e69d Fix and simplify fedora_release test
Also simplify os_release a bit thanks to the improved spell_
version_number.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-21 13:09:40 -07:00
Adam Williamson
2f51f3ef20 Add PXE boot test
This adds a whole wodge of stuff to support_server to make it
act as a PXE server, then adds a new test which boots from PXE
and so should hit the PXE server. We use the NFS install repo as
that can be relied on to work for a support_server install.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-20 18:17:57 -07:00
Adam Williamson
a177a63ec1 Factor out s/metalink/mirrorlist/ into a util function
We do this in quite a few places and need to do it in another,
so let's just have a function for it. It takes a file glob
so we can have it run on a different one for _live_build.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-20 14:16:23 -07:00
Adam Williamson
d4206ee732 Drop one old regex bit from install source check
Not needed now F28 is EOL.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-20 12:06:33 -07:00
Adam Williamson
d71daa1fe4 Do the metalink->mirrorlist dodge in live_build mock config
This avoids hitting slow metadata downloads in mock env.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-15 13:36:25 -07:00
Adam Williamson
cac08c170f Remove RHBZ #1704488 workaround (was fixed a while back)
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-15 11:45:16 -07:00
Adam Williamson
cf5704eb3d Goddamn boilerplate...
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-14 14:43:23 -07:00
Adam Williamson
7739bdbbd8 D'oh - check_desktop_clean is in utils...
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-14 14:42:18 -07:00
Adam Williamson
469a1395d0 Revise desktop background checks
Split this out of install_default, because it really is not a
part of that test and we do not want that test to fail because
the desktop background is wrong. Make it its own test module
and test suite instead. Don't do it on Rawhide, because we
really can't assert anything worthwhile about Rawhide at the
moment at least (this means the test runs but is a no-op and
will always pass on Rawhide, unfortunately). Move the needles
to a more appropriate location (this has nothing to do with
anaconda) and use 'background' not 'wallpaper' naming (that's
the name we use elsewhere in the project, e.g. package names).
Also, run the test on updates, and add an F29 needle for this
purpose.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-14 14:30:25 -07:00
Lukas Ruzicka
d06b30e8dc Add new testcases and methods for release identification. 2019-07-30 15:45:02 -07:00
Adam Williamson
c4c998c5a8 Actually set longer timeout before services entry click
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-07-25 14:25:11 -07:00
Adam Williamson
cf709af13c cockpit: retry the 'services entry' click if it fails
cockpit test often seems to fail because the click on a services
entry to open the services detail screen just gets lost. Let's
wait longer before trying it, and retry once if it fails.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-07-25 14:17:33 -07:00
Adam Williamson
be12f07900 GNOME apps: dismiss new LibreOffice 'tip of the day' if it shows
Also add a new needle for Impress startup.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-07-23 20:15:24 -07:00
Adam Williamson
19268619fd Adapt _do_install_and_reboot for Rawhide user/root spoke change
The user and root spokes were moved from during-install to pre-
install hub in Rawhide. This should cope with that, while still
working for older images.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-07-23 12:14:03 -07:00
Adam Williamson
317aadeb2b potential_conflict: still need to call the interpreter
script isn't executable as downloaded, this is easier than chmod.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-07-22 15:00:25 -07:00
Adam Williamson
5a10c537f7 Update potential_conflict invocation for Python 3
The script got ported. Yay.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-07-22 14:54:36 -07:00
Lukáš Růžička
2db77367f9 Add new action to the install test. 2019-07-19 14:22:49 +02:00
Adam Williamson
25f408ea7e Factor out clicking of update and akonadi notifications in KDE
There are three places where we basically want to click away
pop-up update notifications and the buggy akonadi_migration_agent
notification if it's there, in KDE tests. Let's share this code
between them, and also let's record soft failures for the buggy
cases in the desktop_notifications test.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-07-17 09:02:24 -07:00
Adam Williamson
eba6818f5b KDE app start/stop: also get rid of akonadi notification
...because it screws with the test. See
https://bugzilla.redhat.com/show_bug.cgi?id=1716005

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-07-16 15:53:02 -07:00
Adam Williamson
4cc51a51a0 Dismiss multiple update notifications in app start/stop test too
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-07-16 14:40:10 -07:00
Adam Williamson
43417f925c Whoops...actually decrease the counter
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-07-16 14:05:00 -07:00
Adam Williamson
34321dae8f desktop_notifications: handle multiple KDE update notifications
Current KDE seems to like showing us multiple update available
notifications. So the test must dismiss all of them. See:
https://bugzilla.redhat.com/show_bug.cgi?id=1730482

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-07-16 13:58:15 -07:00
Lukas Ruzicka
1475813b75 Update the modularity tests.
* Add test to check module defaults.
* Add whitelist download.
* Fix install test to include selected profile to be on the safe side.
* Add test into templates.
2019-07-15 13:19:06 +02:00
Adam Williamson
dfce3e522d desktop_update: handle multiple KDE update available notifications
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-07-10 12:37:26 -07:00
Adam Williamson
9078bca871 cockpit: update needles for 198, rejig Services page handling
Cockpit 198 comes with a UI overhaul, so almost all needles
need an update.

The 'auditd' service is no longer on the first page. To make
this less fragile (at the cost of not testing that clicking on
a service actually opens the detail page *for that service*,
tweak the needles to just look for *any* running service, click
on it, and check we got to a 'details' page. We also redo the
existing needles for this design.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-07-10 11:43:03 -07:00
Adam Williamson
2ecf97e7a7 Revert "desktop_browser: remove the new tab workaround"
This reverts commit a0c6d1c88e.
Sadly we're still getting problems opening new tabs, even though
I thought the os-autoinst change should solve it.
2019-07-08 10:25:46 -07:00
Adam Williamson
a0c6d1c88e desktop_browser: remove the new tab workaround
I'm pretty sure I worked around the modifier bug in os-autoinst,
so this shouldn't be necessary any more. See:
https://bugzilla.redhat.com/show_bug.cgi?id=1727388

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-07-05 16:42:58 -07:00
Adam Williamson
78dd42b876 Automate two 'mediakit' tests (repoclosure and file conflicts)
We can run these tests much like support_server - have a test
which boots from the support_server disk image but with the ISO
to be tested attached, then mounts it and runs the scripts.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-07-04 13:44:20 -07:00
Adam Williamson
de4a58438b Use existing needle for dismissing KDE update notification
@lruzicka added this to the KDE app tests in 71c4e273, but there
was no need for a new needle as I'd already done the same thing
in the desktop updates tests; let's just use the same needle.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-07-03 08:39:01 -07:00
Lukas Ruzicka
71c4e273ef Click off updates notification if it appears. 2019-07-03 10:52:47 +02:00
Lukas Ruzicka
d538959011 Update modules to test. 2019-07-03 10:42:33 +02:00
Adam Williamson
5bb9f26b5f Try and fix root password setting for ostree installs
This is kind of a pain. I hope this works...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-06-28 14:09:55 -07:00
Adam Williamson
604a476295 Don't setenforce before chpasswd for root
This shouldn't be needed any more (the referenced bug was fixed
in F28) and doesn't work in non-live installer.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-06-28 14:09:55 -07:00
Adam Williamson
c0bad87836 Revamp post-install Doin' Stuff
This whole block where we do various things at the console after
install completes was becoming a real mess. I had secret hopes
of killing it entirely at some point, but...that doesn't look
like it's gonna happen this century. So let's make it better
instead. The conditionals were getting very nested and icky and
it was hard to see what was actually going on. This rationalizes
things so first we figure out all the things we might want to do
at a console, then if we don't have anything to do at a console
we go ahead and hit the reboot button; otherwise we go to the
console and do all the things we need to do, including rebooting
unless this is the memory check test.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-06-28 14:09:55 -07:00
Lukas Ruzicka
9e3b16bfcf Change Settings test due to change functionality. 2019-06-21 14:54:18 +02:00
Adam Williamson
7c78f9a710 Don't try bypass_1691487 for graphical base_selinux runs
This just doesn't work when running the test on a graphical
install because we're at a desktop when we try and do it, not
a text console. So let's only try to do it if there is no
DESKTOP. See e.g.:

https://openqa.fedoraproject.org/tests/408484#step/base_selinux/2

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-06-03 15:32:25 -07:00