Commit Graph

445 Commits

Author SHA1 Message Date
Adam Williamson
7953fb91d1 Try to handle spurious 'auth required' on KDE update test too
Sigh, now *this* is showing up in KDE too. Fun.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-05-30 17:54:27 -07:00
Adam Williamson
2e56facb68 Try to handle changes to KDE update notifications in Rawhide
The way KDE does update notifications has changed - it's now a
permanent pop-up notification. This is a bit awkward for our
logic; it's hard to define a needle that proves this pop-up is
the only notification. Instead, let's dismiss it, then open the
notification tray and assert that there aren't any others. But
we also retain the old behaviour (more or less) for testing old
releases.

The popup notification also blocks the 'refresh' needle in the
systray and so breaks the desktop update test, so we deal with
that too.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-05-30 17:34:09 -07:00
Adam Williamson
43b990195c Add a wrapper around assert_and_click to handle signature change
So, turns out new os-autoinst does *not* still accept the old
argument style for assert_and_click...and old os-autoinst
doesn't accept the new one. This adds a wrapper that handles
both, so our tests can work with old and new os-autoinst. We can
drop this once both deployments are on newer os-autoinst.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-05-28 09:41:33 -07:00
Adam Williamson
87456f6b1d Use new arg style for assert_and_click
In the new os-autoinst I just sent to staging, the old style
doesn't work any more, breaks all tests. This style should also
work with the older os-autoinst on stable.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-05-27 14:28:17 -07:00
Adam Williamson
66917e791d Fix the check from the last commit
D'oh.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-05-22 15:27:59 -07:00
Adam Williamson
e1ffd8aabd Fix text install test for 'use password' now being default
'use password' now seems to be default when creating a user in
text mode, so we need to handle that.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-05-22 15:18:08 -07:00
Adam Williamson
0d38d3f292 FreeIPA: get verbose logs from BIND
This was necessary for debugging the FreeIPA 4.8 pre-release
update bug, so let's have it for all runs, just in case.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-05-01 18:59:52 -07:00
Adam Williamson
6c22b55be5 Drop another part of branched buildroot repo workaround
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-05-01 13:49:18 -07:00
Adam Williamson
f4c3c4c2cd Workaround #1704488 in the live image build test
Odd bug which breaks mock until you reboot or restart dbus-broker.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-04-29 16:24:19 -07:00
Adam Williamson
2d709a99ad Remove #1699099 workaround
We got a compose with the fix included, so this should no longer
be needed.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-04-18 07:59:29 -07:00
Adam Williamson
ccb67bd5a7 Extend gnome-software/issues/582 workaround to 2 auth requests
It seems that when this problem happens now we get *two* auth
requests, so we need to handle that.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-04-17 10:32:04 -07:00
Adam Williamson
d0d99946ce Exclude fix for #1699099 from the workaround...
...so we can check it works.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-04-16 22:49:49 -07:00
Michel Normand
37e2f134f1 Bypass dup chars after snapshot brc#1691487
eg failing jobs:
https://openqa.stg.fedoraproject.org/tests/499908#step/_boot_to_anaconda/7
https://openqa.stg.fedoraproject.org/tests/519326#step/_post_network_static/4

Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2019-04-15 16:20:13 +02:00
Adam Williamson
a9c6e69689 Make the 1699099 workaround not assert_script_run
File doesn't exist for ostree installs. No point failing if this
fails, we may as well just continue and see what happens.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-04-13 08:40:57 -07:00
Adam Williamson
33dd7ded6b Sigh fix a syntax error in a previous fix I am bad at this
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-04-13 08:02:52 -07:00
Adam Williamson
d1cb937ea3 Load correct keyboard layout before doing 1699099 workaround
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-04-13 08:00:23 -07:00
Adam Williamson
2e68250bb1 Fix a logic bug in the 1699099 workaround which broke lots
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-04-13 07:56:57 -07:00
Adam Williamson
785eaffb3a Add a workaround for #1699099 so update tests don't fail on it
No purpose is being served by all update tests failing on this
bug, so let's try and work around it.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-04-12 18:55:17 -07:00
Adam Williamson
33f63c3d95 rescue_shell_prompt is really just a root console needle...
...so make it look like all the other ones we have already.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-04-12 15:54:26 -07:00
Guy Menanteau
33fd05f3d0 New install_repository_nfsiso_variation
This is covering the  NFSISO_variation test as per:
https://fedoraproject.org/wiki/QA:Testcase_install_repository_NFSISO_variation

Change support_server to use support_5 disk image in templates
supposed to be previously created by updated createhdds tool
required for new test.

Signed-off-by: Guy Menanteau <menantea@linux.vnet.ibm.com>
2019-04-12 15:49:54 -07:00
Guy Menanteau
29070a54f1 New install_repository_hd_variation
use a new PREINSTALL to execute a rescue script before install.

This is covering the  Hard_drive_variation test as per:
https://fedoraproject.org/wiki/QA:Testcase_install_repository_Hard_drive_variation

Signed-off-by: Guy Menanteau <menantea@linux.vnet.ibm.com>
2019-04-12 15:49:54 -07:00
Michel Normand
4a00301725 new select_rescue_mode in lib/utils from rescue_mode_encrypted.pm
placed in lib/utils.pm to be used by another commit.

Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2019-04-12 15:49:54 -07:00
Adam Williamson
6bd24c3caa Set up FreeIPA reverse DNS on F28, where we still use rolekit
Otherwise the replica tests don't work.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-04-10 13:16:56 -07:00
Adam Williamson
dc81c64f4f Use type_very_safely for the desktop auth workaround
type_safely just ain't safe enough, it seems.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-04-10 12:46:45 -07:00
Lukas Ruzicka
99f93945b3 Change KDE desktop background at start of KDE app test
Since the KDE menus have transparency set, any time the wallpaper
changes the menus will look different, and the app tests may
fail. This sets the desktop wallpaper to black at the start of
the test suite to avoid this problem.
2019-04-09 08:57:43 -07:00
Adam Williamson
794c245786 Restore #1594402 / #1661288 workarounds (aarch64 console)
This reverts commit f40599ee15,
with a few tweaks. It turns out we still need these workarounds
- see https://bugzilla.redhat.com/show_bug.cgi?id=1661288 .
2019-03-29 15:22:47 -07:00
Adam Williamson
b1a0fd7dc3 Add tests for QA:Testcase Remote Logging test case
This commit adds server and client tests to cover the remote
logging Server test case.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-03-28 12:52:54 -07:00
Lukas Ruzicka
662f89f9d4 Fix Firefox failure. 2019-03-28 11:50:41 +01:00
Lukas Ruzicka
f9201fb7d8 Fix kmail and kontakt tests
Fix to pass kontakt tests properly.
2019-03-27 18:00:05 +01:00
Lukas Ruzicka
7dd43daab8 Fix Krusader
It seems that there was a lag in button drawing, which prevented
OpenQA from fully recognizing the krusader_welcome needle because
it was evaluated too soon.
I added "wait_still_screen 3" to make sure, there is time for a
button to be drawn correctly before assertion.
2019-03-27 11:30:01 +01:00
Lukas Ruzicka
ff4ceb55ff Enable a switch for systemwide ABRT during installation.
Change method from assert_script_run to script_run

Change to reboot from console.

Move code in submethod, use ifs to make sure everything runs.

Fix chroot rebooting problem.
2019-03-26 20:10:00 +00:00
Adam Williamson
e396341f31 Handle 'authentication required' appearing on Boxes startup
In the last few weeks Boxes is showing an auth dialog on start.
I've filed a bug on this; let's have the test handle it as a
soft failure, since this isn't a fatal problem. Do this by
making an existing needle for this dialog a bit more generic
in name and using it.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-03-26 12:30:47 -07:00
Adam Williamson
5a1a71c67f Use the existing 'oh hai this is firefox' needle for app tests
The KDE and GNOME 'apps' tests for Firefox both invented their
own 'oh look Firefox is running' needle, even though we already
had one. The GNOME one was broken by the removal of the app
title bar in Firefox 66.

Instead of having three needles for the same thing, let's just
throw the 'apps' ones out and use the pre-existing one from
needles/firefox for all cases.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-03-26 12:23:45 -07:00
Lukas Ruzicka
9ae60de422 Reneedle menus and fix some issues in tests.
This update fixes the strange behaviour in KDE which causes wrong clicking.
With new logic, this is not an issue anymore.
2019-03-20 10:26:33 +01:00
Lukas Ruzicka
426b3447da Delete tests for non-existing menu items. 2019-03-19 12:14:52 -07:00
Lukas Ruzicka
c9a25cb2a6 Create an application start-stop test for KDE.
Add png files as a background for the needles.

Rewrite the test handling methods to make them ready for KDE

Add the ABRT startstop test.

Make changes and corrections to the abrt test.

Add test for Firewall.

Add test for DNF dragora.

Add wait time for dnfdragora test.

Correct syntax.

Add Language test.

Make some changes to the DNF dragora test.

Add Users test case.

Add needles for DBUS viewer test.

Add Dbusviewer test.

Add Mahjong test and needles for games.

Add Minesweeper tests.

Add Patience test.

Add test for Document Viewer.

add test for gwenview

Add test for koulourpaint.

Add test for Kruler

Add test for Kcolorchooser

Reneedle failing needle.

Add ktorrent tests.

Add tests for CPT editors.

Add test for Krfb

Fix names for those files.

Add test for Kget

Add Akregator test.

Add test for Konversation.

Make Konversation end really.

Add tests for Kmail

Add test for PIM exporter.

Add test for KTnef and Krdc.

Fix problems after test runs.

Make more tries.

Fix needle to be found better.

Fix more errors.

Add test for Falkon.

Add tests for browsers.

Add support for closing tabs into Firefox test.

Add tests for K3B

move needles to correct directory

Add Kaddressbook test.

Add Kontakt text.

Add test for korganizer.

Add menu office needle and correct konqueror needle.

Add test for calligra stage.

Add test for Calligra.

Add test for network connections.

Modify needle for kaddressbook to prevent failing.

Add test for system settings and fix others.

Add test for FMW.

Add test for Dolphin

Add test for Infocenter.

Add test for kparted.

Fix a wrong needle.

Test relnotes.

Fix some errors in tests.

Add test for Discover.

Add test for Ksysguard.

Add tests for Konsole.

Add tests for KDE wallet.

Add tests for several utilities.

Add Krusader test.

Finish utility tests.

Fix some errors.

Fix needle for spectacle.

Add wait time to let Dragora wait for network.
2019-03-19 12:14:47 -07:00
Adam Williamson
1ed87382a9 abrt back under utilities, drop sundry
As of Fedora-30-20190316.n.1, gnome-abrt is back under the
Utilities menu and the 'Sundry' menu seems to have disappeared.
Change the test back to using utilities and drop the sundry
needle.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-03-19 10:11:32 -07:00
Michel Normand
ff3768b1d0 grub2-install parm set to /dev/vda1 for PowerPC
Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2019-03-12 19:41:20 +00:00
Adam Williamson
b777094b7b Give live image build command another 10 minutes
40 seems not long enough, it's timing out while the build is
still running sometimes.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-03-06 19:53:31 -08:00
Adam Williamson
d6c339e9ff Disable the 'f30-build' repo for the desktop update test
...GNOME Software doesn't like it.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-03-06 19:51:09 -08:00
Adam Williamson
f3caa1fc53 Bump timeout for install_lang_selected to appear a bit
It seems 3 secs was a bit tight for recent Branched and Rawhide,
test was failing when the screen was just a bit slow to update
for some reason.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-03-04 17:32:25 -08:00
Adam Williamson
450f843389 desktop_browser: try both button and ctrl-t for opening new tab
Sigh. It seems sometimes the button isn't visible because GNOME
pops up a fricking notification that blocks it.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-03-01 09:13:54 -08:00
Adam Williamson
03a56eb0da Avoid spurious repo check error for universal tests on netinst
As there is *always* a base repo error when booting a netinst
image, this check was giving us spurious failures when running
universal tests on a netinst (which we are currently doing on
ppc64 it seems). Tweak it a bit to hopefully avoid this.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-03-01 08:49:08 -08:00
Adam Williamson
6d2031dbfd desktop_browser: fix wait_still_screen, drop wait_idle
wait_idle was deprecated and effectively it just sleeps, so
replace these wait_idles with explicit sleeps - I can't think
of a better choice here as we just need to wait for Firefox to
calm down after various operations.

The wait_still_screens were always hitting the 30 second time
out because of the flashing cursor; setting the similarity level
required to lower than the default 47 should fix this.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-02-13 09:51:34 -08:00
Adam Williamson
965a1a6546 Click instead of using ctrl-t for new browser tab
ctrl-t seems to be failing often in Firefox tests, and I can't
figure out why. Let's try clicking the 'new tab' button instead
as a workaround.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-02-13 09:35:32 -08:00
Adam Williamson
99302c6fd4 Add a live image build test for updates
Just like the installer image build test, only...it builds a live
image. This involves reimplementing quite a chunk of the Koji
livemedia task. Ah, well. Also involves rethinking the flavor
names a bit here, these seem...better.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-02-07 18:28:24 -08:00
Adam Williamson
5cbc7df810 Work around GGO gnome-software/582 till it's fixed
We're getting a spurious "Authentication required" for a
background refresh of the system repos sometimes, depending on
whether it happens while we're at a tty. This is now accepted
by upstream, so let's handle it and make it a soft failure for
now.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-02-07 09:58:10 -08:00
Adam Williamson
8d1d150798 Handle running update tests on Koji tasks
We quite often want to run the update tests on a Koji task (not
a Bodhi update) for some reason - usually to test a potential
fix for an issue, or at a maintainer's request to test a change
before it is merged upstream and officially sent out as an
update. Up till now I've always hacked up utils.pm on the
staging server by hand to do this, which is horrible. Together
with a commit to fedora_openqa, this should allow us to do it in
a nice, sane way via the CLI. It's mostly just tweaking the
"updates" repo setup in utils.pm as you'd expect, but there's a
bit of subtlety to it because of the installer tests that use
%ADVISORY% as a variable substitution in the disk image name;
you can't do something like `%ADVISORY or KOJITASK%`, sadly, so
I had to have almost-redundant variables ADVISORY, KOJITASK and
ADVISORY_OR_TASK (we could kinda just live with ADVISORY_OR_TASK
except I didn't want to drop ADVISORY as it's an unnecessary
change from previous behavior).

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-01-29 11:54:15 +01:00
Lukas Ruzicka
c97c23d485 Recreate needles to make them check different area.
This commit adjusts the area that needles look for to avoid checking
based on a menu item that has disappeared from the system.
2019-01-28 11:11:36 +01:00
Adam Williamson
7aed615a8d Only do the addrepourl check if we're on F28+
inst.addrepo was added in F29. In F28 it should just sorta not
do anything at all. It's not worth doing a big workaround to
make F28 graphically add the repo in this case; let's just allow
F28 to not actually add the repo. It's fine! This is fine.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-01-26 20:49:59 +01:00
Adam Williamson
d1006a38e5 Have update installer test install the update packages (#89)
In https://bugzilla.redhat.com/show_bug.cgi?id=1669256 it became
obvious that there's a missing feature in the new installer test
for updates: the update is both used in the image build process
and built into the installer environment itself, but it is not
actually included in the installed package set. This can be a
problem if the update has a bug that manifests *only* at install
time if it is in the install transaction (which is exactly the
case there), because the test will not catch this, and nor will
any other test.

So this commit makes `support_server` set up the update repo and
serve it out via NFS when it's run in an update context, and
makes the actual update install test run parallel with it and
use that repository. This way the install should include the
package(s) from the update. (It also of course means the test
fails if an update breaks NFS or something like that, but hey,
we want to know that!)

A parallel commit for fedora_openqa is necessary to add the
required CURRREL setting for the updates-installer flavor.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-01-26 19:14:04 +01:00
Adam Williamson
12affb145f Add update tests to build and test a netinst image
This adds a test which builds a netinst image potentially with
the package(s) from the update, and uploads that image. It also
adds a test which runs a default install using that image. This
is intended to check whether the update breaks the creation or
use of install images; particularly this will let us test
anaconda etc. updates. We also update the minimal disk image
name, as we have to make it bigger to accommodate this test,
and making it bigger changes its name - the actual change to
the disk image itself is in createhdds. We also have to redo a
bunch of installer needles for F28 fonts, after I removed them
a month or so back...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-01-18 08:24:44 -08:00
Adam Williamson
510662c1ec Drop all RHBZ #1663040 workarounds, should be fixed now
In Fedora-Rawhide-20190117.n.0 #1663040 should be fixed, so drop
all workarounds for it.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-01-17 08:35:30 -08:00
Adam Williamson
ec762117d0 Wait an extra 2 minutes for the installer 'beta nag' screen
There seems to be an issue in Rawhide ATM which can cause the
'beta nag' screen to take a very long time to appear. Bump the
timeout to avoid tests failing on this.

https://bugzilla.redhat.com/show_bug.cgi?id=1666112

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-01-14 15:22:01 -08:00
Lukas Ruzicka
39d3427471 Create a test suite for application start stop testing.
Merges #86
Fixes #85
2019-01-04 15:23:03 -08:00
Adam Williamson
c6fac00698 More fixing of the workaround...
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-01-04 13:55:31 -08:00
Adam Williamson
42a38009eb Fix workaround (have to wait for desktop to appear)
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-01-04 13:27:01 -08:00
Adam Williamson
4e537684b1 Work around #1663040 in Workstation live installs
We don't want the tests to fail on this now we know what the
bug is, really - we want to find if there are any subsequent
fails, and allow the post-install tests to run also. So, let's
make it a soft failure.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-01-04 13:01:36 -08:00
Adam Williamson
2b29ecbc18 FreeIPA client: set longer timeout for login(1) (#1661273)
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-20 09:02:36 -08:00
Adam Williamson
3d35ec9670 Update a comment to not mention a removed workaround
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-17 14:36:10 -08:00
Adam Williamson
00d1fdbd2c Drop #1600823 workaround
Bug was fixed months ago.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-17 14:30:09 -08:00
Adam Williamson
f40599ee15 Drop all #1594402 workarounds
I'm pretty sure we got all the bugs this was working around
fixed. Again, if not, we can put this back!

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-17 14:27:18 -08:00
Adam Williamson
01fd92cb34 Drop #1444225 workaround
The bug never got explicitly addressed, but it's very old and
anaconda has been substantially changed since. The workaround
sometimes triggers erroneously now (because the icon sometimes
goes black while the spoke is still in 'Checking storage
configuration...' state), which is awkward. I can't be 100% sure
the bug doesn't sometimes still happen, but if it does, we'll
notice fairly soon, and we can tweak this and put it back.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-17 14:18:05 -08:00
Adam Williamson
d6de57c6de Drop RHBZ#1618928 workaround
Bug was fixed back in August.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-17 14:10:58 -08:00
Adam Williamson
7cbd859d81 Remove another old workaround (for #1606541)
This was fixed long ago as well...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-17 13:36:13 -08:00
Adam Williamson
e04ee00e19 Drop various old workarounds from _support_server
I tested a run with these commented out, it worked fine. Seems
the bugs are all fixed now.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-17 13:06:19 -08:00
Adam Williamson
ef3555aaa1 Drop another old workaround (KDE package signature check)
It seems Rawhide auto-signing is working fine now: openQA claims
this needle has 'never' been matched (which really means 'not
for a long time'), and I can't find any test fail in the last
year which looks like it landed on this 'authenticate' screen
but the needle failed to match, or anything like that.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-17 12:24:09 -08:00
Adam Williamson
1fd0097d1d Simplify an 'if >F26' thing
We don't care about anything older than 27 any more.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-17 12:19:44 -08:00
Adam Williamson
d1e7b89efd Fix a potential race in desktop update test
https://openqa.stg.fedoraproject.org/tests/424393 is a failure
where the 'Download' [updates] button was already visible when
we went to the tab. We already checked whether an 'apply' button
is visible and skipped the 'refresh' click if so, but because
the 'download' button is a new thing, we weren't skipping the
'refresh' click if 'download' was already visible.

So in this case, even though we could already see 'download', we
went ahead and clicked 'refresh'...then *immediately* started
looking for 'download'. It seems that Software did not refresh
and remove the 'Download' button *immediately* when we pressed
'refresh' - it left the 'Download' button visible briefly, and
*in this brief window*, we clicked it. *Then* Software kinda
'noticed' we'd clicked 'Update', and it seems it just sort of
throws away our click on 'Download' at that point and does the
refresh.

So at that point, the test thinks it's clicked 'Download' and
expects to see 'Apply', but actually the 'Download' click got
more or less thrown away, so the test fails, sitting at the
'Download' button.

To solve this, let's just extend the existing check to skip the
'refresh' click if 'download' *or* 'apply' are already visible.

There is a sort of possibility here that we could wind up
downloading and installing some updates that existed and were
noticed *before* we did our python3-kickstart trick, but not
install the python3-kickstart update, and cause the test to fail
because of that, but that doesn't seem to have happened before
when we were seeing the 'update' button, so I think I'm not
going to borrow trouble. If it happens, we'll deal with it I
guess.

The comment talks only about KDE, but clearly it can be the case
that an automatic check makes the button visible on GNOME too,
so let's rewrite the comment too.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-17 12:10:06 -08:00
Adam Williamson
517750443e Remove RHBZ #1638563 workaround
The fix for this bug was sent to all releases now, so we should
not need the workaround any more. Let's kill it.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-17 12:02:57 -08:00
Adam Williamson
0498f4db94 Use 30s timeout on the check_screen in the workaround
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-13 15:29:35 -08:00
Adam Williamson
b3f51e6108 Gah come back missing bracket
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-13 15:23:50 -08:00
Adam Williamson
1f7e5ebe20 Attempt a workaround for RHBZ#1659266
This is breaking the memory_check tests. I just reproduced it
manually and the UI *does* come back to life if you wait some
time; let's see if we can work around the bug this way.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-13 15:15:49 -08:00
Adam Williamson
12e103e3da Factor meat out of advisory_post and do it in postfail too
If an update test fails before reaching advisory_post, we don't
generate the 'what update packages were installed' and 'were
any update packages *not* installed when they should have been'
logs, but these may well be useful for diagnosing the failure -
so let's also do the same stuff there. Only let's not do it all
twice.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-12 22:17:29 -08:00
Adam Williamson
764c6dbd95 Notice when update package should have been installed but wasn't
We hit an interesting case in update testing recently:

https://bodhi.fedoraproject.org/updates/FEDORA-2018-115068f60e

An earlier version of that update failed testing. When we dug
into it a bit, we found that the test was failing because an
earlier version of the `pki-server` package was installed than
the version that was in the update; when asked (as part of
FreeIPA deployment) to install it, dnf had noticed that there
were dependency issues with the version of the package from the
update, but it happened to be able to install the version from
the frozen 'stable' repo...so it just went ahead and did that.

In this case, the 'missed' package resulted in a test failure,
but it'd actually be possible for this to happen and the test
to complete; we really ought to notice when this happens, and
treat it as a test failure.

So what this attempts to do is: at the end of all update tests,
check for all installed packages with the same name as a package
from the update, and compare their full NEVR to the one of the
package from the update. If a package with the same name as one
of the update packages is installed, but does not appear to be
the *same NEVR*, we fail, and upload the lists of packages for
manual investigation as to what the heck's going on.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-12 22:17:29 -08:00
Adam Williamson
e76ad4bbc1 Crank sss debuglevel up to 9, and also do it for cockpit test
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-11-30 14:58:53 -08:00
Adam Williamson
14ad5b97f1 Still try and upload testedpkgs even if comm 'fails'
From local experimentation, it still actually produces the
output, even though it prints the message about the order being
wrong and exits 1.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-11-08 17:46:45 -08:00
Adam Williamson
ddf6ba5a6b update tests: don't fail if comm is unhappy about the alphabet
Weirdly, occasionally some update tests seem to fail because
the 'comm' util we use to produce the list of packages from the
update that were actually tested during the job doesn't think
one of the input files is in alphabetical order, even though we
sort them both when they're produced. I don't know if this is
possibly due to the definition of 'alphabetical order' changing
as part of the update, or what. But we really shouldn't *fail*
the test when this happens, as it's not part of the functional
test, we're just producing convenience data. So, let's handle
the command failing, and if it happens, upload the input files
so we can maybe figure out why it's unhappy...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-11-08 16:59:06 -08:00
Adam Williamson
70ef3404f0 Tweak previous commit to avoid some bugs
The previous commit would lead to the 'workaround' getting hit
incorrectly, and might have had some other issues...tweak it a
bit.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-10-31 12:45:11 -07:00
Adam Williamson
fd753b2e3a Handle split of 'download' and 'apply' phases in gnome-software
GNOME Software 3.30.5 split the offline update process into two
separate 'download' and 'apply' phases. So we need to handle
clicking 'download' before 'apply', if that happens.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-10-31 11:50:14 -07:00
Adam Williamson
e6c8c5f0ff Work around Firefox 'close multiple tabs' warning
Somehow, recently, FreeIPA tests are running into Firefox not
quitting because it's showing a warning about closing multiple
tabs. (I think we didn't *get* multiple tabs before but now we
do, for some reason). So let's work around this by clicking
"Close tabs" if the warning appears.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-10-30 18:34:37 -07:00
Adam Williamson
129d1316a6 Fix tty switching for desktop_notifications
Lately, we can't be sure the desktop will be on tty1 after we
do 'systemctl isolate graphical.target'. For recent Workstation
lives it actually shows up on tty2.

We could be 'clever' and switch to tty2 on F29+ Workstation
lives...but actually it seems like if we just don't do anything,
systemd switches us to the correct tty. So let's rely on that,
at least as long as it's working.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-10-12 21:16:33 -07:00
Adam Williamson
22c0b5bc04 Disable hidden grub menu for uploaded base installs
At least one test (desktop_notifications_postinstall) boots from
the disk image uploaded by install_default_upload, and needs to
access the grub menu. On F29+ Workstation this is failing,
because the grub menu is now hidden by default, so when the test
boots, it never sees the bootloader screen, and fails.

I considered trying to teach it to hold down shift or hit f8 or
esc at the right time, but that seems like it might be hard. So
instead let's just try to disable the hidden menu when we're
about to upload the installed system image. This is kinda going
against the 'preserve natural system behaviour' principle we try
to use for openQA, but I think it's OK as we do have other tests
that will exercise the 'hidden boot menu' stuff to some extent.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-10-12 15:36:55 -07:00
Adam Williamson
17b6d9f708 Tweak the workaround to work for F27 too
On F27 we don't get a 'Software is up to date' screen because
there's an upgrade available. Let's work with the refresh button
instead.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-10-11 22:23:53 -07:00
Adam Williamson
63d8f34a0e Tweak the workaround loop a bit, refresh the comments
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-10-11 16:18:04 -07:00
Adam Williamson
db4ab638da Restore modified version of the #1314991 workaround for #1638563
We're not seeing *exactly* #1314991 any more, but we're seeing
something that looks quite similar: the first attempt to find
updates just doesn't find any. No error message, no updates. I
have reported a bug for this and am investigating it, in the
meantime, let's restore the workaround, elaborated a bit, and
looking for the 'Software is up to date' screen instead of the
error message.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-10-11 16:07:58 -07:00
Adam Williamson
25ad8a6aeb Drop workaround for #1314991, it doesn't work any more
I rather suspect the *bug* is still basically present and it's
why this test often fails, but we no longer seem to see the
*error message* which lets us detect the bug happening. This
needle has not been hit by any test for six months. So let's
remove the workaround as it adds complexity.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-10-10 13:45:51 -07:00
Adam Williamson
9869920f5b Use longer timeout for root console switch after liveinst
For some reason, in recent tests, switching to a console after
live install completes is taking a long time, and tests are
failing because we 'only' allow 10 seconds for the login prompt
to appear. This seems to indicate some kind of performance bug,
but we don't really want all liveinst tests to fail on in, this
is not primarily a performance testing framework. So let's
tweak the root_console / console_login bits a bit to allow a
configurable timeout for the login prompt to appear, and use
that to wait 30 secs instead of 10 in this case.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-10-06 08:44:34 -07:00
Adam Williamson
6e262be28b Add a check that FreeIPA is actually up after upgrade
The FreeIPA upgrade test didn't actually check that FreeIPA is
actually running after the upgrade and reboot, it just kinda
assumed it is. Let's add a check to the start of the 'check'
test module that makes sure ipa.service actually comes up to
'active' state. This'll make it clearer when tests are failing
because FreeIPA didn't come up right after the upgrade. The
check will run on non-upgrade tests too, but that's fine.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-10-05 23:53:17 -07:00
Adam Williamson
0bf76db7d5 Add a test of bootchain stuff for updates
This adds a new test intended to just check boot chain things
for updates. It doesn't run any test modules besides the stock
update ones, but sets a variable, ADVISORY_BOOT_TEST, which
causes _advisory_update to do some additional stuff after
installing the updates but before rebooting: it forces regen
of the initramfs and bootloader config, and reinstalls the
bootloader on BIOS (not UEFI as it's not relevant). If the
following boot fails, we probably have a bug somewhere.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-10-04 16:43:15 -07:00
Adam Williamson
d2d6bfa695 Try something different for screen corruption after IPA uninstall
There's this annoying problem where the screen sometimes goes
messed up after ipa-server-uninstall. 'clear' doesn't seem to
really work to fix it up either. Let's try flipping between
ttys. I don't like this much as it's already a pain trying to
work out / remember what tty we might possibly be on at any
given time, but I think we're always on either 1 or 3 here, so
let's do ctrl-alt-f1 ctrl-alt-f3 to ensure at least one change
and wind up on tty3...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-09-28 18:09:58 -07:00
Adam Williamson
c2fb886d6f Add another wait to avoid a transition animation in anaconda
I swear, these transitions drive me nuts.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-09-28 17:46:36 -07:00
Adam Williamson
a49f328dc6 Tweak how update-upgrade tests are handled a bit
Looking at this, it's a bit weird: the updated packages are
actually included in the upgrade process, but we still run
_advisory_update, which does basically nothing...then reboots.
That's kinda silly and makes the tests a bit flaky, let's fix
it. I don't think there's actually any problem with doing the
upload of updatepkgs.txt in _repo_setup_updates, becase that
already guards against being run more than once, it just bails
very early if it's already been run.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-09-28 17:23:51 -07:00
Lukas Ruzicka
24e68aa8a2 Create openqa tests to test modularity. 2018-09-26 23:09:36 -07:00
Adam Williamson
923267574d Drop the workaround for #1625572
It's fixed everywhere now, and the workaround can misfire if
the first g-i-s is slow quitting (see
https://openqa.fedoraproject.org/tests/280482)
2018-09-16 08:39:30 -07:00
Adam Williamson
f334df1337 Bump ipa-replica-install timeout a bit (it takes a long time)
I'm going to figure out if it's a bug that it takes so long, but
for now let's just bump the timeout.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-09-07 15:56:11 -07:00
Adam Williamson
e2eb794a87 Tweak the workaround yet again (with soft fail this time)
Sigh, sorry, just perfecting. This way it won't fail when the
bug is fixed (hopefully).

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-09-06 20:36:29 -07:00
Adam Williamson
7dff1843db Tweak that workaround again (forgot the version check)
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-09-06 19:53:07 -07:00
Adam Williamson
211cc221b3 Rejig the 1625572 workaround to just use an existing conditional
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-09-06 19:00:49 -07:00
Adam Williamson
af6b9b15aa Gah, fix version testing in previous commit
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-09-06 18:23:13 -07:00