Commit Graph

1918 Commits

Author SHA1 Message Date
Adam Williamson
a9704de4bd FreeIPA: disable dnssec validation till weird bug is fixed
I noticed today that if we deploy FreeIPA with dnssec validation
enabled, dnf can't resolve dl.fedoraproject.org afterwards, which
is a problem because it means we wind up falling through to
random mirrors for metadata and package download once the server
is deployed, which can be slow and give old packages. This seems
to be why the server upgrade test on F33 is sometimes failing
because we get an older FreeIPA package on upgrade, even though
the newer one has been stable for a week.

It's difficult to pin down exactly where this bug is and fix it,
I've mailed some folks to try and work it out, but until that's
figured out, let's just disable dnssec validation.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-10-23 11:38:01 -07:00
Adam Williamson
0d8ceec820 Try to make desktop_browser more robust
We've been getting failures lately on the first page load, I
think because Firefox is getting even more grindy on startup. So
turn the 'sleep' into a 'wait_still_screen', extend another wait,
and tweak the 'browser' needle so it only matches after the
bookmark bar has loaded rather than as soon as half the chrome
appears. Also make all the wait_still_screens use similarity 45
for consistency (flashing cursor could be there on any of them).

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-10-23 09:48:09 -07:00
Adam Williamson
bd7d3cd663 Fix desktop_terminal command check (thanks defolos)
This check wasn't working, the test passed whatever wait_serial
found. This version suggested by defolos works, I checked.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-10-15 16:25:26 -07:00
Adam Williamson
33c290d15a Remove now-unused desktop_network_notification needle
Dropped the use of this in the recent notification simplification,
but forgot to remove the needle. Should've run tox...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-10-15 15:36:37 -07:00
Adam Williamson
e5f396a785 Drop workaround for fixed GNOME submenu bug
https://gitlab.gnome.org/GNOME/gnome-shell/-/issues/2878 was
fixed (or just rendered obsolete by a change to how submenus
work), so we don't need this any more.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-10-15 15:36:09 -07:00
Adam Williamson
1c33d07d38 Drop workaround_ble26, bug was fixed months ago
https://pagure.io/background-logo-extension/issue/26 was fixed
months back, we don't need this any more.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-10-15 15:36:04 -07:00
Adam Williamson
a008ffb8be Simplify desktop notification checks (#195)
This is the best option I can come up with to deal with #195.
Update notifications seem to have become transient in KDE lately
(even in F31 and F32, if I'm looking at these screenshots right).
This actually simplifies things a lot to do more or less the
same in the KDE and GNOME paths: open the 'permanent' store of
notifications (in GNOME you get to it by clicking on the clock,
in KDE via the systray) and then look for no notifications (live
path) or only an update notification (post-install path). We
only run this test for composes so we shouldn't need to worry
about anything older than F32, and I believe this should work
for KDE in F32 and F33. I left out click_unwanted_notifications
for now as I'm hoping it should be unnecessary, but we can add
it back in if necessary.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-10-14 23:30:00 -07:00
Adam Williamson
656e1f6856 Add another F33 background needle
Yeesh, this background is fun.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-10-11 10:04:37 -07:00
Adam Williamson
db99e5462c Drop F33 FreeIPA update from workarounds, it went stable
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-10-10 17:27:33 -07:00
Adam Williamson
da0b079686 Update KDE desktop no notifications needle
Probably changed with the background, darn translucency.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-10-09 18:06:19 -07:00
Adam Williamson
415dabd92e Update Firefox addon_install alt needle too (for F31)
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-10-09 17:53:22 -07:00
Adam Williamson
10f7e7a7d2 Update Firefox addon_install needle for 81.0.1
Seems to have changed.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-10-09 17:25:10 -07:00
Adam Williamson
9de75af60b Drop f33-backgrounds from workarounds (it went stable)
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-10-09 16:57:21 -07:00
Adam Williamson
94b47afc53 Tweak setup_tap_static and FreeIPA tests for resolved
This does some of the things suggested by cheimes in
https://bugzilla.redhat.com/show_bug.cgi?id=1880628#c24 . It
seems to make the replica tests work with resolved, still work
with pre-F33 resolving, and not break anything. Also remove the
workaround to disable resolved if it's running, as we can now
work with it.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-10-09 16:54:46 -07:00
Adam Williamson
92f5bf90c0 Update Krusader settings close needle
The button changed a bit.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-10-08 16:22:44 -07:00
Adam Williamson
40e07b487d Add another F33 background needle
For the transition between dawn and day. This is right in the
middle of it.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-10-08 16:21:59 -07:00
Adam Williamson
40974c2f94 Simplify Krusader app test
We don't need a separate 'welcome' needle because it just matches
on an OK button anyway. So turn that needle into an OK needle
(we don't have any existing 'blue OK button' needle) and simplify
the logic to a single loop for kde_ok and krusader_settings_close.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-10-08 14:47:46 -07:00
Adam Williamson
401ebecc10 Add F33 FreeIPA 4.8.10-5 update as workaround to fix upgrades
FreeIPA upgrade test is failing because of
https://bugzilla.redhat.com/show_bug.cgi?id=1886205. The test
failing every time is not useful as we know what the issue is,
so add the update as a workaround to avoid it.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-10-08 11:28:10 -07:00
Adam Williamson
e89de5fffb Bump a timeout a bit for aarch64
Seems like 10 seconds here isn't always long enough for aarch64,
so bump it.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-10-07 15:21:17 -07:00
Adam Williamson
200cab3899 disk_custom_resize_lvm: add some waits
On ppc64le it looks like this test is often failing because it
takes a second or two to update the partition list after we
click update settings, but we're not waiting for that, so we
wind up clicking in the wrong place because we match the next
partition needle before the list is refreshed but click after
it's refreshed. Let's hope these waits solve it.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-10-07 14:52:49 -07:00
Adam Williamson
5e5dbac636 Add updated cockpit_updates_reconnect needle
We only use this one rarely. Font rendering on it changed at
some point.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-10-07 14:42:26 -07:00
Adam Williamson
d2adecf03d Add some more updated F33 background needles
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-10-07 14:41:54 -07:00
Adam Williamson
36fd4c10b1 Correct tag for new kde_ok needle
It had krusader_settings_close as its tag, not kde_ok. That's
why the krusader test module was failing weirdly.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-10-07 14:39:08 -07:00
Adam Williamson
4fcea15610 Updates testing: add f33-backgrounds workaround
Add FEDORA-2020-27f80050a2 as a workaround because without it the
KDE desktop background test fails due to the bug in -6.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-10-07 12:40:27 -07:00
Adam Williamson
8d9684dd80 Update tests: don't install git any more
We were using it to checkout a git version of python-fedora to
work around a bug, long ago, but we don't do that any more so
we don't need it.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-10-07 10:19:33 -07:00
Adam Williamson
aafe460f85 Update testing: drop all workarounds
They're all stable now.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-10-07 10:09:59 -07:00
Adam Williamson
1fdf7cd567 Handle update build not having any packages for target arch
openQA sometimes winds up testing an update that doesn't have
any packages for x86_64 (or aarch64). The most common case is
s390utils, which is on the critpath but only has packages for
s390x. I would ideally like to skip scheduling entirely if the
update has no packages for the arch we're scheduling on, but
sadly that involves using the Koji API which is XML-RPC and I
don't really want to deal with that again. This deals with it
at the test level instead, by checking the error message if
`koji download-build` fails and carrying on if it's the "no
packages for this arch" error. That means if the update has no
packages at all for our arch we're not really testing anything,
but that's better than a bunch of false failures, I guess.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-10-07 09:49:03 -07:00
Adam Williamson
ea0dc63fc3 Add updated F33 day background needle
The backgrounds were tweaked so I get to take a bunch more
screenshots, fun fun fun.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-10-06 14:54:58 -07:00
Lukáš Růžička
b5a7b49f6e Correct the misconfigured needle from the previous commit and add a proper one. 2020-10-01 12:42:56 +02:00
Lukáš Růžička
3986984eb3 Fix needles for Krusader and Notifications. 2020-09-30 09:08:20 +02:00
Dan Čermák
3316e04d20
Update live_anaconda_icon KDE needle 2020-09-20 11:59:16 +02:00
Adam Williamson
fdf142dbd5 Disable systemd-resolved before deploying FreeIPA server/replica
Having systemd-resolved in use seems to cause problems for
FreeIPA servers:
https://bugzilla.redhat.com/show_bug.cgi?id=1880628
until the scripts are enhanced to do this or something, let's
disable it before server/replica deployment.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-09-18 13:49:11 -07:00
Adam Williamson
aab6935707 FreeIPA replica: don't re-do setup_tap_static after deployment
ipa-replica-install already changes the DNS config to use the
local bind instance, we don't need to do this and it's actually
wrong (as it bypasses the local BIND we should use and uses
the VM host's DNS servers instead).

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-09-18 13:43:28 -07:00
Adam Williamson
fba3a3a290 Update Cheese and Videos app needles, icons changed
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-09-18 13:40:23 -07:00
Adam Williamson
fb1bc1dad8 Add mock 2.6 to workarounds, drop kernel for F31 and F32
Kernel updates for F31 and F32 went stable so they can come out.
mock 2.6 fixes the bug that occurs when /etc/resolv.conf is a
dangling symlink - this breaks the live_build test.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-09-17 16:40:53 -07:00
Adam Williamson
c2e7ddba45 Fix DNS config in realmd_join_sssd and realmd_join_cockpit
Seems what they had before worked until systemd-resolved became
the default; now we need to make sure we do nmcli mod and then
bring the connection down and up, as we do in tapnet.pm. Writing
to resolv.conf is kinda "wrong" for resolved but I don't think
it really breaks anything so I think I'll just leave those bits
in until F32 goes EOL just in case they're still somehow needed
on F31 or F32.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-09-17 16:01:15 -07:00
Adam Williamson
f69e627dc3 Update some cockpit needles for version 228
They dropped Title Case from various things, some of which we
have needles for.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-09-16 12:27:54 -07:00
Adam Williamson
314f8f84eb Another attempt to improve robustness of desktop_browser
https://openqa.fedoraproject.org/tests/667693#step/desktop_browser/8
shows us matching on Save File when the window is in kind of a
borked state; we'd probably wind up clicking on Open with,
because by the time we click the content of the window will have
moved to where it's actually supposed to be...so let's try this
to slow it down a bit.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-09-16 11:55:31 -07:00
Adam Williamson
ed5c06baa8 Upload pylorax.log when done building installer image
Handy to have it around to check for oddities.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-09-15 14:54:44 -07:00
Adam Williamson
a5d37e4c67 Need to bind mount the workarounds repo too
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-09-15 14:44:34 -07:00
Adam Williamson
e4d89b6d6b Tweak how we add repos to mock config a bit
Not sure if the other way was valid.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-09-15 14:25:05 -07:00
Adam Williamson
3dd33e3ef1 Use workarounds repo for installer and live build tests
Need this to pull in the kernel fix that's breaking install tests
at the moment.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-09-15 14:21:07 -07:00
Adam Williamson
ee666d5f13 Add kernel 5.8.9 updates to workarounds (fix parted problems)
Kernel 5.8.8 broke the installer by changing return codes for
partition operations, these updates are listed as fixing it.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-09-14 18:48:08 -07:00
Adam Williamson
d86960166d drop FEDORA-2020-2bef864cab as a workaround (it went stable)
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-09-14 18:47:23 -07:00
Adam Williamson
a2bbd5ecfb Add some needles for universal tests with btrfs default
The 'universal' tests have quite a few failures if you run them
on an image where btrfs is the default (currently they usually
run on the Server DVD, where xfs-on-LVM is still the default).
This fixes some of them, the others would need code fixes.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-09-11 16:20:12 -07:00
Adam Williamson
9824f20566 Slow down desktop_browser a bit to try and make it more reliable
Getting some odd failures where the downloaded file doesn't show
up in the right place which I think might be due to over rapid
clicking here. Try and slow it down a bit.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-09-11 16:18:15 -07:00
Adam Williamson
0350977ba3 Update workarounds: drop stable one, add F33 FreeIPA upgrade one
The F32 FreeIPA update that changed the web UI has gone stable
now, so remove it. A pki-core update has just come out that fixes
F32 -> F33 FreeIPA server upgrade test: add this as a workaround
for F33 so that test stops failing on every update.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-09-08 14:13:29 -07:00
Adam Williamson
0999921fe3 Tweak detection of installed update packages
Previously we were relying on `rpm -q` always outputting the
right package last. We saw some test failures on recent kernel
updates, e.g. https://openqa.fedoraproject.org/tests/658768 ,
which indicate this isn't always the case; there the 'right'
package was second of three for kernel, third of three for
kernel-core and first of three for kernel-modules. So we need to
make it more robust. This uses an additional call:
`rpm -q $pkg --last | head -1` to find the most recent package,
if there are more than one; this should always be the right one,
I hope. Note we cannot just add `--last` to the `rpm -q --qf...`
call because the output when you do that is weird; you get the
output you'd get if you just called `rpm -q --last` first, and
*then* the query-formatted output afterwards (though with the
modified order as expected). There doesn't seem to be any way to
get only the latter.

I also tweaked the log uploading so we always upload the working
logs even when the test passes; it can't hurt anything and it is
sometimes useful to have them.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-09-08 14:03:23 -07:00
Adam Williamson
8e794a6c30 Update a Cockpit needle that changed in the latest release
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-09-03 15:13:54 -07:00
Adam Williamson
e99a3fbdf9 os_release: adjust for Fedora CoreOS
...which just has to be another special flower.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-09-02 16:22:20 -07:00