Commit Graph

1542 Commits

Author SHA1 Message Date
Lukas Ruzicka
d82f4ecd7f Develop a new test for system updates via Cockpit.
* server-cockpit-updates tests that Cockpit can be used to update the system.
* server-cockpit-autoupdate tests that users can use dnf-automatic for system
updates.
* cockpit functions were removed from utils.pm and put into an extra library
for cockpit - cockpit.pm which all cockpit tests are now using.

Review cockpit.pm

Review autoupdate test.

Review the update test.

Fix typo in cockpit.pm

Add sleep.

Add missing command.

Delete an unused needle.
2019-10-01 16:04:01 +02:00
Adam Williamson
0782f4faf2 Add new akonadi migration agent needle for F31 background
Light blue throwing us off again.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-09-26 10:24:37 -07:00
Adam Williamson
8be0541ce7 Tweak live_build mock handling to work with templates
mock is changing the config files to be based on templates,
which breaks how we generate the live build mock config. This
should work with both the old and new config files.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-09-23 16:00:08 -07:00
Adam Williamson
b32449f72a Add missing bracket
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-09-20 11:04:31 -07:00
Adam Williamson
6538a82096 Download by ADVISORY_NVRS with koji for updates, not ADVISORY
This goes with a corresponding fedora_openqa commit that passes
ADVISORY_NVRS when scheduling update jobs. This is to address
https://pagure.io/fedora-qa/fedora_openqa/issue/78 so we can
publish spec-compliant and correct CI Messages: we will decide
what NVRs are in the update at scheduling time and always get
and test those NVRs, rather than the test downloading whatever
is in the update when it is run.

Some consequences: manual restarting or cloning of an update
test scheduled before this change lands will result in failure;
to do this you'll have to add the ADVISORY_NVRS value manually
with clone_job.pl, or simply reschedule the update with
fedora-openqa.

Manual restarting or cloning of an update test scheduled with
this change will always test the same NVRs. If the update has
changed and you want to re-test with the new packages in the
update, you must change ADVISORY_NVRS manually with clone_job,
or reschedule with fedora-openqa.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-09-20 10:58:27 -07:00
Adam Williamson
7f96c3aa49 Remove a now-unneeded workaround
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-09-20 10:51:16 -07:00
Michel Normand
a631030698 Add Workstation-live-iso all tests for ppc64le
Add HDDSIZEGB for Workstation-live-iso ppc64le
as install_default_upload needs at least 11.07GB

Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2019-09-20 13:11:38 +02:00
Michel Normand
200913547c Add Everything-boot-iso install_default for ppc64le
Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2019-09-20 13:08:13 +02:00
Michel Normand
f247ea7511 Remove install_default when install_default_upload
two tests only differ by saving of qcow2 on test end for last one.

That concerns ppc64le and aarch64 flavors:
    AtomicHost-dvd_ostree-iso
    Server-dvd-iso

Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2019-09-20 13:08:13 +02:00
Michel Normand
c2a59b8f01 Add more missing tests for ppc64le
server-dvd-iso:
    release_identification
    server_freeipa_replication_master
    server_freeipa_replication_replica
    server_freeipa_replication_client

Silverblue-dvd_ostree-iso:
    release_identification

universal:
    install_package_set_kde
    install_updates_img_local

Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2019-09-20 13:08:13 +02:00
Guy Menanteau
3e6568c7df Add base_system_logging test for ppc64le
Signed-off-by: Guy Menanteau <menantea@linux.vnet.ibm.com>
2019-09-20 13:08:13 +02:00
Michel Normand
6455319ef5 typo in lib/utils.pm
reported in autoinst-log.txt
===
"my" variable $files masks earlier declaration in same scope at /var/lib/openqa/share/tests/fedora/lib/utils.pm line 352.
"my" variable $beta masks earlier declaration in same scope at /var/lib/openqa/share/tests/fedora/lib/utils.pm line 995.
===

Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2019-09-20 07:15:21 +02:00
Adam Williamson
61f8adefdf Add some new update-related needles for KDE
The new background seems to be throwing us off here. I think
the menu is *very slightly* transparent so the needles usually
work for most Fedora backgrounds as they tend to have a similar
base color, but the 31 one is unusually light.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-09-19 10:23:21 -07:00
Adam Williamson
6e9a7a57c5 Add several more variant keyboard layout needles for new icon
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-09-16 12:26:26 -07:00
Adam Williamson
96cf14d555 Add new variant us_rtl layout needle
For RTL languages the keyboard icon and language indicator are
the other way around, so we have different needles even for the
US layout indicator which is otherwise the same in all langs.
This is a variant needle for the new keyboard icon that showed
up in Rawhide recently.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-09-16 11:32:58 -07:00
Adam Williamson
0f9fd1a76a Add alternative vdb selected needle for new blue checkboxes
This checkbox sometimes renders slightly differently, especially
on non-x86_64. We're not really sure why, but we just add more
needles to handle it.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-09-16 11:23:47 -07:00
Adam Williamson
ea11c8344f Shrink KDE Firefox launcher icon to avoid background issues
Existing one doesn't match if the background is too different,
shrink it to just the center of the icon.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-09-11 15:12:03 -07:00
Adam Williamson
c0f911067f Disable prerelease text check in disk_guided_empty
This is causing all kinds of trouble, because when the test is
run on the Server DVD - with the 'orange to blue' gradient - the
prerelease note is dark red text on a dark blue background.
os-autoinst actually reduces the color depth of images/needles
and greyscales them before performing the match...but for this
dark red text on dark blue background, the result seems to be
that the text and background come out *the same grey*, so *any*
text will match the needle (even if it's completely different
text), as will *no text at all*. I've tried finessing around
this a few times but it just keeps happening, so for now I'm
just disabling the pre-release text check at this point. We still
have the check during _boot_to_anaconda, when the text appears
on a *grey* background and so isn't a problem. I'm not removing
the needles yet, until we hear back from upstream:

https://progress.opensuse.org/issues/56822

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-09-11 15:01:54 -07:00
Adam Williamson
a1d9b14383 Some more prerelease needle updates 2019-09-11 15:01:44 -07:00
Adam Williamson
dc8ed1c9ba Yet more pre-release check fixes
We need another needle for French installs, and we need to use
ISO_URL for checking the ISO file name if ISO isn't set.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-09-11 14:31:54 -07:00
Adam Williamson
29a5e0bda7 Add variant 'permission denied' needle
For some reason, occasionally the 'm' character looks slightly
different. I have no idea why.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-09-11 10:47:13 -07:00
Adam Williamson
be1baa90e5 Update main hub needles for changed keyboard icon
Someone is pretty OCD about this damn keyboard icon, it just
keeps getting changed...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-09-09 11:44:28 -07:00
Adam Williamson
a3c207a045 Add a few more 'blue button' needle variants
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-09-05 18:09:14 -07:00
Adam Williamson
23ad087ed3 Add blue button variants for workstation_selected needle too
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-09-05 17:28:41 -07:00
Adam Williamson
cf6f8db3e7 Add new variant 'minimal selected' needle for radio button change
Radio buttons look different with recent GTK, it seems.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-09-05 16:14:59 -07:00
Lukas Ruzicka
1d6e2aaa19 Add changed needle that failed for cyrillic test. 2019-09-05 17:10:50 +02:00
Lukas Ruzicka
46bc71afd6 Delete the KDE test for Release Notes due to retirement. 2019-09-05 11:41:18 +02:00
Adam Williamson
df88196357 Add F31 background needle
Now we have the backgrounds!

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-09-04 12:44:55 -07:00
Adam Williamson
243dfbdacd desktop_vt: hackaround 'tty?' problem on F31
On F31 update tests, desktop_vt is broken because the 'ps -C'
command is showing 'tty?' as the tty on which Xwayland/Xorg is
running. Let's try using loginctl as a workaround for this.
This ordering of commands should ensure the ps -C output takes
precedence when it's correct.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-09-03 16:19:08 -07:00
Adam Williamson
6d68b2c6fa Add yet another prerelease note needle
Seriously starting to think about separating out this ident
stuff into a separate test case :/

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-29 17:49:27 -07:00
Adam Williamson
8b236592d9 Disable SELinux for live image build test
Building images with SELinux in permissive mode worked fine in
F29 and F30, but it seems to be broken in F31. Releng seems to
have it set up so the build environment has SELinux disabled,
so let's do the same. Will file a bug for this, as bcl says it
should be considered a bug.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-29 17:49:27 -07:00
Adam Williamson
53778fe494 Add language tags to the new language prerelease note needles
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-28 14:46:23 -07:00
Adam Williamson
b19e6bbda7 Add some needles for pre-release note translations
These showed up because I did a manual universal flavor run on
an ISO today, but we'll need them as soon as we do a candidate
compose too.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-28 14:14:57 -07:00
Adam Williamson
2cb224a184 Make generic topbar needle narrower
We did this for the orange (Server) one already, but not the
blue generic one.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-28 14:07:33 -07:00
Adam Williamson
26308a6b47 Update rarely-used installer ext4/xfs partitioning needles
These only get used when we run universal tests on a non-Server
image, which is pretty rare (these days, as Server DVD is a
critical image and composes fail if it fails, it only really
happens when we do it manually for some reason, like I did
today). So they get stale and aren't updated for font rendering
changes and stuff. As I said I had to do a run like that today,
so I had to update all these needles...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-28 14:04:13 -07:00
Adam Williamson
567b2f1a3f Wait longer for console to appear after quitting Firefox
A couple of tests failed in F31/Rawhide lately because quitting
Firefox (running on a bare X server) took more than 10 seconds
to get back to a console...not sure why, but it's not something
we ought to waste a lot of time on, let's just bump the timeout.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-28 11:07:13 -07:00
Adam Williamson
b9934c9816 Fix prerelease check for Branched update tests
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-27 08:36:40 -07:00
Adam Williamson
4bb89046d3 Add one more updated GNOME app test needle
'Simple Scan' renamed itself.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-26 17:51:20 -07:00
Adam Williamson
28443abf04 Add new variants for GNOME app start/stop tests
Several apps changed icon.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-26 16:29:37 -07:00
Adam Williamson
77a6c0ed4d Add yet another variant 'vdb selected' needle
This screen often seems to show slight variances in color,
really not sure why.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-26 16:28:44 -07:00
Lukas Ruzicka
022865ab9f KDE app start/stop: launch apps with commands not menus
This is doing slightly less to exercise the launcher menus and see
whether icons appear or disappear, but it's much faster and more
reliable. We do still use menu launch for one app, just to check
the mechanism works in general.
2019-08-23 12:41:49 -07:00
Adam Williamson
06e26e9721 Narrow topbar and add new version needle to cover languages
The top bar differs in non-English languages - for RTL the
version ident is on the left (so background is different), and
the width of the area with no text differs depending on the
length of the translations.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-21 14:36:53 -07:00
Adam Williamson
bf5691e69d Fix and simplify fedora_release test
Also simplify os_release a bit thanks to the improved spell_
version_number.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-21 13:09:40 -07:00
Adam Williamson
23552c777d Add a new KDE desktop update needle variant
Little black bits on the icon are in a different place now. Why,
I do not know.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-21 11:55:07 -07:00
Adam Williamson
efbfe95213 Fix wrong repos_mirrorlist call which broke all update tests
Ugh. Sorry. Teach me to fire off changes without testing them.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-21 11:29:30 -07:00
Adam Williamson
ecfc9a5234 Add new variant abrt_runs needle for KDE app tests
Text rendering changed.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-21 10:44:25 -07:00
Adam Williamson
86488d2611 Expect Workstation branding on Silverblue installer
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-21 00:01:47 -07:00
Adam Williamson
397dc9a9cf Don't redefine $prerelease
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-20 23:42:01 -07:00
Adam Williamson
2f51f3ef20 Add PXE boot test
This adds a whole wodge of stuff to support_server to make it
act as a PXE server, then adds a new test which boots from PXE
and so should hit the PXE server. We use the NFS install repo as
that can be relied on to work for a support_server install.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-20 18:17:57 -07:00
Adam Williamson
a177a63ec1 Factor out s/metalink/mirrorlist/ into a util function
We do this in quite a few places and need to do it in another,
so let's just have a function for it. It takes a file glob
so we can have it run on a different one for _live_build.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-20 14:16:23 -07:00