Commit Graph

220 Commits

Author SHA1 Message Date
Adam Williamson
54a6b21c28 Run autocloud tests on Cloud_Base qcow2 images
autocloud is dying soon. So, we want to run the tests in openQA
instead. This adds a test module called 'autocloud' and a test
suite called 'cloud_autocloud' which basically replicate what
autocloud does (download a tarball full of tests and run each
one), and the necessary template bits to run it on Cloud_Base
qcow2 images.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-21 14:29:43 -08:00
Lukáš Růžička
5a28c0c0f2 Add uefi variations of the minimal upgrade test case. 2019-11-20 19:00:13 -08:00
Michel Normand
bbb5d76849 install_vnc_client & install_vncconnect_client to be ARCH specific
Use  ARCH specific desktop_4 images supposed to be
generated by createhdds tool.

Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2019-11-20 14:05:28 -08:00
Adam Williamson
00c81f9a30 Drop 'anaconda' from serial console test name
It's really not useful.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-20 09:35:23 -08:00
Michel Normand
73ca0f2d3f Add missing groupe_name for PowerPC and AArch64 added tests
That concerns:
  test_suite => { name => "install_vnc_server" },
  test_suite => { name => "install_vnc_client" },
  test_suite => { name => "install_vncconnect_server" },
  test_suite => { name => "install_vncconnect_client" },

Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2019-11-20 08:46:43 +01:00
Lukáš Růžička
b93a197c22 Enable Anaconda Text install via serial console.
This adds the Anaconda text installation test over
serial console and FIXES #115.
2019-11-19 22:54:55 -08:00
Adam Williamson
3e6764dbb5 Fix French install test on aarch64 by skipping uefi_postinstall
The French install/boot test fails on aarch64 due to a bit of an
ordering issue:

https://openqa.stg.fedoraproject.org/tests/665124

we run `uefi_postinstall`, which does `loadkeys us`, before we
run `_console_login`, which still expects the French layout, so
it breaks. The safest way to solve this I think is to add a new
variable that lets us skip `uefi_postinstall` - I don't want to
change the ordering so we load `uefi_postinstall` after we read
`POSTINSTALL` in case that breaks things somewhere else.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-08 11:21:37 -08:00
Adam Williamson
13eb4ac2e6 Disable snapshotting on all language tests
There's no point at which the snapshot might be restored, and
this works around a difficult-to-debug crash on ppc64le (it seems
the crash only happens when resuming after the snapshot).

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-08 11:00:14 -08:00
Adam Williamson
a8dc6c425c Use comma not space for QEMUMACHINE value
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-06 19:48:15 -08:00
Adam Williamson
f2ef1f1a37 Workaround RHBZ #1769600 by using pseries-4.0 machine type
ppc64le tests are failing to boot at all with stock qemu in F31.
Using pseries-4.0 machine type (the default is pseries-4.1)
seems to work around the problem.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-06 19:41:15 -08:00
Adam Williamson
1c75256f9d Add new VNC tests to aarch64 and ppc64le too
Forgot to do this.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-06 18:14:23 -08:00
Adam Williamson
4c05f19a9f Add QA:Testcase_Anaconda_User_Interface_VNC_Vncconnect tests
This adds tests for vncconnect, a variation on VNC install using
reverse VNC (where the client waits for the server to connect to
it). Very similar to the regular-VNC test, but we have to use
tigervnc as Boxes doesn't do reverse VNC.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-06 17:42:29 -08:00
Adam Williamson
b82a1dec6f Add test for QA:Testcase_Anaconda_User_Interface_VNC
This adds a test for QA:Testcase_Anaconda_User_Interface_VNC -
the VNC install test case. It's implemented as a server/client
pair, with the server booting from the Server DVD image with
`inst.vnc` and the client booting from the desktop base disk
image, setting up networking, then running Boxes to connect to
the server and run the install.

There are various little tweaks to test loading and logic to
handle this, mostly pretty clear. We also move the workaround
for 'spurious auth prompt appears on desktop after you switch
away to a VT and back' out of the desktop update test and into
the `desktop_vt` helper function, since now this test can hit
it as well. We enhance _graphical_wait_login to handle the boot
loader if needed (it has never needed to until now).

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-11-05 14:46:10 -08:00
Adam Williamson
80dea7efe2 Tweak PXE boot test to also check kickstart file path
One of the test cases we didn't yet automate is:
https://fedoraproject.org/wiki/QA:Testcase_Kickstart_File_Path_Ks_Cfg
Now we have a PXE test, it's actually a good opportunity to test
that at the same time. I don't usually like combining tests like
this but in this case it sort of makes sense as otherwise we'd
have to have a whole parallel PXE install just to test this one
other detail. So, instead of doing an interactive PXE install as
we did at first, let's tweak the test to include a kickstart in
the initramfs and run the install from that.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-10-07 14:11:48 -07:00
Lukas Ruzicka
d82f4ecd7f Develop a new test for system updates via Cockpit.
* server-cockpit-updates tests that Cockpit can be used to update the system.
* server-cockpit-autoupdate tests that users can use dnf-automatic for system
updates.
* cockpit functions were removed from utils.pm and put into an extra library
for cockpit - cockpit.pm which all cockpit tests are now using.

Review cockpit.pm

Review autoupdate test.

Review the update test.

Fix typo in cockpit.pm

Add sleep.

Add missing command.

Delete an unused needle.
2019-10-01 16:04:01 +02:00
Michel Normand
a631030698 Add Workstation-live-iso all tests for ppc64le
Add HDDSIZEGB for Workstation-live-iso ppc64le
as install_default_upload needs at least 11.07GB

Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2019-09-20 13:11:38 +02:00
Michel Normand
200913547c Add Everything-boot-iso install_default for ppc64le
Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2019-09-20 13:08:13 +02:00
Michel Normand
f247ea7511 Remove install_default when install_default_upload
two tests only differ by saving of qcow2 on test end for last one.

That concerns ppc64le and aarch64 flavors:
    AtomicHost-dvd_ostree-iso
    Server-dvd-iso

Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2019-09-20 13:08:13 +02:00
Michel Normand
c2a59b8f01 Add more missing tests for ppc64le
server-dvd-iso:
    release_identification
    server_freeipa_replication_master
    server_freeipa_replication_replica
    server_freeipa_replication_client

Silverblue-dvd_ostree-iso:
    release_identification

universal:
    install_package_set_kde
    install_updates_img_local

Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2019-09-20 13:08:13 +02:00
Guy Menanteau
3e6568c7df Add base_system_logging test for ppc64le
Signed-off-by: Guy Menanteau <menantea@linux.vnet.ibm.com>
2019-09-20 13:08:13 +02:00
Adam Williamson
2f51f3ef20 Add PXE boot test
This adds a whole wodge of stuff to support_server to make it
act as a PXE server, then adds a new test which boots from PXE
and so should hit the PXE server. We use the NFS install repo as
that can be relied on to work for a support_server install.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-20 18:17:57 -07:00
Adam Williamson
469a1395d0 Revise desktop background checks
Split this out of install_default, because it really is not a
part of that test and we do not want that test to fail because
the desktop background is wrong. Make it its own test module
and test suite instead. Don't do it on Rawhide, because we
really can't assert anything worthwhile about Rawhide at the
moment at least (this means the test runs but is a no-op and
will always pass on Rawhide, unfortunately). Move the needles
to a more appropriate location (this has nothing to do with
anaconda) and use 'background' not 'wallpaper' naming (that's
the name we use elsewhere in the project, e.g. package names).
Also, run the test on updates, and add an F29 needle for this
purpose.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-08-14 14:30:25 -07:00
Lukas Ruzicka
d06b30e8dc Add new testcases and methods for release identification. 2019-07-30 15:45:02 -07:00
Adam Williamson
1165fdc969 Switch graphics to virtio to work around #1732113
This probably means we'll run into #1403365 again, but that's
better than the alternative.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-07-22 12:38:03 -07:00
Lukas Ruzicka
1475813b75 Update the modularity tests.
* Add test to check module defaults.
* Add whitelist download.
* Fix install test to include selected profile to be on the safe side.
* Add test into templates.
2019-07-15 13:19:06 +02:00
Michel Normand
d8e327ab1e Add Silverblue tests for ppc64le
Note that before Silverblue
there was no "AtomicWorkstation-dvd_ostree-iso" for ppc64le

Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2019-07-12 15:44:43 +02:00
Adam Williamson
277e0d7493 Drop all use of the old 'ppc64' (BE) arch/products/machine
Fedora stopped supporting big-endian ppc64 and producing images
for it after F28, which is now EOL. So we can drop all these
bits for it.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-07-10 12:58:49 -07:00
Adam Williamson
1e729f1281 Revert "Set version for AtomicHost tests to 29"
This reverts commit 5bc343fd7e.
Turns out this created a new job group, which I wasn't really
meaning to do. I'll tweak checkwiki to not choke on this.
2019-07-08 10:26:13 -07:00
Adam Williamson
5bc343fd7e Set version for AtomicHost tests to 29
We stopped building Atomic Host images from F30 onwards, so we
shouldn't expect to run tests for any version other than 29 any
more (28 is EOL).

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-07-04 17:26:06 -07:00
Adam Williamson
0a4043aad7 Drop "Atomic" subvariant references from templates
It hasn't existed since early 2018.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-07-04 17:23:33 -07:00
Adam Williamson
78dd42b876 Automate two 'mediakit' tests (repoclosure and file conflicts)
We can run these tests much like support_server - have a test
which boots from the support_server disk image but with the ISO
to be tested attached, then mounts it and runs the scripts.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-07-04 13:44:20 -07:00
Adam Williamson
3a8a5f4de0 install_no_user for Silverblue is now pointless...
...install_default_upload is now the same thing, since you
cannot create a user during install any more.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-06-28 14:09:55 -07:00
Adam Williamson
650c4bb11d Bump a couple of disk sizes (KDE package set, Workstation boot)
Some tests on Rawhide started failing back in April due to
insufficient space. I dug into it for an hour or two but can't
find any big smoking guns where unnecessary packages got added
to groups or some package suddenly got bigger or anything, so I
can't find any real bugs to report here; we're just gonna have
to bump the sizes.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-05-01 12:37:21 -07:00
Adam Williamson
abeeb650b2 Fix a stray ; in REPOSITORY_VARIATION for hd_variation test
This, for some reason, made the test fail on x86_64 but not on
ppc64le. I do not understand computers.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-04-12 15:55:02 -07:00
Guy Menanteau
33fd05f3d0 New install_repository_nfsiso_variation
This is covering the  NFSISO_variation test as per:
https://fedoraproject.org/wiki/QA:Testcase_install_repository_NFSISO_variation

Change support_server to use support_5 disk image in templates
supposed to be previously created by updated createhdds tool
required for new test.

Signed-off-by: Guy Menanteau <menantea@linux.vnet.ibm.com>
2019-04-12 15:49:54 -07:00
Guy Menanteau
29070a54f1 New install_repository_hd_variation
use a new PREINSTALL to execute a rescue script before install.

This is covering the  Hard_drive_variation test as per:
https://fedoraproject.org/wiki/QA:Testcase_install_repository_Hard_drive_variation

Signed-off-by: Guy Menanteau <menantea@linux.vnet.ibm.com>
2019-04-12 15:49:54 -07:00
Michel Normand
09e81d6510 Add tests for QA:Testcase Remote Logging also ppc64le
Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2019-04-01 11:24:50 +02:00
Adam Williamson
b1a0fd7dc3 Add tests for QA:Testcase Remote Logging test case
This commit adds server and client tests to cover the remote
logging Server test case.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-03-28 12:52:54 -07:00
Adam Williamson
94818cd1aa Remove DESKTOP key from apps_startstop test suite
This should not be set here; it is set in the 'product' (i.e.
the flavor), so it's set to 'gnome' for Workstation live flavor,
'kde' for KDE live flavor. Setting here overrides that and makes
it always 'gnome', we don't want that.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-03-19 12:15:03 -07:00
Adam Williamson
f77d240407 Add KDE startstop test to templates
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-03-19 12:15:03 -07:00
Adam Williamson
126109f8ac D'oh, missed a quote mark
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-03-13 11:25:58 -07:00
Adam Williamson
4c281e3d23 Set new PARALLEL_CANCEL_WHOLE_CLUSTER for all parallel parents
This is the magic variable to disable openQA's "cancel whole
cluster on child failure" behaviour that I'm adding in
https://github.com/os-autoinst/openQA/pull/2017 . We want it
set for all parallel parent suites in Fedora.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-03-13 11:20:15 -07:00
Michel Normand
1b546261c5 Add PowerPC support for tests upgrade of FreeIPA
This is using the two tests previously added by
"Test upgrade of FreeIPA server and client deployment"

Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2019-03-12 19:41:20 +00:00
Adam Williamson
8a68b04a3b Run the install_default_update test on UEFI as well
Obviously an installer image may work fine for a BIOS install
but fail for a UEFI install, so we should test it both ways.
This requires the invention of a new (terribly named, but I
couldn't think of anything better...) variable to allow us to
use the 'run test on machine A after test run on machine B'
mechanism without just hardcoding '64bit' as the machine name
in the START_AFTER setting (which would break if we ever wanted
to run update tests on any *other* arch).

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-01-29 15:49:59 +01:00
Adam Williamson
12affb145f Add update tests to build and test a netinst image
This adds a test which builds a netinst image potentially with
the package(s) from the update, and uploads that image. It also
adds a test which runs a default install using that image. This
is intended to check whether the update breaks the creation or
use of install images; particularly this will let us test
anaconda etc. updates. We also update the minimal disk image
name, as we have to make it bigger to accommodate this test,
and making it bigger changes its name - the actual change to
the disk image itself is in createhdds. We also have to redo a
bunch of installer needles for F28 fonts, after I removed them
a month or so back...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-01-18 08:24:44 -08:00
Adam Williamson
510662c1ec Drop all RHBZ #1663040 workarounds, should be fixed now
In Fedora-Rawhide-20190117.n.0 #1663040 should be fixed, so drop
all workarounds for it.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-01-17 08:35:30 -08:00
Adam Williamson
e718f76c19 Boot workstation live with enforcing=0 (RHBZ #1663040)
Today's Workstation live doesn't boot at all without enforcing=0.
We *still* want to see how subsequent tests run, so let's work
around this for now.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-01-14 18:19:43 -08:00
Lukas Ruzicka
39d3427471 Create a test suite for application start stop testing.
Merges #86
Fixes #85
2019-01-04 15:23:03 -08:00
Adam Williamson
2d63bd9702 Bump HDDSIZEGB to 13GiB for language tests
12GiB seems not to be enough for aarch64.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-18 11:05:10 -08:00
Adam Williamson
59ec361700 Temporarily disable modularity_update_norepo subtest
This is known to not work at present as it's not implemented in
DNF yet; failing on it every day isn't proving much. Once it's
implemented (expected by end Jan) we can turn it back on again.
See https://bugzilla.redhat.com/show_bug.cgi?id=1616167

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-12-12 22:13:20 -08:00