Commit Graph

78 Commits

Author SHA1 Message Date
akatch
2c427618ac
8.6 release fixes (#92)
* Handle the "reclaim space" dialog

Adds handling for the "Reclaim Space" dialog, which can appear after the
user clicks "Done" in the partitioning spoke.

* Fix indentations

* Set default HDD size to 15GB

* Remove redundant HDDSIZEGB definitions

* Add more GiBs

* Workaround for #82

* Needle with new security policy spoke icon

* Add new needles to fix cockpit tests in 8.6

* Add contribution SOP to README

* Words

* Script fixes (#99)

* Increment version

* Fix this header

* Fix URL for updates.img (#98)

* ---
title: [8.6 Release Issues] Test Suite: install_delete_partial and install_custom_gui_lvm_ext4 on rocky 8.6
labels: 'test suite'
assignees: '@akatch'

---

# Description

Running openQA test suite `install_delete_partial` as above throws `Test died: no candidate needle with tag(s) 'anaconda_install_destination_reclaim_space_btn' matched` at module `disk_guided_delete_partial`.

At this stage, the "Reclaim Space" button in the lower right corner of the dialog is disabled.

Additionally, the dialog shows that 5GB will be reclaimed by the steps taken to that point in the test, but installation requires around 9GB. Do we need to reclaim enough space for installation in order to enable that button?
_Yes, the button is enabled when enough space to install is reclaimed._

Just adding HDDSIZEGB=20 to `templates.fif.json` did not increase volume size shown in the dialog. Do we need to recreate the img file?
_Yes, recreating the img file using createhdds.py against a larger size in hdds.json resolved this error._

The Reclaim Space dialog claimed 9.06GB was required to perform installation, and after doubling the size of disk_full_XXX.img we got past the Reclaim Space dialog. However, `_do_install_and_reboot` failed to install citing not enough disk space.

Fixes #80 when merged.

# How Has This Been Tested?

```
# NOTE: was not able to reproduce for install_custom_gui_lvm_ext4
openqa-cli api -X POST isos ISO=Rocky-8.6-x86_64-dvd1.iso ARCH=x86_64 DISTRI=rocky FLAVOR=dvd-iso VERSION=8.6 BUILD=8.6_dvd-iso_$(date +%Y%m%d.%H%M%S).0 TEST=install_custom_gui_lvm_ext4 PACKAGE_SET=graphical-server

openqa-cli api -X POST isos ISO=Rocky-8.6-x86_64-dvd1.iso ARCH=x86_64 DISTRI=rocky FLAVOR=universal VERSION=8.6 BUILD=8.6_universal_$(date +%Y%m%d.%H%M%S).0 TEST=install_delete_partial PACKAGE_SET=graphical-server
```

All tests must pass `_do_install_and_reboot`.
NOTE: These tests will fail at `_console_wait_login` with the issue in #81.

# Checklist:

- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [ ] Any dependent changes have been merged and published in downstream modules

* Add new needle to mitigate the changed default
for install source on the network from http to https

Co-authored-by: lumarel <lumarel@users.noreply.github.com>
2022-06-09 18:15:17 -05:00
akatch
ff8ce569c7
Test Suite: install_default on rocky-8.5-[minimal|boot|dvd]-iso-x86_64 (#77)
Merging per testing team conversation meeting.

* Correctly check for default packagesets for each flavor

* FIF updates for default packagesets

- Set DESKTOP=gnome to trigger graphical_wait_login instead of
  console_wait_login
- Increase virtual disk size to account for default packagesets

* Correctly check for minimal-iso default packageset

* Updated needles for issue 16

* Update templates.fif.json with package set test suites

Package sets tested per ISO will follow these conventions:
- Test suites will not be duplicated between the boot and dvd ISOs
- All default package sets will be tested against dvd-iso
- Boot ISO will be considered valid if `minimal` package set installs
  cleanly

* Do not add updated needle with changed hostname
2022-03-31 18:23:01 -05:00
33dd37fdea
connect network during boot (#66)
- updated configuration for boot-iso in templates.fif.json. NOTE: If you 
provide GRUB="value" during POST you must include "ip=dhcp" along with 
your custom GRUB settings.
- short circuit _anaconda_network_enable.pm to check for connection at 
start of test.
- provide needle to detect connected network interface using Network 
Spoke icon instead of Installation Source text.

NOTE: While not required with minimal-iso or dvd-iso flavors specifying 
GRUB="ip=dhcp" will also connect the network interface at boot and 
bypass _anaconda_network_enable.
2021-12-02 18:18:43 -06:00
Lukas Magauer
44e8a76c56
Fixes multiple tests in regard of the PACKAGE_SETs graphical-server and workstation (#61)
tcooper confirmed on the call. Merging it.

* Extend product test commands

* Increase disk size for `install_standard_partition_ext4` because some PACKAGE_SETs are too big

* Limit the license acceptance to only run on tests which run for ISO installations
It is unfortunately not that easy to limit this to ISOs due to that the misterous
limit to HDD_1 and not `custom_resize_lvm`

* Make sure that the disk post installs run in a shell
it happens that the previous test ends in the GUI

Co-authored-by: lumarel <lumarel@users.noreply.github.com>
2021-11-11 19:39:34 -06:00
akatch
b5a08825e4
Replace blivet tests with custom GUI tests (#51)
* Our default partitioning scheme includes swap, so omit these tests

* Remove redundant test suites

The suites removed from this commit are functionally identical to the
test suites with the same name but omitting the word "blivet".

* Add needles for custom partitioning GUI

* Replace install_blivet_lvm_ext4 with install_custom_gui_lvm_ext4

- Add subroutine for adding a partition using the "Custom" GUI
- Replace references to blivet with custom_gui in FIF template
- Define test suite for the "Custom" GUI

* syntax fix

* Needle fixes per feedback from lumarel

* Needle reorg

* Replace mouse actions with keypresses

* A few comments

* Replace blivet_standard_partition suites with custom_gui_standard_partition suites

* Rewrite XFS partitioning suite for custom-gui

* Fix filesystem for swap

* Add test suite and needles for RAID using the custom-gui partitioner

* Make raidlevel parameter less confusing

* Fix needles
2021-10-26 08:56:01 -07:00
Russ Butler
b86506d766
Add aarch64 to templates (#41)
* Add aarch64 to templates

* Fix _do_install_and_reboot on aarch64

On aarch64 the _do_install_and_reboot test runs the "consoletty0"
action, which has a command with the string "fedora". This causes the
command to fail with "No such file or directory". This patch changes
"fedora" to "rocky" to which allows _do_install_and_reboot to pass.
2021-10-07 16:26:42 -05:00
be2fc0baed
Needles for _graphical_wait_login allowing English language Gnome environments to Getting Started (#49)
* add needles for _graphical_wait_login
* remove install_package_set_kde
* add new test suites and job group to simplify package_set testing
* modify _software_selection for pre-selected graphical-server case
* restore non-Gnome installs to package-set test suite
2021-09-29 19:18:45 -07:00
c8bcbbc8bf add install_minimal_upload test suite 2021-08-17 18:23:52 -07:00
eef2beb027 remove btrfs test suites 2021-08-14 22:08:56 -07:00
7995d27282 remove refs to platform specific media 2021-08-09 21:55:18 -07:00
ca157bb6b7 vastly decrease scope of configuration 2021-08-09 03:04:29 -07:00
Lukáš Růžička
e0d9409c74 Create a test suite for Evince.
This PR fixes issue #188. It adds a test suite to test basic
functionality of Evince and brings the following features:

* test scripts for various Evince functions.
* needles to support the Evince test scripts
* new template variables `TESTPATH` and `POSTINSTALL_LOAD_ALL` (see
  below)
* new logic in `main.py` (see below)

The new variables and the new logic make it easier to create test
suites for post-installation tests. If TESTPATH is used, OpenQA
will take all tests mentioned in POSTINSTALL from that specified
TESTPATH. If both TESTPATH and POSTINSTALL_LOAD_ALL are used, then
OpenQA will run all tests it can find at the TESTPATH location.
If POSTINSTALL and POSTINSTALL_LOAD_ALL are set simultaneously,
then only POSTINSTALL will be taken into account and OpenQA will
only load tests mentioned there.
2021-07-28 08:58:23 +02:00
Adam Williamson
910fbf2c2f Also run Cloud tests on x86_64 UEFI, on Rawhide only
Cloud images are now BIOS/UEFI hybrid in Rawhide (but not F33
or F34), so we want to run the Cloud tests on UEFI as well, but
only on Rawhide.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-07-22 09:58:23 -07:00
Adam Williamson
088dc9d5ff Correct postinstall test name for install_blivet_resize_lvm
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-07-06 10:56:16 -07:00
Lukáš Růžička
6d1d549941 Introduce a resize check and a postinstall test for custom and blivet
tests.

This PR fixes #228.
2021-06-30 11:01:50 +02:00
Adam Williamson
e5229fd969 Run desktop upgrade tests on aarch64 (#220)
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-06-25 10:27:38 -07:00
Lukáš Růžička
395ed61a1a Automate the Anaconda_Save_Traceback_to_Bugzilla_Text test case.
This PR adds a new test that automates the above mentioned test case.
It starts the installation in text mode using the `install_text` test
case, which it interrupts using the Anaconda crash trigger.
When the crash happens, it goes through the process of reporting
the bug to Bugzilla, checks that Bugzilla sends a positive
confirmation of the action, but also performs some REST API
calls to do a proper check and then it closes the bug to clean up.
2021-06-17 16:28:29 -07:00
Lukáš Růžička
b87049217c Add testcase for Memtest. 2021-06-16 18:11:01 +00:00
Lukáš Růžička
c6d443a20f Automate the QA:Testcase_Anaconda_help.
This PR automates the mentioned testcase to test that Help can be
displayed in Anaconda during the installation. It navigates through
the available Help screens and if it can see it, it finishes.

This test runs after `install_default_upload` to override the
installation defaults defined for all primary tests.

Delete a duplicated needle.

Reformat list extensions to make it nicer.

Get rid of wrong export and an empty line.

Delete empty line.

Use _boot_to_anaconda for booting and move subroutine accordingly.

Add variable to templates.fif.json

Delete trailing whitespace.

Fix calling the pretest.

Move help checking to another place.
2021-05-27 21:39:26 +00:00
Adam Williamson
592f50d3c2 Fix Cloud tests to run on correct image
For Cloud, we want to run these tests directly on the disk image
from the compose. But for other flavors, they are run on a disk
image produced by install_default_upload, so the test suites
specify HDD_1. This causes a problem as the value from the test
suite is used as the filename when downloading the image, but
that file name does not change between composes, so instead of
downloading the image to be tested for each compose, we just
wound up downloading it one time and then re-using that same
file every day.

Solving this is a bit tricky for reasons explained in the
fedora_openqa commit, but this is the best option I could think
of. The scheduler has been changed to schedule the downloaded
image as HDD_2_URL, not HDD_1_URL; so now in the templates we just
override the HDD_1 value for the Cloud flavors to "%HDD_2%",
meaning to take the value of HDD_2 (which will be parsed from
HDD_2_URL). We do not actually attach HDD_2 at all, it's only
used to be copied to HDD_1.

We also explicitly set DEPLOY_UPLOAD_TEST to "" for all flavors
(it was only set for one before), just for clarity.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-05-20 13:15:34 -07:00
Adam Williamson
6c5507b174 Add an 'unwanted packages' test, run on Workstation for now (#218)
This adds a test that just fails if any one of a given list of
unwanted packages is installed. This was a request for the
Workstation edition from @catanzaro so I've just implemented it
for Workstation so far, but it's designed to be easily extended
to cover other subvariants too if we want.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-05-03 08:59:46 -07:00
Lukáš Růžička
477d918b64 Add testcase to test the Package_install_remove testcase. 2021-05-01 00:11:43 +02:00
Adam Williamson
b4f3e6ea5a Move install_blivet_resize_lvm, fix install_resize_lvm (#201)
This is the same thing we did for install_resize_lvm, to address
issue #201. We just didn't get around to doing it for the blivet
test yet. We also change the HDDSIZEGB for the parent test to
15GB so the resizing stuff actually works in both resize tests;
ever since we changed this the install_resize_lvm test has not
been working properly, it hasn't actually been doing any resize.
Also drop the swap partition stuff from that test as it's for
sure no longer needed.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-04-30 11:10:21 -07:00
Lukáš Růžička
d82eae850d Add UEFI versions of cust_btrfs_pres_home and custom_std_part. 2021-03-25 11:28:36 +01:00
Lukáš Růžička
866a8efa72 Add a new testcase to test creation of LVM with ext4 using Blivet.
This PR adds a test that uses the Blivet interface to create an LVM
layout with ext4 filesystem as well as a postinstall test that checks
that the LVM layout has been created correctly.
2021-02-19 13:42:20 -08:00
Lukáš Růžička
a868b1241a Create Blivet standard partitioning with ext4 testcase.
This PR introduces a test case that uses the Blivet partitioning
tool to create a standard partitioning layout with / and /boot
(and specific partitions for UEFI and ARM64) using ext4 as
the selected filesystem.

It also adds a postinstallation test to check that the partitions
have been created correctly.
2021-02-19 13:27:45 -08:00
Adam Williamson
154328e590 Bump KDE package set install disk size to 15GiB
KDE netinst installs more packages than before, now, so it needs
more space. After investigation this is not a bug, it is intended.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-02-08 13:28:08 -08:00
Lukáš Růžička
ecca7d3c01 Create testcase to recreate Btrfs layout using Blivet and preserve home.
This PR uses the Anaconda Blivet partitioning to recreate a partition
layout while preserving the content of the /home subvolume.

It also adds the postinstall test to check that the home has been
preserved.
2021-01-28 09:51:32 +01:00
Lukáš Růžička
ba90c70c8a Add several tests to test QA:Testcase_partitioning_custom_btrfs_preserve_home.
This PR adds the `install_btrfs_upload` to install the btrfs based
image, the `btrfs_preserve_home_extras` to prepare and test the data
on the home partition, as well as the `custom_btrfs_preserve_home` that
uses the preinstalled btrfs image and uses its current partitioning to
preserve the home partition and the data on it.
2021-01-20 16:50:39 +00:00
Adam Williamson
7631dd3c4d Update root-user-crypted-net kickstart usages
Use the copy in the group dir (not jskladan's), which I've fixed
to not use "install" any more.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-01-19 15:03:48 -08:00
Adam Williamson
014492cbc5 Update templates for new disk_ks image version
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-01-19 14:44:10 -08:00
Adam Williamson
940848b628 Use 'test.openqa.fedoraproject.org' not 'domain.local'
Using .local is apparently Bad Form because it's reserved for
mDNS. However there doesn't appear to be any particularly Good
Form for what to call a test domain you never want to exist
outside of a closed system, apparently. Sigh. Let's try this.
Includes a bump to disk_ks version because the kickstarts on
that image also need to have this change applied.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-12-15 14:40:30 -08:00
Adam Williamson
a5c0f7ffbb Enable several desktop tests on Workstation aarch64 disk image
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-12-04 11:55:33 -08:00
Lukáš Růžička
6b0de3aec3 Add an installation test using Standard partitioning with ext4.
This PR add the installation test which uses Standard partitioning
with an ext4 filesystem to cover one of the new requirements as
described in issue #202.

Fixed after a review
2020-11-27 11:48:31 +01:00
Lukáš Růžička
5718c7f1d4 Add an installation test using LVM partitioning as asset for resize tests.
When Fedora went to BTRFS as a default, we lost the LVM based image to
run LVM resize tests with.

This PR introduces the `install_lvm.pm` installation test that creates
an LVM based qcow2 image to be used by follow-up tests.
2020-11-26 09:55:19 +01:00
Adam Williamson
46d89432ae Drop ext3 tests
Per https://pagure.io/fedora-qa/issue/650 , we dropped these
from the wiki, and I agree with @kparal that it doesn't make much
sense to run them any more.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-11-06 11:29:26 -08:00
Adam Williamson
886a22c35f arm image deploy: don't use HDD_1
Setting HDD_1 to %HDD_2% is broken in recent openQA:
https://github.com/os-autoinst/openQA/pull/3309#issuecomment-721906935
I count this as a bug, but we can solve it like this, I think -
we don't actually need to set this test up so carefully to only
have the disk image as HDD_1 and no HDD_2, we can actually just
let the disk image be HDD_2 and have an empty disk as HDD_1 and
the test still works, qemu will boot from the second disk and we
can upload it and everything's fine. So let's just go with that.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-11-04 11:42:13 -08:00
Adam Williamson
b284e79250 Use PREVREL image not CURRREL for resize_lvm tests (#201)
These tests assume the desktop base disk image has an ext4-on-LVM
layout, but from F33 onwards it doesn't, it uses btrfs as that's
the new distro default.

We need a proper fix for this, but for now, just make the test
use the F32 disk image. This buys us six months at least.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-11-02 16:09:03 -08:00
Adam Williamson
a569af3ed0 Enable more base tests on aarch64 images, fix up priorities a bit
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-10-30 15:57:24 -07:00
Adam Williamson
9deb95a876 Unify base tests across normal, Cloud and arm
We have variant versions of several of the base tests which exist
to account for differences in required variable settings to run
essentially the same test in different situations. By twiddling
the variables a little, including inventing a new variable
defined in the flavors and substituted into the test suites so
the same test suite can have a different START_AFTER_TEST when
run on different flavors, I think we can unite them all and make
this a lot simpler.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-10-30 15:27:25 -07:00
Adam Williamson
92d52c6ac6 Split VNC client steps in two for aarch64 timing issue
So, there's a problem with how we figure out the NetworkManager
connection to use in setup_tap_static: it expects the first
connection in the list to be the right one, but this is only
actually true so long as it's *active*. When we're in the tap
case, it's usually not going to actually *work* out of the box
on boot (or else we wouldn't need setup_tap_static at all...),
so some time after boot, NetworkManager gives up on it and marks
it as inactive. And after that, setup_tap_static won't work any
more.

I never noticed this as a problem before because usually we do
setup_tap_static before that point. But it seems in the vnc
client tests, on aarch64, desktop boot and login is slow enough
that by the time we switch to a VT and try to setup the network,
we're very close to that cutoff, and sometimes miss it.

This, I hope, avoids the problem by doing the network setup in
that test before we deal with the desktop login, then doing the
desktop login, then doing the actual VNC bits.

The alternative here would be to figure out a better way to do
setup_tap_static, but I can't.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-10-30 12:39:52 -07:00
Adam Williamson
c7a1b94c84 Enable aarch64 disk image testing, related fixes
This sets us up to test the release-blocking aarch64 disk images
(Minimal, Server and Workstation). It also allows for testing
armhfp disk images on aarch64 worker hosts (though my testing of
that isn't going too well so far), and fixes the initial-setup
handling for a change upstream ('use password' is now the default
so we don't need to choose it). We rewire disk image deployment
test loading to work through the generic loader code rather than
using ENTRYPOINT, as it allows us to more gracefully handle
graphical (Workstation) vs. console (Server, Minimal), moving
the code for handling console initial-setup to a helper function
just like the code for gnome-initial-setup and having _console_
wait_login call it when appropriate. We also tweak desktop_vt a
bit because now we need to switch from a console running as test
to a desktop, which breaks the assumption that the highest
numbered session of user test is the desktop...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-10-30 12:39:49 -07:00
Adam Williamson
478b7eff9e Add initial CoreOS product and test templates
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-09-02 14:49:29 -07:00
Adam Williamson
170ef0733a Use nmcli for static network stuff, not ifcfg files
This should work even if the ifcfg plugin is not present (hi,
CoreOS) or 'predictable' (har) network names are on.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-09-02 14:49:29 -07:00
Adam Williamson
47b4b3af60 Bump disk_ks version to 4
We changed the FreeIPA kickstarts for 172.16 network.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-08-24 12:39:17 -07:00
Adam Williamson
232b224d22 Add 'with swap' tests, drop swap parts from other tests (#180)
In Fedora 33, we generally no longer include a disk-based swap
partition by default (instead swap-on-ZRAM is used, see
https://fedoraproject.org/wiki/Changes/SwapOnZRAM ). This tweaks
our tests to account for that. In tests that aren't to do with
swap at all, we stop including a swap partition in order to be
closer to the default layout. We replace the old _no_swap blivet
and custom tests with _with_swap tests that, as the name implies,
*explicitly include* a swap partition, and adjust the postinstall
test to check the disk swap partition is there.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-08-11 15:09:33 -07:00
Adam Williamson
aa41fe4e4e Automate QA:Testcase_Zezere_Ignition
This is a bit complex to automate, because we cannot really use
the production Zezere server (provision.fedoraproject.org) as
the test case shows, as we'd have to solve authentication and
we also don't really want to constantly keep registering new
hosts to it that are going to disappear and never be seen again.

So, instead we'll do it by setting up our *own* Zezere, and
provisioning our IoT system in that. We run two tests. The
'ignition' test is the actual IoT 'device'; all it really does
is boot up, sit around, and wait to be provisioned. The 'server'
test first sets up a Zezere server, then logs into it, adds an
ssh key, claims the IoT device, provisions it, and connects to
it to create a special file which tells the 'ignition' test
everything worked and it can close out.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-07-23 18:01:06 -07:00
Adam Williamson
72edbfe991 Use qemu host IP 172.16.2.2 not 10.0.2.2
This is to make the infra folks happy, apparently using 10.0.x.x
and 10.1.x.x is causing conflicts since our actual infra network
uses those ranges too.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-07-23 16:40:45 -07:00
Adam Williamson
3a28227155 Bump QEMURAM on install_updates_nfs too
It's hitting the 2GiB crash bug as well.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-07-17 16:25:52 -07:00
Adam Williamson
6a46dfc2a4 Try harder to apply that 3GiB RAM setting
Seems it may need the magic override...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-07-17 12:44:54 -07:00