OpenQA tests for the Fedora distribution

This repository contains tests and images for testing Fedora with OpenQA. For additional tools, Installation Guide and Docker images, see this repository.

Test development

See official documentation on basic concept, test development (including API specification), needles specification and supported variables for backend. See this example repo on how tests should be structured.

In short, since OpenQA uses only one entrypoint for all tests (main.pm), we have decided to utilize this feature and make tests modular. It means that basic passing through main.pm (without any variables set) results in most basic installation test executed. Developer can then customize it with additional variables (for example by setting PACKAGE_SET=minimal to do installation only with minimal package set).

Make your test modular, so that it utilizes _boot_to_anaconda() and _do_install_and_reboot() tests (that are loaded automatically). Break your test into smaller parts, each dealing with one specific feature (e. g. partitioning, user creation...) and add their loading into main.pm based on reasonable variable setting (so they can be used in other tests also).

Test inheritance

Your test can inherit from basetest, fedorabase, installedtest or anacondatest.

• basetest is basic class provided by os-autoinst - it has empty post_fail_hook() and doesn't set any flags.
• fedorabase doesn't neither set flags nor does anything in post_fail_hook(), but it provides basic functions that will be useful during testing Fedora, like console_login() or boot_to_login_screen(). It should be used when no other, more specific class can be used.
• anacondatest should be used in tests where Anaconda is running. It uploads Anaconda logs (for example anaconda.log or packaging.log) in post_fail_hook(). It also provides convenient methods for Anaconda like select_disks().
• installedtest should be used in tests that are running on installed system (either in postinstall phase or in upgrade tests). It uploads /var/log in post_fail_hook().

Test development checklist

1. Select test from this document or from phabricator page
2. Put each part of your test as a separate file into tests/ directory, reimplementing run() method and test_flags() method, inheriting from one of the classes mentioned above.
3. Set correct variables (so that all test parts you have made are executed) in WebUI -> Test suites.
4. Link your newly created Test suite to medium type in WebUI -> Job groups.
5. Run test (see openqa_fedora_tools repository).
6. Create needles (images) by using interactive mode and needles editor in WebUI.
7. Add new Job template and Test suite into templates file.
8. Add new Test suite and Test case into conf_test_suites.py file in openqa_fedora_tools repository.
9. Mark your test in PhaseSeparation.md as done.
10. Open differential request via phabricator.