testing
/
wiki
generated from sig_core/wiki-template
10
0
Fork 3
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

Numerous edits

This commit is contained in:
Alan Marshall 2024-04-27 16:28:30 +01:00
parent e263f02eaa
commit d4295d8d66
Signed by: alangm
GPG Key ID: 4DF85D1B967F51A6
1 changed files with 16 additions and 14 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

30
docs/guidelines/openqa_manual_install.md
View File

@ -1,7 +1,7 @@
---
title: Manual Install of OpenQA for rockylinux
author: Alan Marshall
revision_date: 2024-04-25
revision_date: 2024-04-27
rc:
prod: Rocky Linux
level: Draft
@ -9,26 +9,25 @@ rc:
#### Intended Audience
Those who wish to use the OpenQA automated testing system configured for Rocky Linux tests. To run your own automated tests in OpenQA, you need a PC or server with hardware-virtualisation and an up-to-date Fedora Linux.
Those who wish to use the OpenQA automated testing system configured for Rocky Linux tests. If so, you will need a PC or server with hardware-virtualisation running an up-to-date Fedora Linux.
### Introduction
This guide explains the use of the OpenQA automated testing system to test various aspects of Rocky Linux releases either at the pre-release stage or thereafter.
OpenQA is an automated test tool that makes it possible to test the whole installation process. It uses virtual machines which it sets up to reproduce the process, check the output (both serial console and GUI screen) in every step and send the necessary keystrokes and commands to proceed to the next step. OpenQA checks whether the system can be installed, whether it works properly, whether applications work and whether the system responds as expected to different installation options and commands.
Rocky Linux OpenQA tests can be found in the [os-autoinst-distri-rocky](https://github.com/rocky-linux/os-autoinst-distri-rocky) repository
OpenQA can run numerous combinations of tests for every revision of the operating system, reporting the errors detected for each combination of hardware configuration, installation options and variant of the operating system.
Upstream documentation is useful for reference but since it is a mixture of advice and instructions relating to openSUSE and Fedora which have substantial differences between them it is not always clear which are significant for Rocky. However, as an rpm based distribution, Rocky Linux use is closely related to the Fedora version.
### WebUI
The web UI is a very useful feature of the OpenQA system since it provides an easily accessed view of the progress and details of OpenQA tests either on the local machine or remotely or both. It is intended to be intuitive and self-explanatory. Look out for the little blue help icons and click them for detailed help on specific sections.
The web UI is a very useful feature of the OpenQA system since it provides an easily accessed view of the progress and details of OpenQA tests either on the local machine or remotely or both. It is intended to be intuitive and self-explanatory.
Some pages use queries to select what should be shown. The query parameters are generated on clickable links, for example starting from the index page or the group overview page clicking on single builds. On the query pages there can be UI elements to control the parameters, for example to look for older builds or show only failed jobs, or other settings. Additionally, the query parameters can be tweaked by hand if you want to provide a link to specific views.
## Step-by-step Install Guide
OpenQA can be installed only on a Fedora (or OpenSUSE) server or workstation.
Tested on Fedora 40 Server
OpenQA can be installed only on a Fedora (or OpenSUSE) server or workstation. The following install procedure was tested on Fedora 40 Server.
```
# install Packages
@ -134,8 +133,8 @@ sudo openqa-cli api -X POST isos ISO=Rocky-9.3-x86_64-minimal.iso ARCH=x86_64 DI
sudo openqa-cli api -X POST isos ISO=Rocky-9.3-x86_64-boot.iso ARCH=x86_64 DISTRI=rocky FLAVOR=boot-iso VERSION=9.3 BUILD="$(date +%Y%m%d.%H%M%S).0"-boot
```
At this point your system is configured for single vm deployment and it can be used as such. Pause here if you wish to get some practice using openqa before progressing further.
Installation of facilities for multi-vm testing which is substantially more complicated is beyond the (present) scope of this document.
At this point your system is configured for single vm deployment and it can be used as such. Pause here if you wish to get some practice using openqa before progressing further (recommended).
Installation of facilities for multi-vm testing, which is substantially more complicated, will be described in this document in a later revision (watch this space).
### Helper Scripts
@ -147,7 +146,7 @@ cancel-build.sh is especially useful when you discover that you have initiated a
### Using Templates
#### Problem
One of the problems that arises when testing an operating system, especially when doing continuous testing, is that there is always a certain combination of jobs, each one with its own settings, that needs to be run for every revision. These combinations can be different for different 'flavors' of the same revision, like running a different set of jobs for each architecture. This combinational problem can go one step further if OpenQA is being used for different kinds of tests, like running some simple pre-integration tests for some snapshots combined with more comprehensive post-integration tests for release candidates.
One of the problems that arises when testing an operating system, especially when doing continuous testing, is that there is always a certain combination of jobs, each one with its own settings, that needs to be run for every revision. These combinations can be different for different FLAVORs of the same revision, like running a different set of jobs for each architecture. This combinational problem can go one step further if OpenQA is being used for different kinds of tests, like running some simple pre-integration tests for some snapshots combined with more comprehensive post-integration tests for release candidates.
This section describes how an instance of OpenQA *could* be configured using the options in the admin area of the webUI to automatically create all the required jobs for each revision of your operating system that needs to be tested. *If* you were starting from scratch (the difficult way), you would probably go through the following order:
@ -157,12 +156,12 @@ This section describes how an instance of OpenQA *could* be configured using the
1. Define job groups in 'Job groups' menu for groups of tests
1. Select individual 'Job groups' and decide what combinations make sense and need to be tested
If you followed the install guide above then the cloned Rocky tests from github.com/rocky-linux/os-autoinst-distri-rocky.git will have pre-configured the admin area of the webUI. You may find it useful when reading the following sections.
If you followed the install guide above then the cloned Rocky tests from [os-autoinst-distri-rocky](https://github.com/rocky-linux/os-autoinst-distri-rocky) will have pre-configured the admin area of the webUI. You may find it useful when reading the following sections.
Machines, mediums, test suites and job templates can all set various configuration variables. The job templates within the job groups define how the test suites, mediums and machines should be combined in various ways to produce individual 'jobs'. All the variables from the test suite, medium, machine and job template are combined and made available to the actual test code run by the 'job', along with variables specified as part of the job creation request. Certain variables also influence OpenQA’s and/or os-autoinst’s own behavior in terms of how it configures the environment for the job.
#### Machines
You need to have at least one machine set up to be able to run any tests. Those machines represent virtual machine types that you want to test. To make tests actually happen, you have to have an 'OpenQA worker' connected that can fulfill those specifications.
You need to have at least one machine set up to be able to run any tests. Those machines represent virtual machine types that you want to test. Realistically to make tests actually happen, you have to have a number of 'OpenQA workers' connected that can fulfill these specifications.
- Name. User defined string - only needed for operator to identify the machine configuration.
- Backend. What backend should be used for this machine. Recommended value is qemu as it is the most tested one, but other options (such as kvm2usb or vbox) are also possible.
@ -173,7 +172,7 @@ You need to have at least one machine set up to be able to run any tests. Those
- USBBOOT when set to 1, the image will be loaded through an emulated USB stick.
#### Medium Types (products)
A medium type (product) in OpenQA is a simple description without any definite meaning. It basically consists of a name and a set of variables that define or characterize this product in os-autoinst.
A medium type (product) in OpenQA is a simple description without any definite meaning. It basically consists of a name and a set of variables that define or characterise this product in os-autoinst.
Some example variables are:
@ -184,7 +183,7 @@ Some example variables are:
- RESCUECD is set to 1 for rescue CD images.
#### Test Suites
A test suite consists of a name and a set of test variables that are used inside this particular test together with an optional description. The test variables can be used to parameterize the actual test code and influence the behaviour according to the settings.
A test suite consists of a name and a set of test variables that are used inside this particular test together with an optional description. The test variables can be used to parameterise the actual test code and influence the behaviour according to the settings.
Some sample variables are:
@ -211,6 +210,9 @@ The scenario definitions within the job groups can be created and configured by
### Needles
Needles are very precise and the slightest deviation from the specified display will be detected. This means that every time there is a new release, very small changes occur in layout of displays resulting in many new or modified needles being required. There is always a significant amount of work needed by the Test Team to produce the automatic tests for a new version.
### Upstream Documentation
[Upstream documentation](https://github.com/os-autoinst/openQA/blob/master/docs/Installing.asciidoc) is useful for reference but since it is a mixture of advice and instructions relating to openSUSE and Fedora which have substantial differences between them it is not always clear which are significant for Rocky. However, as an rpm based distribution, Rocky Linux use is loosely related to the [Fedora](https://fedoraproject.org/wiki/OpenQA) version.
### Glossary
The following terms are used within the context of OpenQA:-