generated from sig_core/wiki-template
Testing: Post-Install test docs (#15)
* Update Release Criteria and Test Cases * Testcase batch 1 * Update the post install release criteria for Rocky Linux 9 * Update the test case overview * Update text-formatting * Testcase batch 2 * Fix words and add some links * Missed one word * Make sure the authors are correct * Add text about the release relevance Co-authored-by: lumarel <lumarel@users.noreply.github.com>
This commit is contained in:
Lukas Magauer
GitHub
parent
08c6cad3b9
commit
154e709632
55
QA/Testcase_Post_Application_Functionality.md
Normal file
55
QA/Testcase_Post_Application_Functionality.md
Normal file
@ -0,0 +1,55 @@
|
|||||||
|
---
|
||||||
|
title: QA:Testcase Application Functionality
|
||||||
|
author: Lukas Magauer
|
||||||
|
revision_date: 2022-05-31
|
||||||
|
rc:
|
||||||
|
prod: Rocky Linux
|
||||||
|
ver:
|
||||||
|
- 8
|
||||||
|
- 9
|
||||||
|
level: Final
|
||||||
|
---
|
||||||
|
|
||||||
|
!!! info "Release relevance"
|
||||||
|
This Testcase applies the following versions of {{ rc.prod }}: {% for version in rc.ver %}{{ version }}{% if not loop.last %}, {% endif %}{% endfor %}
|
||||||
|
|
||||||
|
!!! info "Associated release criterion"
|
||||||
|
This test case is associated with the [Release_Criteria#default-application-functionality-desktop-only](9_release_criteria.md#default-application-functionality-desktop-only) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion.
|
||||||
|
|
||||||
|
!!! error "REFERENCED RELEASE CRITERIA IS OVERLY GENERAL AND UNTESTABLE"
|
||||||
|
The associated release criteria, [Release_Criteria#default-application-functionality-desktop-only](9_release_criteria.md#default-application-functionality-desktop-only), for this test case is overly general and **must** be modified to specific enough to be testable.
|
||||||
|
|
||||||
|
## Description
|
||||||
|
|
||||||
|
This testcase handles all applications, considered as core applications of the desktop environment GNOME or user facing commandline applications.
|
||||||
|
|
||||||
|
The following tasks apply in general to all of the following applications:
|
||||||
|
|
||||||
|
- Firefox
|
||||||
|
- Files (Nautilus)
|
||||||
|
- GNOME Software
|
||||||
|
- (Image Viewer)
|
||||||
|
- (Document Viewer)
|
||||||
|
- Gedit (Text Editor)
|
||||||
|
- Archive Manager
|
||||||
|
- GNOME Terminal (Terminal Emulator)
|
||||||
|
- Problem Reporter
|
||||||
|
- Help Viewer
|
||||||
|
- System Settings
|
||||||
|
- vim (Console Text Editor)
|
||||||
|
|
||||||
|
## Setup
|
||||||
|
|
||||||
|
Obtain access to a suitable system with either a Workstation or a Graphical Server installation.
|
||||||
|
|
||||||
|
## How to test
|
||||||
|
|
||||||
|
1. Check that the application starts without any errors
|
||||||
|
2. Further check that the context menus for the correct function
|
||||||
|
3. Open files to test the functionality of the individual applications
|
||||||
|
|
||||||
|
## Expected Results
|
||||||
|
|
||||||
|
Make sure that the individual applications behave as they should.
|
||||||
|
|
||||||
|
{% include 'testing/qa_testcase_bottom.md' %}
|
||||||
44
QA/Testcase_Post_Artwork_and_Assets.md
Normal file
44
QA/Testcase_Post_Artwork_and_Assets.md
Normal file
@ -0,0 +1,44 @@
|
|||||||
|
---
|
||||||
|
title: QA:Testcase Artwork and Assets
|
||||||
|
author: Lukas Magauer
|
||||||
|
revision_date: 2022-05-31
|
||||||
|
rc:
|
||||||
|
prod: Rocky Linux
|
||||||
|
ver:
|
||||||
|
- 8
|
||||||
|
- 9
|
||||||
|
level: Final
|
||||||
|
---
|
||||||
|
|
||||||
|
!!! info "Release relevance"
|
||||||
|
This Testcase applies the following versions of {{ rc.prod }}: {% for version in rc.ver %}{{ version }}{% if not loop.last %}, {% endif %}{% endfor %}
|
||||||
|
|
||||||
|
!!! info "Associated release criterion"
|
||||||
|
This test case is associated with the [Release_Criteria#artwork-and-assets-server-and-desktop](9_release_criteria.md#artwork-and-assets-server-and-desktop) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion.
|
||||||
|
|
||||||
|
## Description
|
||||||
|
|
||||||
|
There are several brand artworks and assets throughout the whole OS, this testcase will take care of checking, that these are actually in place, and don't produce any UI errors. This is exclusively for installations with the default desktop environment GDM and GNOME.
|
||||||
|
|
||||||
|
## Setup
|
||||||
|
|
||||||
|
1. Acquire access to either a baremetal machine or a VM host, to install a new machine
|
||||||
|
2. Prepare appropriate media for the selected ISO to be tested.
|
||||||
|
- Example: [QA:Testcase Media USB dd](Testcase_Media_USB_dd.md)
|
||||||
|
|
||||||
|
## How to test
|
||||||
|
|
||||||
|
1. While booting the image check, that the correct logo is visible in the loading screen before Anaconda comes up
|
||||||
|
2. Look at the Anaconda images in the [rocky-logos repo](https://github.com/rocky-linux/rocky-logos/tree/r8-fedora/anaconda) and check if all assets are correctly applied in Anaconda (they will generally be visible right away while going through the install process)
|
||||||
|
3. Install the system with either the Workstation install set or Graphical Server
|
||||||
|
4. While the OS does its first boot, check that the correct logo is visible in the loading screen before the boot login screen shows up
|
||||||
|
5. Check the logo and background of the boot login screen
|
||||||
|
6. After the login check the desktop background and further all available options in the settings menu for the desktop background
|
||||||
|
7. Lock the screen and check the background visible in the flyover
|
||||||
|
8. At last check the logo and background of the login screen
|
||||||
|
|
||||||
|
## Expected Results
|
||||||
|
|
||||||
|
The tests during the process could be successfully finished.
|
||||||
|
|
||||||
|
{% include 'testing/qa_testcase_bottom.md' %}
|
||||||
45
QA/Testcase_Post_GNOME_UI_Functionality.md
Normal file
45
QA/Testcase_Post_GNOME_UI_Functionality.md
Normal file
@ -0,0 +1,45 @@
|
|||||||
|
---
|
||||||
|
title: QA:Testcase GNOME UI Functionality
|
||||||
|
author: Lukas Magauer
|
||||||
|
revision_date: 2022-05-31
|
||||||
|
rc:
|
||||||
|
prod: Rocky Linux
|
||||||
|
ver:
|
||||||
|
- 8
|
||||||
|
- 9
|
||||||
|
level: Final
|
||||||
|
---
|
||||||
|
|
||||||
|
!!! info "Release relevance"
|
||||||
|
This Testcase applies the following versions of {{ rc.prod }}: {% for version in rc.ver %}{{ version }}{% if not loop.last %}, {% endif %}{% endfor %}
|
||||||
|
|
||||||
|
!!! error "REFERENCED RELEASE CRITERIA IS OVERLY GENERAL AND UNTESTABLE"
|
||||||
|
The associated release criteria, [Release_Criteria#default-panel-functionality-desktop-only](9_release_criteria.md#default-panel-functionality-desktop-only), for this test case is overly general and **must** be modified to specific enough to be testable.
|
||||||
|
|
||||||
|
!!! info "Associated release criterion"
|
||||||
|
This test case is associated with the [Release_Criteria#default-panel-functionality-desktop-only](9_release_criteria.md#default-panel-functionality-desktop-only) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion.
|
||||||
|
|
||||||
|
## Description
|
||||||
|
|
||||||
|
This test collection takes care of the correct functionality of the GNOME UI.
|
||||||
|
|
||||||
|
## Setup
|
||||||
|
|
||||||
|
Obtain access to a suitable system with either a Workstation or a Graphical Server installation.
|
||||||
|
|
||||||
|
## How to test
|
||||||
|
|
||||||
|
1. Login to the Rocky Machine via the UI
|
||||||
|
2. Navigate through the GNOME UI
|
||||||
|
|
||||||
|
## Expected Results
|
||||||
|
|
||||||
|
1. After the login you should have landed on the desktop with the background and the top bar of GNOME visible
|
||||||
|
2. Clicking the the Activities button in the upper right should bring up the overview
|
||||||
|
3. Further there should be the favourite applications ribbon on the left
|
||||||
|
4. And after clicking the 9-dot-icon all applications should appear
|
||||||
|
5. Back on the desktop check the function of the system and clock panel on the upper right and middle
|
||||||
|
|
||||||
|
It is also a good measure to do some more basic clicking through the GNOME UI, like opening applications, interacting with the application headerbar, moving applications to different desktops or changing settings in the System settings.
|
||||||
|
|
||||||
|
{% include 'testing/qa_testcase_bottom.md' %}
|
||||||
60
QA/Testcase_Post_Keyboard_Layout.md
Normal file
60
QA/Testcase_Post_Keyboard_Layout.md
Normal file
@ -0,0 +1,60 @@
|
|||||||
|
---
|
||||||
|
title: QA:Testcase Keyboard Layout
|
||||||
|
author: Lukas Magauer
|
||||||
|
revision_date: 2022-05-31
|
||||||
|
rc:
|
||||||
|
prod: Rocky Linux
|
||||||
|
ver:
|
||||||
|
- 8
|
||||||
|
- 9
|
||||||
|
level: Final
|
||||||
|
---
|
||||||
|
|
||||||
|
!!! info "Release relevance"
|
||||||
|
This Testcase applies the following versions of {{ rc.prod }}: {% for version in rc.ver %}{{ version }}{% if not loop.last %}, {% endif %}{% endfor %}
|
||||||
|
|
||||||
|
!!! info "Associated release criterion"
|
||||||
|
This test case is associated with the [Release_Criteria#keyboard-layout](9_release_criteria.md#keyboard-layout) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion.
|
||||||
|
|
||||||
|
## Description
|
||||||
|
|
||||||
|
As there are a lot of different keyboard layouts available, it is necessary to test if the keyboard functionality works without any issues throughout the system.
|
||||||
|
|
||||||
|
## Setup
|
||||||
|
|
||||||
|
- Obtain access to a few different system configurations, especially with and without UI, and not to forget with disk encryption.
|
||||||
|
- Acquire access to either a baremetal machine or a VM host, to install a new machine
|
||||||
|
- Prepare appropriate media for the selected ISO to be tested.
|
||||||
|
- Example: [QA:Testcase Media USB dd](Testcase_Media_USB_dd.md)
|
||||||
|
|
||||||
|
## How to test
|
||||||
|
|
||||||
|
### Installer
|
||||||
|
|
||||||
|
1. Bootup the installer
|
||||||
|
2. Choose a language
|
||||||
|
3. Make sure that the keyboard layout got chosen correctly corresponding to the language setting
|
||||||
|
4. Change the keyboard layout if needed to test
|
||||||
|
5. Enter text all over Anaconda to make sure the keyboard layout works correctly with the chosen keyboard layout
|
||||||
|
|
||||||
|
### Disk Encryption
|
||||||
|
|
||||||
|
1. Setup a system with disk encryption
|
||||||
|
2. Check that the password for the disk encryption works on bootup with graphical UI
|
||||||
|
3. Check that the password for the disk encryption works on bootup with text mode
|
||||||
|
|
||||||
|
### Text mode
|
||||||
|
|
||||||
|
Check that the chosen keyboard layout works correctly on text mode.
|
||||||
|
|
||||||
|
### GNOME and Application
|
||||||
|
|
||||||
|
1. Check the login, that the keyboard layout works correctly on the graphical UI login screen
|
||||||
|
2. Also check that the GNOME UI works correctly with the chosen keyboard layout
|
||||||
|
3. And finally check some applications, that the keyboard works as expected
|
||||||
|
|
||||||
|
## Expected Results
|
||||||
|
|
||||||
|
The tests during the process could be successfully finished.
|
||||||
|
|
||||||
|
{% include 'testing/qa_testcase_bottom.md' %}
|
||||||
49
QA/Testcase_Post_Module_Streams.md
Normal file
49
QA/Testcase_Post_Module_Streams.md
Normal file
@ -0,0 +1,49 @@
|
|||||||
|
---
|
||||||
|
title: QA:Testcase Module Streams
|
||||||
|
author: Lukas Magauer
|
||||||
|
revision_date: 2022-05-31
|
||||||
|
rc:
|
||||||
|
prod: Rocky Linux
|
||||||
|
ver:
|
||||||
|
- 8
|
||||||
|
- 9
|
||||||
|
level: Final
|
||||||
|
---
|
||||||
|
|
||||||
|
!!! info "Release relevance"
|
||||||
|
This Testcase applies the following versions of {{ rc.prod }}: {% for version in rc.ver %}{{ version }}{% if not loop.last %}, {% endif %}{% endfor %}
|
||||||
|
|
||||||
|
!!! info "Associated release criterion"
|
||||||
|
This test case is associated with the [Release_Criteria#packages-and-module-installation](9_release_criteria.md#packages-and-module-installation) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion.
|
||||||
|
|
||||||
|
## Description
|
||||||
|
|
||||||
|
This test case takes care of testing the module streams, that they are all installable, all available and working as expected.
|
||||||
|
|
||||||
|
## Setup
|
||||||
|
|
||||||
|
For this tests you will need to install every module stream on its own, so it's the best to setup a new system which gets snapshoted after the initial setup. After that it can be rolled back for every module install.
|
||||||
|
|
||||||
|
It's enough to setup a system with the Minimal Install group.
|
||||||
|
|
||||||
|
## How to test
|
||||||
|
|
||||||
|
1. Login to the machine
|
||||||
|
2. Get a list of all module streams and compare it to the [stream list from RHEL](https://access.redhat.com/support/policy/updates/rhel-app-streams-life-cycle) and to the source in [Git source repo](https://git.rockylinux.org/rocky/rocky-module-defaults)
|
||||||
|
3. The easiest way to test all streams is to install the package groups in the individual streams, i.e. for postgresql:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
dnf module install postgresql
|
||||||
|
dnf module install postgresql:13
|
||||||
|
dnf module install postgresql:13/client
|
||||||
|
```
|
||||||
|
|
||||||
|
Repeat step 3 as often as module streams and package groups are available.
|
||||||
|
|
||||||
|
This could be automated with i.e. Ansible to do all the `install -> rollback -> install -> rollback -> ...` and emiting the output via Ansible.
|
||||||
|
|
||||||
|
## Expected Results
|
||||||
|
|
||||||
|
All module streams should be available and there shouldn't be any errors while installing any of the package groups of the individual streams. (some of the installs will show warnings though because they are incompatible with other streams)
|
||||||
|
|
||||||
|
{% include 'testing/qa_testcase_bottom.md' %}
|
||||||
36
QA/Testcase_Post_Multimonitor_Setup.md
Normal file
36
QA/Testcase_Post_Multimonitor_Setup.md
Normal file
@ -0,0 +1,36 @@
|
|||||||
|
---
|
||||||
|
title: QA:Testcase Multimonitor Setup
|
||||||
|
author: Lukas Magauer
|
||||||
|
revision_date: 2022-05-31
|
||||||
|
rc:
|
||||||
|
prod: Rocky Linux
|
||||||
|
ver:
|
||||||
|
- 8
|
||||||
|
- 9
|
||||||
|
level: Final
|
||||||
|
---
|
||||||
|
|
||||||
|
!!! info "Release relevance"
|
||||||
|
This Testcase applies the following versions of {{ rc.prod }}: {% for version in rc.ver %}{{ version }}{% if not loop.last %}, {% endif %}{% endfor %}
|
||||||
|
|
||||||
|
!!! info "Associated release criterion"
|
||||||
|
This test case is associated with the [Release_Criteria#dual-monitor-setup-desktop-only](9_release_criteria.md#dual-monitor-setup-desktop-only) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion.
|
||||||
|
|
||||||
|
## Description
|
||||||
|
|
||||||
|
This test covers the check if GNOME behaves as it should in multi-monitor setups.
|
||||||
|
|
||||||
|
## Setup
|
||||||
|
|
||||||
|
You will need either a machine which can be reinstalled with multiple screens, or a virtualization software which is capable of providing multiple screens (like VMware Workstation ([Pro](https://www.vmware.com/products/workstation-pro/workstation-pro-evaluation.html) or [Player](https://www.vmware.com/products/workstation-player/workstation-player-evaluation.html)) or [VMware Fusion](https://www.vmware.com/products/fusion/fusion-evaluation.html), but there is also a [hack](https://communities.vmware.com/t5/VMware-vSphere-Discussions/ESXi-6-7-Multiple-Monitors-for-VMs/td-p/2748906) for [VMware ESXi](https://customerconnect.vmware.com/en/web/vmware/evalcenter?p=vsphere-eval-7))
|
||||||
|
|
||||||
|
## How to test
|
||||||
|
|
||||||
|
1. Run installer with multiple screens connected and install with either the Workstation or Graphical Server group
|
||||||
|
2. Login to the machine after the finished install
|
||||||
|
|
||||||
|
## Expected Results
|
||||||
|
|
||||||
|
There shouldn't be any graphical glitches, or scaling issues through the install and the usage.
|
||||||
|
|
||||||
|
{% include 'testing/qa_testcase_bottom.md' %}
|
||||||
55
QA/Testcase_Post_Package_installs.md
Normal file
55
QA/Testcase_Post_Package_installs.md
Normal file
@ -0,0 +1,55 @@
|
|||||||
|
---
|
||||||
|
title: QA:Testcase Basic Package installs
|
||||||
|
author: Lukas Magauer
|
||||||
|
revision_date: 2022-05-31
|
||||||
|
rc:
|
||||||
|
prod: Rocky Linux
|
||||||
|
ver:
|
||||||
|
- 8
|
||||||
|
- 9
|
||||||
|
level: Final
|
||||||
|
---
|
||||||
|
|
||||||
|
!!! info "Release relevance"
|
||||||
|
This Testcase applies the following versions of {{ rc.prod }}: {% for version in rc.ver %}{{ version }}{% if not loop.last %}, {% endif %}{% endfor %}
|
||||||
|
|
||||||
|
!!! info "Associated release criterion"
|
||||||
|
This test case is associated with the [Release_Criteria#packages-and-module-installation](9_release_criteria.md#packages-and-module-installation) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion.
|
||||||
|
|
||||||
|
!!! error "REFERENCED RELEASE CRITERIA IS OVERLY GENERAL AND UNTESTABLE"
|
||||||
|
The associated release criteria, [Release_Criteria#packages-and-module-installation](9_release_criteria.md#packages-and-module-installation), for this test case is overly general and **must** be modified to specific enough to be testable.
|
||||||
|
|
||||||
|
## Description
|
||||||
|
|
||||||
|
Installing several packages should work without any issues.
|
||||||
|
|
||||||
|
Please also test these usecases (it's basically the fun of learning to install software, it's even good if it's done differently each other time):
|
||||||
|
|
||||||
|
- httpd
|
||||||
|
- httpd with php and ssl
|
||||||
|
- nginx
|
||||||
|
- nginx with php and ssl
|
||||||
|
- mysql-server
|
||||||
|
- mysql-server with secure setup
|
||||||
|
- mariadb-server
|
||||||
|
- postgresql-server
|
||||||
|
- postgresql-server with secure setup
|
||||||
|
- compiling packages with:
|
||||||
|
- cmake
|
||||||
|
- g++
|
||||||
|
- ipa-server
|
||||||
|
- ipa-server with dns
|
||||||
|
|
||||||
|
## Setup
|
||||||
|
|
||||||
|
Obtain access to a suitable system where any of the tested packages can be installed without any issues.
|
||||||
|
|
||||||
|
## How to test
|
||||||
|
|
||||||
|
1. Install a list of packages or usecases
|
||||||
|
|
||||||
|
## Expected Results
|
||||||
|
|
||||||
|
Installs work without any issues.
|
||||||
|
|
||||||
|
{% include 'testing/qa_testcase_bottom.md' %}
|
||||||
38
QA/Testcase_Post_SELinux_Errors_Desktop.md
Normal file
38
QA/Testcase_Post_SELinux_Errors_Desktop.md
Normal file
@ -0,0 +1,38 @@
|
|||||||
|
---
|
||||||
|
title: QA:Testcase SELinux Errors on Desktop clients
|
||||||
|
author: Lukas Magauer
|
||||||
|
revision_date: 2022-05-31
|
||||||
|
rc:
|
||||||
|
prod: Rocky Linux
|
||||||
|
ver:
|
||||||
|
- 8
|
||||||
|
- 9
|
||||||
|
level: Final
|
||||||
|
---
|
||||||
|
|
||||||
|
!!! info "Release relevance"
|
||||||
|
This Testcase applies the following versions of {{ rc.prod }}: {% for version in rc.ver %}{{ version }}{% if not loop.last %}, {% endif %}{% endfor %}
|
||||||
|
|
||||||
|
!!! info "Associated release criterion"
|
||||||
|
This test case is associated with the [Release_Criteria#selinux-and-crash-notifications-desktop-only](9_release_criteria.md#selinux-and-crash-notifications-desktop-only) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion.
|
||||||
|
|
||||||
|
## Description
|
||||||
|
|
||||||
|
Basically running a Workstation or Graphical Server install for a longer amount of time, while using it and then checking if there were any SELinux audit messages.
|
||||||
|
|
||||||
|
## Setup
|
||||||
|
|
||||||
|
Obtain access to a suitable system with either a Workstation or a Graphical Server installation.
|
||||||
|
|
||||||
|
## How to test
|
||||||
|
|
||||||
|
1. Setup new machine or get access to a installed machine
|
||||||
|
2. Click through the system and various applications, to mimic user behavior
|
||||||
|
3. Leave the system running for a few more minutes, if possible hours
|
||||||
|
|
||||||
|
## Expected Results
|
||||||
|
|
||||||
|
1. Open the SETroubleshoot Application and invoke the error summarization.
|
||||||
|
2. There must not be shown any errors / denials
|
||||||
|
|
||||||
|
{% include 'testing/qa_testcase_bottom.md' %}
|
||||||
40
QA/Testcase_Post_SELinux_Errors_Server.md
Normal file
40
QA/Testcase_Post_SELinux_Errors_Server.md
Normal file
@ -0,0 +1,40 @@
|
|||||||
|
---
|
||||||
|
title: QA:Testcase SELinux Errors on Server installations
|
||||||
|
author: Lukas Magauer
|
||||||
|
revision_date: 2022-05-31
|
||||||
|
rc:
|
||||||
|
prod: Rocky Linux
|
||||||
|
ver:
|
||||||
|
- 8
|
||||||
|
- 9
|
||||||
|
level: Final
|
||||||
|
---
|
||||||
|
|
||||||
|
!!! info "Release relevance"
|
||||||
|
This Testcase applies the following versions of {{ rc.prod }}: {% for version in rc.ver %}{{ version }}{% if not loop.last %}, {% endif %}{% endfor %}
|
||||||
|
|
||||||
|
!!! info "Associated release criterion"
|
||||||
|
This test case is associated with the [Release_Criteria#selinux-errors-server](9_release_criteria.md#selinux-errors-server) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion.
|
||||||
|
|
||||||
|
## Description
|
||||||
|
|
||||||
|
Basically running a text mode installation for a longer amount of time, while using it and then checking if there were any SELinux audit messages.
|
||||||
|
|
||||||
|
## Setup
|
||||||
|
|
||||||
|
Obtain access to a suitable system with one of the text mode installation base groups.
|
||||||
|
|
||||||
|
It might also be beneficial to run this test with other than the core installation, but that is a long term test and a bit out of scope of this test.
|
||||||
|
|
||||||
|
## How to test
|
||||||
|
|
||||||
|
1. Setup new machine or get access to a installed machine
|
||||||
|
2. As this test is mostly about the stability of the core system it is mostly only needed to let the system run for a few minutes, if possible hours
|
||||||
|
|
||||||
|
## Expected Results
|
||||||
|
|
||||||
|
1. Install `sealert` with `dnf install setroubleshoot-server`
|
||||||
|
2. Run `sealert -a /var/log/audit/audit.log`
|
||||||
|
3. There must not be shown any errors / denials
|
||||||
|
|
||||||
|
{% include 'testing/qa_testcase_bottom.md' %}
|
||||||
41
QA/Testcase_Post_System_Services.md
Normal file
41
QA/Testcase_Post_System_Services.md
Normal file
@ -0,0 +1,41 @@
|
|||||||
|
---
|
||||||
|
title: QA:Testcase System Services
|
||||||
|
author: Lukas Magauer
|
||||||
|
revision_date: 2022-05-31
|
||||||
|
rc:
|
||||||
|
prod: Rocky Linux
|
||||||
|
ver:
|
||||||
|
- 8
|
||||||
|
- 9
|
||||||
|
level: Final
|
||||||
|
---
|
||||||
|
|
||||||
|
!!! info "Release relevance"
|
||||||
|
This Testcase applies the following versions of {{ rc.prod }}: {% for version in rc.ver %}{{ version }}{% if not loop.last %}, {% endif %}{% endfor %}
|
||||||
|
|
||||||
|
!!! info "Associated release criterion"
|
||||||
|
This test case is associated with the [Release_Criteria#system-services](9_release_criteria.md#system-services) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion.
|
||||||
|
|
||||||
|
## Description
|
||||||
|
|
||||||
|
This test covers the check, that all basic system service which are getting installed with the base groups are starting / running normally.
|
||||||
|
|
||||||
|
## Setup
|
||||||
|
|
||||||
|
1. Acquire access to either a baremetal machine or a VM host, to install a new machine
|
||||||
|
2. Prepare appropriate media for the selected ISO to be tested.
|
||||||
|
- Example: [QA:Testcase Media USB dd](Testcase_Media_USB_dd.md)
|
||||||
|
|
||||||
|
## How to test
|
||||||
|
|
||||||
|
Startup the system and check that all services are running without any failure:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
systemctl status
|
||||||
|
```
|
||||||
|
|
||||||
|
## Expected Results
|
||||||
|
|
||||||
|
The tests during the process could be successfully finished.
|
||||||
|
|
||||||
|
{% include 'testing/qa_testcase_bottom.md' %}
|
||||||
@ -48,17 +48,17 @@ This page lists all test cases in work and who is working on them...
|
|||||||
|
|
||||||
## Post-Installation Requirements
|
## Post-Installation Requirements
|
||||||
|
|
||||||
| Requirement | Test Case | Assignee | Status |
|
| Requirement | Test Case | Assignee | Status |
|
||||||
| --------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------- | ----------------------- | --------------------------------------- |
|
|--------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------|----------|----------------------------------------------------------------------|
|
||||||
| System Services | [QA:Testcase System Services](Testcase_Template.md) | @lumarel | testcase filename reserved |
|
| System Services | [QA:Testcase System Services](Testcase_Post_System_Services.md) | @lumarel | manual guide documented or needs new openQA testcase |
|
||||||
| Keyboard Layout | [QA:Testcase Keyboard Layout](Testcase_Template.md) | @lumarel | testcase filename reserved |
|
| Keyboard Layout | [QA:Testcase Keyboard Layout](Testcase_Post_Keyboard_Layout.md) | @lumarel | implemented in openQA |
|
||||||
| SELinux Errors (Server) | [QA:Testcase SELinux Errors on Server installations](Testcase_Template.md) | @lumarel, @raktajino | testcase filename reserved |
|
| SELinux Errors (Server) | [QA:Testcase SELinux Errors on Server](Testcase_Post_SELinux_Errors_Server.md) | @lumarel | implemented in openQA |
|
||||||
| SELinux and Crash Notifications (Desktop Only) | [QA:Testcase SELinux Errors on Desktop clients](Testcase_Template.md) | @lumarel, @raktajino | testcase filename reserved |
|
| SELinux and Crash Notifications (Desktop Only) | [QA:Testcase SELinux Errors on Desktop](Testcase_Post_SELinux_Errors_Desktop.md) | @lumarel | partly implemented in openQA |
|
||||||
| Default Application Functionality (Desktop Only) | [QA:Testcase Application Functionality](Testcase_Template.md) | @lumarel | testcase filename reserved |
|
| Default Application Functionality (Desktop Only) | [QA:Testcase Application Functionality](Testcase_Post_Application_Functionality.md) | @lumarel | manual guide documented |
|
||||||
| Default Panel Functionality (Desktop Only) | [QA:Testcase GNOME UI Functionality](Testcase_Template.md) | @lumarel | testcase filename reserved |
|
| Default Panel Functionality (Desktop Only) | [QA:Testcase GNOME UI Functionality](Testcase_Post_GNOME_UI_Functionality.md) | @lumarel | implemented in openQA, additionally documented for manual inspection |
|
||||||
| Dual Monitor Setup (Desktop Only) | [QA:Testcase Multimonitor Setup](Testcase_Template.md) | @lumarel | testcase filename reserved |
|
| Dual Monitor Setup (Desktop Only) | [QA:Testcase Multimonitor Setup](Testcase_Post_Multimonitor_Setup.md) | @lumarel | manual guide documented |
|
||||||
| Artwork and Assets (Server and Desktop) | [QA:Testcase Artwork and Assets](Testcase_Template.md) | @lumarel | testcase filename reserved |
|
| Artwork and Assets (Server and Desktop) | [QA:Testcase Artwork and Assets](Testcase_Post_Artwork_and_Assets.md) | @lumarel | implemented in openQA, additionally documented for manual inspection |
|
||||||
| Packages and Module Installation | [QA:Testcase Basic Package installs](Testcase_Template.md)<br>[QA:Testcase Module Streams](Testcase_Template.md) | @lumarel | |
|
| Packages and Module Installation | [QA:Testcase Basic Package installs](Testcase_Post_Package_installs.md)<br>[QA:Testcase Module Streams](Testcase_Post_Module_Streams.md) | @lumarel | partly implemented in openQA, manual guide documented |
|
||||||
|
|
||||||
|
|
||||||
{% include 'content_bottom.md' %}
|
{% include 'content_bottom.md' %}
|
||||||
|
|||||||
@ -1,7 +1,9 @@
|
|||||||
---
|
---
|
||||||
title: Rocky Linux 8 Release Criteria
|
title: Rocky Linux 8 Release Criteria
|
||||||
author: Trevor Cooper
|
author:
|
||||||
revision_date: 2022-04-01
|
- Trevor Cooper
|
||||||
|
- Lukas Magauer
|
||||||
|
revision_date: 2022-05-07
|
||||||
rc:
|
rc:
|
||||||
prod: Rocky Linux
|
prod: Rocky Linux
|
||||||
ver: 8
|
ver: 8
|
||||||
@ -164,82 +166,102 @@ Release-blocking cloud disk images must be published to appropriate cloud provid
|
|||||||
- [QA:Testcase TBD](Testcase_Template.md)
|
- [QA:Testcase TBD](Testcase_Template.md)
|
||||||
|
|
||||||
### Post-Installation Requirements
|
### Post-Installation Requirements
|
||||||
|
|
||||||
#### System Services
|
#### System Services
|
||||||
|
|
||||||
All system services present after installation must start properly, with the exception of services that require hardware which is not present. Examples of such services would be:
|
All system services present after installation must start properly, with the exception of services that require hardware which is not present. Examples of such services would be:
|
||||||
- sshd
|
|
||||||
- firewalld
|
- sshd
|
||||||
- auditd
|
- firewalld
|
||||||
- chronyd
|
- auditd
|
||||||
|
- chronyd
|
||||||
|
|
||||||
??? tldr "References"
|
??? tldr "References"
|
||||||
- Test cases:
|
- Test cases:
|
||||||
- [QA:Testcase TBD](Testcase_Template.md)
|
- [QA:Testcase System Services](Testcase_Post_System_Services.md)
|
||||||
|
|
||||||
#### Keyboard Layout
|
#### Keyboard Layout
|
||||||
|
|
||||||
If a particular keyboard layout has been configured for the system, that layout must be used:
|
If a particular keyboard layout has been configured for the system, that layout must be used:
|
||||||
- When unlocking storage volumes (encrypted by LUKS)
|
|
||||||
- When logging in at a TTY console
|
- When unlocking storage volumes (encrypted by LUKS)
|
||||||
- When logging in via GDM
|
- When logging in at a TTY console
|
||||||
- After logging into a GNOME desktop system, if the user does not have their own layout configuration set.
|
- When logging in via GDM
|
||||||
|
- After logging into a GNOME desktop system, if the user does not have their own layout configuration set.
|
||||||
|
|
||||||
??? tldr "References"
|
??? tldr "References"
|
||||||
- Test cases:
|
- Test cases:
|
||||||
- [QA:Testcase TBD](Testcase_Template.md)
|
- [QA:Testcase Keyboard Layout](Testcase_Post_Keyboard_Layout.md)
|
||||||
|
|
||||||
#### SELinux Errors (Server)
|
#### SELinux Errors (Server)
|
||||||
|
|
||||||
There must be no SELinux denial logs in /var/log/audit/audit.log
|
There must be no SELinux denial logs in /var/log/audit/audit.log
|
||||||
|
|
||||||
??? tldr "References"
|
??? tldr "References"
|
||||||
- Test cases:
|
- Test cases:
|
||||||
- [QA:Testcase TBD](Testcase_Template.md)
|
- [QA:Testcase SELinux Errors on Server installations](Testcase_Post_SELinux_Errors_Server.md)
|
||||||
|
|
||||||
#### SELinux and Crash Notifications (Desktop Only)
|
#### SELinux and Crash Notifications (Desktop Only)
|
||||||
|
|
||||||
There must be no SELinux denial notifications or crash notifications on boot, during installation, or during first login.
|
There must be no SELinux denial notifications or crash notifications on boot, during installation, or during first login.
|
||||||
|
|
||||||
??? tldr "References"
|
??? tldr "References"
|
||||||
- Test cases:
|
- Test cases:
|
||||||
- [QA:Testcase TBD](Testcase_Template.md)
|
- [QA:Testcase SELinux Errors on Desktop clients](Testcase_Post_SELinux_Errors_Desktop.md)
|
||||||
|
|
||||||
#### Default Application Functionality (Desktop Only)
|
#### Default Application Functionality (Desktop Only)
|
||||||
|
|
||||||
Applications that can be launched within GNOME or on the command line must start successfully and withstand basic functionality tests. This includes:
|
Applications that can be launched within GNOME or on the command line must start successfully and withstand basic functionality tests. This includes:
|
||||||
- Web browser
|
|
||||||
- File manager
|
- Web browser
|
||||||
- Package manager
|
- File manager
|
||||||
- Image/Document Viewers
|
- Package manager
|
||||||
- Text editors (gedit, vim)
|
- Image/Document Viewers
|
||||||
- Archive manager
|
- Text editors (gedit, vim)
|
||||||
- Terminal Emulator (gnome terminal)
|
- Archive manager
|
||||||
- Problem Reporter
|
- Terminal Emulator (GNOME Terminal)
|
||||||
- Help Viewer
|
- Problem Reporter
|
||||||
- System Settings
|
- Help Viewer
|
||||||
|
- System Settings
|
||||||
|
|
||||||
??? tldr "References"
|
??? tldr "References"
|
||||||
- Test cases:
|
- Test cases:
|
||||||
- [QA:Testcase TBD](Testcase_Template.md)
|
- [QA:Testcase Application Functionality](Testcase_Post_Application_Functionality.md)
|
||||||
|
|
||||||
#### Default Panel Functionality (Desktop Only)
|
#### Default Panel Functionality (Desktop Only)
|
||||||
|
|
||||||
All elements of GNOME should function properly in regular use.
|
All elements of GNOME should function properly in regular use.
|
||||||
|
|
||||||
??? tldr "References"
|
??? tldr "References"
|
||||||
- Test cases:
|
- Test cases:
|
||||||
- [QA:Testcase TBD](Testcase_Template.md)
|
- [QA:Testcase GNOME UI Functionality](Testcase_Post_GNOME_UI_Functionality.md)
|
||||||
|
|
||||||
#### Dual Monitor Setup (Desktop Only)
|
#### Dual Monitor Setup (Desktop Only)
|
||||||
|
|
||||||
Computers using two monitors, the graphical output is correctly shown on both monitors.
|
Computers using two monitors, the graphical output is correctly shown on both monitors.
|
||||||
|
|
||||||
??? tldr "References"
|
??? tldr "References"
|
||||||
- Test cases:
|
- Test cases:
|
||||||
- [QA:Testcase TBD](Testcase_Template.md)
|
- [QA:Testcase Multimonitor Setup](Testcase_Post_Multimonitor_Setup.md)
|
||||||
|
|
||||||
#### Artwork and Assets (Server and Desktop)
|
#### Artwork and Assets (Server and Desktop)
|
||||||
|
|
||||||
Proposed final artwork (such as wallpapers and other assets) must be included. A wallpaper from this package should show up as a default for GDM and GNOME.
|
Proposed final artwork (such as wallpapers and other assets) must be included. A wallpaper from this package should show up as a default for GDM and GNOME.
|
||||||
|
|
||||||
??? tldr "References"
|
??? tldr "References"
|
||||||
- Test cases:
|
- Test cases:
|
||||||
- [QA:Testcase TBD](Testcase_Template.md)
|
- [QA:Testcase Artwork and Assets](Testcase_Post_Artwork_and_Assets.md)
|
||||||
|
|
||||||
#### Packages and Module Installation
|
#### Packages and Module Installation
|
||||||
|
|
||||||
Packages (non-module) should be able to be installed without conflicts or dependent on repositories outside of {{ rc.prod }}.
|
Packages (non-module) should be able to be installed without conflicts or dependent on repositories outside of {{ rc.prod }}.
|
||||||
- Default modules (as listed in dnf module list) should be installed without requiring them to be enabled.
|
|
||||||
- Module streams should be able to be switched and those packages should be able to be installed without errors or unresolved dependencies.
|
- Default modules (as listed in dnf module list) should be installed without requiring them to be enabled.
|
||||||
|
- Module streams should be able to be switched and those packages should be able to be installed without errors or unresolved dependencies.
|
||||||
|
|
||||||
??? tldr "References"
|
??? tldr "References"
|
||||||
- Test cases:
|
- Test cases:
|
||||||
- [QA:Testcase TBD](Testcase_Template.md)
|
- [QA:Testcase Basic Package installs](Testcase_Post_Package_installs.md)
|
||||||
|
- [QA:Testcase Module Streams](Testcase_Post_Module_Streams.md)
|
||||||
|
|
||||||
{% include 'testing/rc_content_bottom.md' %}
|
{% include 'testing/rc_content_bottom.md' %}
|
||||||
|
|||||||
@ -1,6 +1,8 @@
|
|||||||
---
|
---
|
||||||
title: Rocky Linux 9 Release Criteria
|
title: Rocky Linux 9 Release Criteria
|
||||||
author: Trevor Cooper
|
author:
|
||||||
|
- Trevor Cooper
|
||||||
|
- Lukas Magauer
|
||||||
revision_date: 2022-04-01
|
revision_date: 2022-04-01
|
||||||
rc:
|
rc:
|
||||||
prod: Rocky Linux
|
prod: Rocky Linux
|
||||||
@ -164,82 +166,102 @@ Release-blocking cloud disk images must be published to appropriate cloud provid
|
|||||||
- [QA:Testcase TBD](Testcase_Template.md)
|
- [QA:Testcase TBD](Testcase_Template.md)
|
||||||
|
|
||||||
### Post-Installation Requirements
|
### Post-Installation Requirements
|
||||||
|
|
||||||
#### System Services
|
#### System Services
|
||||||
|
|
||||||
All system services present after installation must start properly, with the exception of services that require hardware which is not present. Examples of such services would be:
|
All system services present after installation must start properly, with the exception of services that require hardware which is not present. Examples of such services would be:
|
||||||
- sshd
|
|
||||||
- firewalld
|
- sshd
|
||||||
- auditd
|
- firewalld
|
||||||
- chronyd
|
- auditd
|
||||||
|
- chronyd
|
||||||
|
|
||||||
??? tldr "References"
|
??? tldr "References"
|
||||||
- Test cases:
|
- Test cases:
|
||||||
- [QA:Testcase TBD](Testcase_Template.md)
|
- [QA:Testcase System Services](Testcase_Post_System_Services.md)
|
||||||
|
|
||||||
#### Keyboard Layout
|
#### Keyboard Layout
|
||||||
|
|
||||||
If a particular keyboard layout has been configured for the system, that layout must be used:
|
If a particular keyboard layout has been configured for the system, that layout must be used:
|
||||||
- When unlocking storage volumes (encrypted by LUKS)
|
|
||||||
- When logging in at a TTY console
|
- When unlocking storage volumes (encrypted by LUKS)
|
||||||
- When logging in via GDM
|
- When logging in at a TTY console
|
||||||
- After logging into a GNOME desktop system, if the user does not have their own layout configuration set.
|
- When logging in via GDM
|
||||||
|
- After logging into a GNOME desktop system, if the user does not have their own layout configuration set.
|
||||||
|
|
||||||
??? tldr "References"
|
??? tldr "References"
|
||||||
- Test cases:
|
- Test cases:
|
||||||
- [QA:Testcase TBD](Testcase_Template.md)
|
- [QA:Testcase Keyboard Layout](Testcase_Post_Keyboard_Layout.md)
|
||||||
|
|
||||||
#### SELinux Errors (Server)
|
#### SELinux Errors (Server)
|
||||||
|
|
||||||
There must be no SELinux denial logs in /var/log/audit/audit.log
|
There must be no SELinux denial logs in /var/log/audit/audit.log
|
||||||
|
|
||||||
??? tldr "References"
|
??? tldr "References"
|
||||||
- Test cases:
|
- Test cases:
|
||||||
- [QA:Testcase TBD](Testcase_Template.md)
|
- [QA:Testcase SELinux Errors on Server installations](Testcase_Post_SELinux_Errors_Server.md)
|
||||||
|
|
||||||
#### SELinux and Crash Notifications (Desktop Only)
|
#### SELinux and Crash Notifications (Desktop Only)
|
||||||
|
|
||||||
There must be no SELinux denial notifications or crash notifications on boot, during installation, or during first login.
|
There must be no SELinux denial notifications or crash notifications on boot, during installation, or during first login.
|
||||||
|
|
||||||
??? tldr "References"
|
??? tldr "References"
|
||||||
- Test cases:
|
- Test cases:
|
||||||
- [QA:Testcase TBD](Testcase_Template.md)
|
- [QA:Testcase SELinux Errors on Desktop clients](Testcase_Post_SELinux_Errors_Desktop.md)
|
||||||
|
|
||||||
#### Default Application Functionality (Desktop Only)
|
#### Default Application Functionality (Desktop Only)
|
||||||
|
|
||||||
Applications that can be launched within GNOME or on the command line must start successfully and withstand basic functionality tests. This includes:
|
Applications that can be launched within GNOME or on the command line must start successfully and withstand basic functionality tests. This includes:
|
||||||
- Web browser
|
|
||||||
- File manager
|
- Web browser
|
||||||
- Package manager
|
- File manager
|
||||||
- Image/Document Viewers
|
- Package manager
|
||||||
- Text editors (gedit, vim)
|
- Image/Document Viewers
|
||||||
- Archive manager
|
- Text editors (gedit, vim)
|
||||||
- Terminal Emulator (gnome terminal)
|
- Archive manager
|
||||||
- Problem Reporter
|
- Terminal Emulator (GNOME Terminal)
|
||||||
- Help Viewer
|
- Problem Reporter
|
||||||
- System Settings
|
- Help Viewer
|
||||||
|
- System Settings
|
||||||
|
|
||||||
??? tldr "References"
|
??? tldr "References"
|
||||||
- Test cases:
|
- Test cases:
|
||||||
- [QA:Testcase TBD](Testcase_Template.md)
|
- [QA:Testcase Application Functionality](Testcase_Post_Application_Functionality.md)
|
||||||
|
|
||||||
#### Default Panel Functionality (Desktop Only)
|
#### Default Panel Functionality (Desktop Only)
|
||||||
|
|
||||||
All elements of GNOME should function properly in regular use.
|
All elements of GNOME should function properly in regular use.
|
||||||
|
|
||||||
??? tldr "References"
|
??? tldr "References"
|
||||||
- Test cases:
|
- Test cases:
|
||||||
- [QA:Testcase TBD](Testcase_Template.md)
|
- [QA:Testcase GNOME UI Functionality](Testcase_Post_GNOME_UI_Functionality.md)
|
||||||
|
|
||||||
#### Dual Monitor Setup (Desktop Only)
|
#### Dual Monitor Setup (Desktop Only)
|
||||||
|
|
||||||
Computers using two monitors, the graphical output is correctly shown on both monitors.
|
Computers using two monitors, the graphical output is correctly shown on both monitors.
|
||||||
|
|
||||||
??? tldr "References"
|
??? tldr "References"
|
||||||
- Test cases:
|
- Test cases:
|
||||||
- [QA:Testcase TBD](Testcase_Template.md)
|
- [QA:Testcase Multimonitor Setup](Testcase_Post_Multimonitor_Setup.md)
|
||||||
|
|
||||||
#### Artwork and Assets (Server and Desktop)
|
#### Artwork and Assets (Server and Desktop)
|
||||||
|
|
||||||
Proposed final artwork (such as wallpapers and other assets) must be included. A wallpaper from this package should show up as a default for GDM and GNOME.
|
Proposed final artwork (such as wallpapers and other assets) must be included. A wallpaper from this package should show up as a default for GDM and GNOME.
|
||||||
|
|
||||||
??? tldr "References"
|
??? tldr "References"
|
||||||
- Test cases:
|
- Test cases:
|
||||||
- [QA:Testcase TBD](Testcase_Template.md)
|
- [QA:Testcase Artwork and Assets](Testcase_Post_Artwork_and_Assets.md)
|
||||||
|
|
||||||
#### Packages and Module Installation
|
#### Packages and Module Installation
|
||||||
|
|
||||||
Packages (non-module) should be able to be installed without conflicts or dependent on repositories outside of {{ rc.prod }}.
|
Packages (non-module) should be able to be installed without conflicts or dependent on repositories outside of {{ rc.prod }}.
|
||||||
- Default modules (as listed in dnf module list) should be installed without requiring them to be enabled.
|
|
||||||
- Module streams should be able to be switched and those packages should be able to be installed without errors or unresolved dependencies.
|
- Default modules (as listed in dnf module list) should be installed without requiring them to be enabled.
|
||||||
|
- Module streams should be able to be switched and those packages should be able to be installed without errors or unresolved dependencies.
|
||||||
|
|
||||||
??? tldr "References"
|
??? tldr "References"
|
||||||
- Test cases:
|
- Test cases:
|
||||||
- [QA:Testcase TBD](Testcase_Template.md)
|
- [QA:Testcase Basic Package installs](Testcase_Post_Package_installs.md)
|
||||||
|
- [QA:Testcase Module Streams](Testcase_Post_Module_Streams.md)
|
||||||
|
|
||||||
{% include 'testing/rc_content_bottom.md' %}
|
{% include 'testing/rc_content_bottom.md' %}
|
||||||
|
|||||||
Loading…
Reference in New Issue
Block a user