From dc3cda9b5bbbe86ea0722f8866b7304328145fd6 Mon Sep 17 00:00:00 2001 From: nazunalika Date: Wed, 8 Dec 2021 19:15:52 -0700 Subject: [PATCH 01/45] add testing team --- index.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) create mode 100644 index.md diff --git a/index.md b/index.md new file mode 100644 index 0000000..65cdfd7 --- /dev/null +++ b/index.md @@ -0,0 +1,12 @@ +--- +title: Testing +--- + +This section is primarily for documentation for the Testing Team. The testing team handles testing and QA for Rocky Linux. + +## Members + +| Role | Name | Email | Mattermost Name | IRC Name | +|------------------------------------: |------------------------------- |--------------------- |--- | --- | +| Testing Lead | Trevor Cooper | tcooper@rockylinux.org | @tcooper | | +| Testing Lead | Chris Stackpole | stack@rockylinux.org | @stack | | From b7177ba1b1a7fd90db7c67a91560497d0789f5df Mon Sep 17 00:00:00 2001 From: Trevor Cooper Date: Mon, 25 Apr 2022 14:02:08 -0700 Subject: [PATCH 02/45] Testing Team Additions / Modifications (#5) * update testing team primary page * add mkdocs-macros-plugin to support page includes * add release_criteria page * add collapsible admonitions * add QA/Testcase_Template * add testcase examples * hide the QA directory from navigation while building content * add support for tabbed content * Update index.md * Update release_criteria.md * Update rc_content_example_only.md * Update Testcase_Boot_Methods_Boot_Iso.md * Update Testcase_Boot_Methods_Dvd.md * add mkdocs-autolinks-plugin - autogenerate relative URLs for interpage links * rewrite Fedora example text * add templates and QA Test Case index * add testcase for USB media creation and testing * add testcase for basic-graphics-mode requirement * add testcases for no-broken-packages requirement * add testcases for repositories must match upstream requirement * add command samples and sample output to no broken packages testcases Co-authored-by: Alan Marshall Co-authored-by: Trevor Cooper Signed-off-by: Neil Hanlon --- .pages | 4 + QA/.pages | 2 + QA/Testcase_Basic_Graphics_Mode.md | 44 +++++ QA/Testcase_Boot_Methods_Boot_Iso.md | 28 +++ QA/Testcase_Boot_Methods_Dvd.md | 28 +++ QA/Testcase_Debranding.md | 28 +++ QA/Testcase_Media_File_Conflicts.md | 175 +++++++++++++++++++ QA/Testcase_Media_Repoclosure.md | 86 ++++++++++ QA/Testcase_Media_USB_dd.md | 58 +++++++ QA/Testcase_Packages_No_Insights.md | 96 +++++++++++ QA/Testcase_Packages_No_RHSM.md | 63 +++++++ QA/Testcase_Repo_Compare.md | 28 +++ QA/Testcase_Template.md | 28 +++ QA/index.md | 61 +++++++ index.md | 14 +- release_criteria.md | 247 +++++++++++++++++++++++++++ 16 files changed, 986 insertions(+), 4 deletions(-) create mode 100644 .pages create mode 100644 QA/.pages create mode 100644 QA/Testcase_Basic_Graphics_Mode.md create mode 100644 QA/Testcase_Boot_Methods_Boot_Iso.md create mode 100644 QA/Testcase_Boot_Methods_Dvd.md create mode 100644 QA/Testcase_Debranding.md create mode 100644 QA/Testcase_Media_File_Conflicts.md create mode 100644 QA/Testcase_Media_Repoclosure.md create mode 100644 QA/Testcase_Media_USB_dd.md create mode 100644 QA/Testcase_Packages_No_Insights.md create mode 100644 QA/Testcase_Packages_No_RHSM.md create mode 100644 QA/Testcase_Repo_Compare.md create mode 100644 QA/Testcase_Template.md create mode 100644 QA/index.md create mode 100644 release_criteria.md diff --git a/.pages b/.pages new file mode 100644 index 0000000..d0662e6 --- /dev/null +++ b/.pages @@ -0,0 +1,4 @@ +--- +nav: + - Team Members: index.md + - Release Criteria: release_criteria.md diff --git a/QA/.pages b/QA/.pages new file mode 100644 index 0000000..9dc1966 --- /dev/null +++ b/QA/.pages @@ -0,0 +1,2 @@ +--- +hide: true \ No newline at end of file diff --git a/QA/Testcase_Basic_Graphics_Mode.md b/QA/Testcase_Basic_Graphics_Mode.md new file mode 100644 index 0000000..7207935 --- /dev/null +++ b/QA/Testcase_Basic_Graphics_Mode.md @@ -0,0 +1,44 @@ +--- +title: QA:Testcase Basic Graphics Mode +author: Trevor Cooper +revision_date: 2022-04-24 +rc: + prod: Rocky Linux + ver: 8 + level: Final +--- + +!!! error "REFERENCED RELEASE CRITERIA IS OVERLY GENERAL AND UNTESTABLE" + The associated release criteria, [Release_Criteria#basic-graphics-mode-behaviors](../release_criteria.md#basic-graphics-mode-behaviors), for this test case is overly general and **must** be modified to specific enough to be testable. + +!!! info "Associated release criterion" + This test case is associated with the [Release_Criteria#basic-graphics-mode-behaviors](../release_criteria.md#basic-graphics-mode-behaviors) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + +## Description +This test case will verify that release-blocking installers function as intended using the generic video driver option (“basic graphics mode”) on supported systems and classes of hardware. + +{% include 'testing/qa_testcase_supported_systems.md' %} + +## Setup +1. Obtain access to supported system and hardware class to be installed. +2. Prepare appropriate media for the selected ISO to be tested. + - Example: [QA:Testcase Media USB dd](Testcase_Media_USB_dd.md) + +## How to test +1. Boot the system from the prepared optical, USB media or virtual device attachment. + - Examples: [QA:Testcase Boot Methods Boot ISO](Testcase_Boot_Methods_Boot_Iso.md), [QA:Testcase Boot Methods DVD](Testcase_Boot_Methods_Dvd.md) +2. In the boot menu select the appropriate option to boot the installer. +3. In the installer select the appropriate option to intall in basic graphics mode. +4. Proceed with installation on the test system.
**Depending on installer choices this MAY destroy all the data on the test system.** + +!!! error "DATA LOSS" + If you choose to complete the installation of the test system any/all data on the system may be lost. Please do not install on a system whose contents you need to keep. + +## Expected Results +1. Selection of basic graphics mode in the Anaconda installer is possible. +2. Anaconda installer presents a usable graphical intallation environment. +3. System under test can be installed normally. +4. After reboot system boots into graphical environment. +5. After login user is able to operate the graphical environment. + +{% include 'testing/qa_testcase_bottom.md' %} diff --git a/QA/Testcase_Boot_Methods_Boot_Iso.md b/QA/Testcase_Boot_Methods_Boot_Iso.md new file mode 100644 index 0000000..5b2662a --- /dev/null +++ b/QA/Testcase_Boot_Methods_Boot_Iso.md @@ -0,0 +1,28 @@ +--- +title: QA:Testcase Boot Methods Boot Iso +author: Trevor Cooper +revision_date: 2021-04-21 +rc: + prod: Rocky Linux + ver: 8 + level: Final +--- + +!!! info "Associated release criterion" + This test case is associated with the [Release_Criteria#initialization-requirements](../release_criteria.md#initialization-requirements) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + +## Description +This is to verify that the Anaconda installer starts correctly when booting from the Rocky Linux boot.iso. + +## Setup +1. Prepare your system for booting the boot.iso image. This may involve writing the image to a USB key or burning it to an optical disk. Additionally, attaching the boot.iso to a virtual machine instance as a Virtual Optical Disk or mounting the boot.iso to server via baseboard management controller virtual media attach should be possible but is not expressly required. + +## How to test +1. Boot the system from the prepared optical, USB media or virtual device attachment. +2. In the boot menu select the appropriate option to boot the installer. + +## Expected Results +1. Graphical boot menu is displayed for users to select install options. Navigating the menu and selecting entries must work. If no option is selected, the installer should load after a reasonable timeout. +2. System boots into the Anaconda installer. + +{% include 'testing/qa_testcase_bottom.md' %} diff --git a/QA/Testcase_Boot_Methods_Dvd.md b/QA/Testcase_Boot_Methods_Dvd.md new file mode 100644 index 0000000..422886a --- /dev/null +++ b/QA/Testcase_Boot_Methods_Dvd.md @@ -0,0 +1,28 @@ +--- +title: QA:Testcase Boot Methods DVD +author: Trevor Cooper +revision_date: 2021-04-21 +rc: + prod: Rocky Linux + ver: 8 + level: Final +--- + +!!! info "Associated release criterion" + This test case is associated with the [Release_Criteria#initialization-requirements](../release_criteria.md#initialization-requirements) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + +## Description +This is to verify that the Anaconda installer starts correctly when booting from DVD.iso. + +## Setup +1. Prepare your system for booting the DVD.iso image. This may involve writing the image to a USB key or burning it to an optical disk of sufficient capacity. Additionally, attaching the DVD.iso to a virtual machine instance as a Virtual Optical Disk or mounting the DVD.iso to server via baseboard management controller virtual media attach should be possible but is not expressly required. + +## How to test +1. Boot the system from the prepared optical, USB media or virtual device attachment. +2. In the boot menu select the appropriate option to boot the installer. + +## Expected Results +1. Graphical boot menu is displayed for users to select install options. Navigating the menu and selecting entries must work. If no option is selected, the installer should load after a reasonable timeout. +2. System boots into the Anaconda installer. + +{% include 'testing/qa_testcase_bottom.md' %} diff --git a/QA/Testcase_Debranding.md b/QA/Testcase_Debranding.md new file mode 100644 index 0000000..dcb69e5 --- /dev/null +++ b/QA/Testcase_Debranding.md @@ -0,0 +1,28 @@ +--- +title: QA:Testcase Debranding +author: Trevor Cooper +revision_date: 2022-04-20 +rc: + prod: Rocky Linux + ver: 8 + level: Final +--- + +!!! info "Associated release criterion" + This test case is associated with the [Release_Criteria - Debranding](../release_criteria.md#debranding) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + +## Description +Add a short description here for this Testcase. + +## Setup +1. Add steps for setup for this Testcase. + +## How to test +1. Do this first... +2. Then do this... + +## Expected Results +1. This is what you should see/verify. +2. You should also see/verify this. + +{% include 'testing/qa_testcase_bottom.md' %} diff --git a/QA/Testcase_Media_File_Conflicts.md b/QA/Testcase_Media_File_Conflicts.md new file mode 100644 index 0000000..eded5ca --- /dev/null +++ b/QA/Testcase_Media_File_Conflicts.md @@ -0,0 +1,175 @@ +--- +title: QA:Testcase Media File Conflicts +author: Trevor Cooper +revision_date: 2022-04-24 +rc: + prod: Rocky Linux + ver: 8 + level: Final +--- + +!!! info "Associated release criterion" + This test case is associated with the [Release_Criteria#no-broken-packages](../release_criteria.md#no-broken-packages) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + +## Description +This testcase will verify that the offline repository included on release blocking images will not contain any file conflicts between packages without explicit `Conflicts:` tag in the package metadata. + +## Setup +1. Obtain access to an environment with the `dnf` and `python3` commands. +2. Download the ISO to be tested to that machine. +3. Download the `potential_conflict.py` script provided by Rocky Linux Testing Team. + +## How to test +1. Mount the ISO to be tested locally. + - Example:
`mount -o loop Rocky-8.5-x86_64-minimal.iso /media` +2. Determine the path to the `repodata` directory(ies) on the ISO. + - Example:
`find /media -name repodata` +3. Run the `potential_conflict.py` script on the mounted ISO. + - Example:
`python3 /vagrant/scripts/potential_conflict.py --repofrompath BaseOS,/media/BaseOS --repoid BaseOS --repofrompath Minimal,/media/Minimal --repoid Minimal` +4. Unmount the ISO. + - Example:
`umount /media` + +## Expected Results +1. The `potential_conflict.py` script does not report any packages with non-declared conflicts. + +

Sample Output

+ +=== "Success" + + ``` + $ sudo mount -o loop Rocky-8.5-aarch64-minimal.iso /media + mount: /media: WARNING: device write-protected, mounted read-only. + + $ python3 /vagrant/scripts/potential_conflict.py \ + --repofrompath BaseOS,/media/BaseOS --repoid BaseOS \ + --repofrompath Minimal,/media/Minimal --repoid Minimal + + Added BaseOS repo from /media/BaseOS + Added Minimal repo from /media/Minimal + Getting complete filelist for: + file:///media/BaseOS + file:///media/Minimal + 168374 files found. + + Looking for duplicated filenames: + 524 duplicates found. + + Doing more advanced checks to see if these are real conflicts: + 10% complete ( 52/ 524, 1139/sec), 0 found - eta 0:00:00 + 35% complete ( 182/ 524, 1146/sec), 0 found - eta 0:00:00 + 45% complete ( 234/ 524, 1818/sec), 0 found - eta 0:00:00 + 50% complete ( 260/ 524, 592673/sec), 0 found - eta 0:00:00 + 55% complete ( 286/ 524, 778942/sec), 0 found - eta 0:00:00 + 60% complete ( 312/ 524, 801852/sec), 0 found - eta 0:00:00 + 79% complete ( 416/ 524, 234/sec), 0 found - eta 0:00:00 + 84% complete ( 442/ 524, 902/sec), 0 found - eta 0:00:00 + 89% complete ( 468/ 524, 935/sec), 0 found - eta 0:00:00 + 94% complete ( 494/ 524, 1616/sec), 0 found - eta 0:00:00 + 99% complete ( 520/ 524, 1114/sec), 0 found - eta 0:00:00 + + 0 file conflicts found. + 0 package conflicts found. + + == Package conflicts == + + == File conflicts, listed by conflicting packages == + + $ sudo umount /media + ``` + +=== "Failure" + + ``` + $ sudo mount -o loop Rocky-8.5-x86_64-dvd1.iso /media + mount: /media: WARNING: device write-protected, mounted read-only. + + + $ python3 /vagrant/scripts/potential_conflict.py \ + --repofrompath AppStream,/media/AppStream --repoid AppStream \ + --repofrompath BaseOS,/media/BaseOS --repoid BaseOS + + Added AppStream repo from /media/AppStream + Added BaseOS repo from /media/BaseOS + Getting complete filelist for: + file:///media/AppStream + file:///media/BaseOS + 851967 files found. + + Looking for duplicated filenames: + 101865 duplicates found. + + Doing more advanced checks to see if these are real conflicts: + 5% complete ( 5093/101865, 8713/sec), 0 found - eta 0:00:11 + 10% complete ( 10186/101865, 1787281/sec), 0 found - eta 0:00:05 + 15% complete ( 15279/101865, 2223312/sec), 0 found - eta 0:00:03 + 20% complete ( 20372/101865, 23614/sec), 0 found - eta 0:00:03 + 25% complete ( 25465/101865, 57188/sec), 0 found - eta 0:00:02 + 30% complete ( 30558/101865, 3831/sec), 0 found - eta 0:00:05 + 35% complete ( 35651/101865, 48455/sec), 0 found - eta 0:00:04 + 40% complete ( 40744/101865, 32067/sec), 0 found - eta 0:00:03 + 45% complete ( 45837/101865, 2136586/sec), 0 found - eta 0:00:03 + 50% complete ( 50930/101865, 72529/sec), 0 found - eta 0:00:02 + 55% complete ( 56023/101865, 176294/sec), 0 found - eta 0:00:02 + 60% complete ( 61116/101865, 68622/sec), 1 found - eta 0:00:01 + 65% complete ( 66209/101865, 155133/sec), 1 found - eta 0:00:01 + 70% complete ( 71302/101865, 13874/sec), 1 found - eta 0:00:01 + 75% complete ( 76395/101865, 10835/sec), 1 found - eta 0:00:01 + 80% complete ( 81488/101865, 27477/sec), 1 found - eta 0:00:00 + 85% complete ( 86581/101865, 9075/sec), 1 found - eta 0:00:00 + 90% complete ( 91674/101865, 14807/sec), 1 found - eta 0:00:00 + 95% complete ( 96767/101865, 197437/sec), 1 found - eta 0:00:00 + 100% complete (101860/101865, 38727/sec), 1 found - eta 0:00:00 + + 1 file conflicts found. + 11 package conflicts found. + + == Package conflicts == + mariadb-server-utils-3:10.3.28-1.module+el8.4.0+427+adf35707.x86_64 + mysql-server-8.0.26-1.module+el8.4.0+652+6de068a7.x86_64 + + python3-mod_wsgi-4.6.4-4.el8.x86_64 + python38-mod_wsgi-4.6.8-3.module+el8.4.0+570+c2eaf144.x86_64 + python39-mod_wsgi-4.7.1-4.module+el8.4.0+574+843c4898.x86_64 + + libcmpiCppImpl0-2.0.3-15.el8.i686 + tog-pegasus-libs-2:2.14.1-46.el8.i686 + + mariadb-connector-c-devel-3.1.11-2.el8_3.i686 + mariadb-connector-c-devel-3.1.11-2.el8_3.x86_64 + mariadb-devel-3:10.3.28-1.module+el8.4.0+427+adf35707.x86_64 + mysql-devel-8.0.26-1.module+el8.4.0+652+6de068a7.x86_64 + + mariadb-server-3:10.3.28-1.module+el8.4.0+427+adf35707.x86_64 + mysql-server-8.0.26-1.module+el8.4.0+652+6de068a7.x86_64 + + mariadb-test-3:10.3.28-1.module+el8.4.0+427+adf35707.x86_64 + mysql-test-8.0.26-1.module+el8.4.0+652+6de068a7.x86_64 + + mariadb-connector-c-devel-3.1.11-2.el8_3.i686 + mariadb-connector-c-devel-3.1.11-2.el8_3.x86_64 + mysql-devel-8.0.26-1.module+el8.4.0+652+6de068a7.x86_64 + + mariadb-devel-3:10.3.28-1.module+el8.4.0+427+adf35707.x86_64 + mysql-devel-8.0.26-1.module+el8.4.0+652+6de068a7.x86_64 + + mariadb-3:10.3.28-1.module+el8.4.0+427+adf35707.x86_64 + mysql-8.0.26-1.module+el8.4.0+652+6de068a7.x86_64 + + libcmpiCppImpl0-2.0.3-15.el8.x86_64 + tog-pegasus-libs-2:2.14.1-46.el8.x86_64 + + libev-libevent-devel-4.24-6.el8.i686 + libev-libevent-devel-4.24-6.el8.x86_64 + libevent-devel-2.1.8-5.el8.i686 + libevent-devel-2.1.8-5.el8.x86_64 + + + == File conflicts, listed by conflicting packages == + mariadb-server-3:10.3.28-1.module+el8.4.0+427+adf35707.x86_64 + mysql-test-8.0.26-1.module+el8.4.0+652+6de068a7.x86_64 + /usr/bin/mysqld_safe + + $ sudo umount /media + ``` + +{% include 'testing/qa_testcase_bottom.md' %} diff --git a/QA/Testcase_Media_Repoclosure.md b/QA/Testcase_Media_Repoclosure.md new file mode 100644 index 0000000..379cb97 --- /dev/null +++ b/QA/Testcase_Media_Repoclosure.md @@ -0,0 +1,86 @@ +--- +title: QA:Testcase Media Repoclosure +author: Trevor Cooper +revision_date: 2022-04-24 +rc: + prod: Rocky Linux + ver: 8 + level: Final +--- + +!!! info "Associated release criterion" + This test case is associated with the [Release_Criteria#no-broken-packages](../release_criteria.md#no-broken-packages) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + +## Description +This testcase will verify that the offline repository included on release blocking images will not contain broken dependencies. + +## Setup +1. Obtain access to an environment with the `dnf repoclosure` command. +2. Download the ISO to be tested to that machine. + +## How to test +1. Mount the ISO to be tested locally. + - Example:
`mount -o loop Rocky-8.5-x86_64-minimal.iso /media` +2. Determine the path to the `repodata` directory(ies) on the ISO. + - Example:
`find /media -name repodata` +3. Run the `dnf repoclosure` command on the mounted ISO. + - Example:
`dnf --verbose repoclosure --repofrompath BaseOS,/media/BaseOS --repo BaseOS --repofrompath Minimal,/media/Minimal --repo Minimal` +4. Unmount the ISO. + - Example:
`umount /media` + +## Expected Results +1. The `dnf repoclosure` command does not generate any errors. + +

Sample Output

+ +=== "Success" + + ``` + $ sudo mount -o loop Rocky-8.5-x86_64-minimal.iso /media + mount: /media: WARNING: device write-protected, mounted read-only. + + [vagrant@localhost ~]$ dnf --refresh repoclosure \ + --repofrompath BaseOS,/media/BaseOS --repo BaseOS \ + --repofrompath Minimal,/media/Minimal --repo Minimal + Added BaseOS repo from /media/BaseOS + Added Minimal repo from /media/Minimal + BaseOS 102 MB/s | 2.6 MB 00:00 + Minimal 90 kB/s | 384 B 00:00 + + $ sudo umount /media + ``` + +=== "Failure" + + __NOTE: In this example the content of the `Rocky-8.5-x86_64-minimal.iso` was copied to `/tmp` then the BaseOS repository was modified to remove the `setup-2.12.2-6.el8.noarch.rpm` package and the repository metadata was regenerated.__ + + ``` + [vagrant@localhost ~]$ dnf --refresh repoclosure \ + --repofrompath BaseOS,/tmp/media/BaseOS --repo BaseOS \ + --repofrompath Minimal,/tmp/media/Minimal --repo Minimal + Added BaseOS repo from /tmp/media/BaseOS + Added Minimal repo from /tmp/media/Minimal + BaseOS 3.7 MB/s | 3.8 kB 00:00 + Minimal 3.7 MB/s | 3.8 kB 00:00 + package: basesystem-11-5.el8.noarch from BaseOS + unresolved deps: + setup + package: dump-1:0.4-0.36.b46.el8.x86_64 from BaseOS + unresolved deps: + setup + package: filesystem-3.8-6.el8.x86_64 from BaseOS + unresolved deps: + setup + package: initscripts-10.00.15-1.el8.x86_64 from BaseOS + unresolved deps: + setup + package: rpcbind-1.2.5-8.el8.x86_64 from BaseOS + unresolved deps: + setup + package: shadow-utils-2:4.6-14.el8.x86_64 from BaseOS + unresolved deps: + setup + Error: Repoclosure ended with unresolved dependencies. + ``` + +{% include 'testing/qa_testcase_bottom.md' %} diff --git a/QA/Testcase_Media_USB_dd.md b/QA/Testcase_Media_USB_dd.md new file mode 100644 index 0000000..0428200 --- /dev/null +++ b/QA/Testcase_Media_USB_dd.md @@ -0,0 +1,58 @@ +--- +title: QA:Testcase Media USB dd +author: Trevor Cooper +revision_date: 2021-04-21 +rc: + prod: Rocky Linux + ver: 8 + level: Final +--- + +!!! info "Associated release criterion" + This test case is associated with the [Release_Criteria#initialization-requirements](../release_criteria.md#initialization-requirements) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + +## Description +This verifies that Rocky Linux ISO image can be written to USB media using `dd` command, and the resulting USB media successfully boots to the Anaconda Installer. + +!!! error "DATA LOSS" + Any data on the USB stick used for this test is likely to be destroyed. Please do not use a stick whose contents you need to keep. + +## Setup +1. Provide a USB media device that is larger than the ISO image you wish to test and that it can be completely erased. +2. Provide a Linux (or other *nix system) that has the `dd` command available and an unoccupied USB port. +3. Download the Rocky Linux ISO image you wish to test onto the test system. + - Example command:
`curl -LOR http://dl.rockylinux.org/pub/rocky/8/isos/x86_64/Rocky-8.5-x86_64-boot.iso` +4. Download the `CHECKSUM` file that goes with the Rocky Linux ISO image that you wish to test. + - Example command:
`curl -LOR http://dl.rockylinux.org/pub/rocky/8/isos/x86_64/CHECKSUM` +5. Download the `CHECKSUM.sig` file that does with the `CHECKSUM` file. + - Example command:
`curl -LOR http://dl.rockylinux.org/pub/rocky/8/isos/x86_64/CHECKSUM.sig` +6. Download the Rocky Release Engineering GPG key. + - Example command:
`curl -LOR http://dl.rockylinux.org/pub/rocky/RPM-GPG-KEY-rockyofficial` + +## How to test +1. Import the Rocky Release Engineering GPG key. + - Example command:
`gpg --import RPM-GPG-KEY-rockyofficial` +2. Verify the signature of the CHECKSUM file. + - Example command:
`gpg --verify-file CHECKSUM.sig` +3. Verify the CHECKSUM of the Rocky Linux ISO... + - Example command:
`shasum -a 256 --ignore-missing -c CHECKSUM` +4. Write the Rocky Linux ISO to the USB media using `dd`... + - Example command:
`dd if=Rocky-8.5-x86_64-boot.iso of=/dev/sdX bs=16M status=progress oflag=direct`
...where you replace `sdX` with the device identifier of your USB media.
**This will destroy all data on the disk.** +5. Boot the test system with the USB media. +6. In the boot menu select the appropriate option to boot the installer. +7. **[OPTIONAL]** Proceed with installation on the test system.
**Depending on installer choices this MAY destroy all the data on the test system.** + +## Expected Results +1. The gpg signature on the `CHECKSUM` file is valid. +2. The `CHECKSUM` of the Rocky Linux ISO is valid. +3. The Rocky Linux ISO is written to the USB stick without errors. +4. The USB stick boots without errors. +5. The Anaconda Installer starts without errors. + +!!! error "DATA LOSS" + If you choose to complete the installation of the test system any/all data on the system may be lost. Please do not install on a system whose contents you need to keep. + +**[OPTIONALLY]**
+6. The installation finishes successfully and, if the minimal or DVD ISO were used, the package repository on the USB stick (not a network based repository) was used for the installation. + +{% include 'testing/qa_testcase_bottom.md' %} diff --git a/QA/Testcase_Packages_No_Insights.md b/QA/Testcase_Packages_No_Insights.md new file mode 100644 index 0000000..68858cc --- /dev/null +++ b/QA/Testcase_Packages_No_Insights.md @@ -0,0 +1,96 @@ +--- +title: QA:Testcase Packages No Insights +author: Trevor Cooper +revision_date: 2022-04-24 +rc: + prod: Rocky Linux + ver: 8 + level: Final +--- + +!!! info "Associated release criterion" + This test case is associated with the [Release_Criteria#repositories-must-match-upstream](../release_criteria.md#repositories-must-match-upstream) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + +## Description +This test will verify that `insights-client` package is not declared be installed as part of a package group. + +## Setup +1. Obtain access to an environment with the `dnf` command. +2. Download the ISO to be tested to that machine. + +## How to test +1. Mount the ISO to be tested locally. +2. Determine the path to the `comps` file(s) on the ISO. +3. Verify that `insights-client` is not declared to be installed automatically. + - Example 1:
`find /media -name "*comps*.xml" -exec grep -H "insights-client" '{}' \;` + - Example 2:
`dnf --refresh --repofrompath BaseOS,/media/BaseOS --repo BaseOS --repofrompath AppStream,/media/AppStream --repo AppStream groupinfo base | grep -E ":|insights"` +4. Unmount the ISO. + +## Expected Results +1. `insights-client` is not declared to be installed by default. + +

Sample Output

+ +=== "Success" + + + !!! info "UPDATE SAMPLE" + NOTE: This example needs to be refreshed when the 8.6 ISO has been produced. As seen in the Failure section below the `Rocky-8.5-x86_64-dvd1.iso` includes the `insights-client` as part of the `base` group. The package should be included on the DVD ISO but should **not** be installed automatically. + + ``` + $ sudo mount -o loop Rocky-8.5-aarch64-minimal.iso /media + mount: /media: WARNING: device write-protected, mounted read-only. + + $ dnf --refresh --repofrompath BaseOS,/media/BaseOS --repo BaseOS --repofrompath Minimal,/media/Minimal --repo Minimal search insights-client + Added BaseOS repo from /media/BaseOS + Added Minimal repo from /media/Minimal + BaseOS 3.8 MB/s | 3.9 kB 00:00 + Minimal 3.7 MB/s | 3.8 kB 00:00 + No matches found. + + $ find /media -name "*comps*.xml" -exec grep -H "insights-client" '{}' \; + + $ dnf --refresh --repofrompath BaseOS,/media/BaseOS --repo BaseOS --repofrompath Minimal,/media/Minimal --repo Minimal groupinfo base | grep -E ":|insights" + BaseOS 3.8 MB/s | 3.9 kB 00:00 + Minimal 3.7 MB/s | 3.8 kB 00:00 + Group: Base + Description: The standard installation of Rocky Linux. + Mandatory Packages: + Default Packages: + Optional Packages: + + $ sudo umount /media + ``` + +=== "Failure" + + ``` + $ sudo mount -o loop Rocky-8.5-x86_64-dvd1.iso /media + mount: /media: WARNING: device write-protected, mounted read-only. + + $ dnf --refresh --repofrompath BaseOS,/media/BaseOS --repo BaseOS --repofrompath AppStream,/media/AppStream --repo AppStream search insights-client + Added BaseOS repo from /media/BaseOS + Added AppStream repo from /media/AppStream + BaseOS 3.8 MB/s | 3.9 kB 00:00 + AppStream 4.2 MB/s | 4.3 kB 00:00 + ================================= Name Exactly Matched: insights-client ================================== + insights-client.noarch : Uploads Insights information to Red Hat on a periodic basis + + $ find /media -name "*comps*.xml" -exec grep -H "insights-client" '{}' \; + /media/AppStream/repodata/a6742e1300e1c786af91656b152d3b98bb7aff598e650509381417970e1f1b7e-comps-AppStream.x86_64.xml: insights-client + /media/AppStream/repodata/a6742e1300e1c786af91656b152d3b98bb7aff598e650509381417970e1f1b7e-comps-AppStream.x86_64.xml: insights-client + + $ dnf --refresh --repofrompath BaseOS,/media/BaseOS --repo BaseOS --repofrompath AppStream,/media/AppStream --repo AppStream groupinfo base | grep -E ":|insights" + BaseOS 3.8 MB/s | 3.9 kB 00:00 + AppStream 4.2 MB/s | 4.3 kB 00:00 + Group: Base + Description: The standard installation of Rocky Linux. + Mandatory Packages: + Default Packages: + insights-client + Optional Packages: + + $ sudo umount /media + ``` + +{% include 'testing/qa_testcase_bottom.md' %} diff --git a/QA/Testcase_Packages_No_RHSM.md b/QA/Testcase_Packages_No_RHSM.md new file mode 100644 index 0000000..6fae64e --- /dev/null +++ b/QA/Testcase_Packages_No_RHSM.md @@ -0,0 +1,63 @@ +--- +title: QA:Testcase Packages No RHSM +author: Trevor Cooper +revision_date: 2022-04-24 +rc: + prod: Rocky Linux + ver: 8 + level: Final +--- + +!!! info "Associated release criterion" + This test case is associated with the [Release_Criteria#repositories-must-match-upstream](../release_criteria.md#repositories-must-match-upstream) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + +## Description +This test will verify that packages that are availble from upstream do not have hard requirements on `subscription-manager` (RHSM). + +## Setup +1. Obtain access to an environment with the `dnf` command. +2. Download the ISO to be tested to that machine. + +## How to test +1. Mount the ISO to be tested locally. +2. Obtain a list of packages that have `Requires:` for `subscription-manager` + - Example:
`package_list=($(dnf --refresh repoquery --repofrompath BaseOS,/media/BaseOS --repo BaseOS --repofrompath AppStream,/media/AppStream --repo AppStream --whatrequires subscription-manager 2>/dev/null| grep el8))` +3. Download the packages with explicity `Requires:` for `subscription-manager` + - Example:
`dnf --repofrompath BaseOS,/media/BaseOS --repo BaseOS --repofrompath AppStream,/media/AppStream --repo AppStream download "${package_list[@]}"` +4. Obtain the `SOURCEPKG` definition for the above packages + - Example:
`rpm -q --queryformat="%{NAME}|%{SOURCERPM}\n" subscription-manager*.rpm | column -s\| -t` +4. Unmount the ISO. + +## Expected Results +1. No packages have an explicit requirement for `subscription-manager`. + + +

Sample Output

+ +=== "Success" + + ``` + $ sudo mount -o loop Rocky-8.5-aarch64-minimal.iso /media + mount: /media: WARNING: device write-protected, mounted read-only. + + $ package_list=($(dnf --refresh repoquery --repofrompath BaseOS,/media/BaseOS --repo BaseOS --repofrompath AppStream,/media/AppStream --repo AppStream --whatrequires subscription-manager 2>/dev/null| grep el8)) + + $ dnf --repofrompath BaseOS,/media/BaseOS --repo BaseOS --repofrompath AppStream,/media/AppStream --repo AppStream download "${package_list[@]}" + Added BaseOS repo from /media/BaseOS + Added AppStream repo from /media/AppStream + Last metadata expiration check: 0:00:25 ago on Sun 24 Apr 2022 10:57:13 PM UTC. + + $ rpm -q --queryformat="%{NAME}|%{SOURCERPM}\n" subscription-manager*.rpm | column -s\| -t + subscription-manager-cockpit subscription-manager-1.28.21-3.el8.src.rpm + subscription-manager-migration subscription-manager-1.28.21-3.el8.src.rpm + subscription-manager-plugin-ostree subscription-manager-1.28.21-3.el8.src.rpm + + $ sudo umount /media + ``` + +=== "Failure" + + TBD + + +{% include 'testing/qa_testcase_bottom.md' %} diff --git a/QA/Testcase_Repo_Compare.md b/QA/Testcase_Repo_Compare.md new file mode 100644 index 0000000..e7c76e8 --- /dev/null +++ b/QA/Testcase_Repo_Compare.md @@ -0,0 +1,28 @@ +--- +title: QA:Testcase Media Repo Compare +author: Trevor Cooper +revision_date: 2022-04-24 +rc: + prod: Rocky Linux + ver: 8 + level: Final +--- + +!!! info "Associated release criterion" + This test case is associated with the [Release_Criteria#repositories-must-match-upstream](../release_criteria.md#repositories-must-match-upstream) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + +## Description +This test case will verify that repositories and the packages within them match upstream as closely as possible. + +## Setup +1. Verify access to the Rocky Linux repocompare tooling. + +## How to test +1. Access [Rocky Linux repocompare website](https://repocompare.rockylinux.org/). +2. Verify similarity of Rocky Linux repositories with upstream content. + +## Expected Results +1. Rocky Linux repositories should match, as closely as possible, upstream repositories. +2. The content of Rocky Linux packages should match, as closely as possible, upstream repositories. + +{% include 'testing/qa_testcase_bottom.md' %} diff --git a/QA/Testcase_Template.md b/QA/Testcase_Template.md new file mode 100644 index 0000000..8fa1acf --- /dev/null +++ b/QA/Testcase_Template.md @@ -0,0 +1,28 @@ +--- +title: QA:Testcase Template +author: Trevor Cooper +revision_date: 2022-04-01 +rc: + prod: Rocky Linux + ver: 8 + level: Final +--- + +!!! info "Associated release criterion" + This test case is associated with the [Release_Criteria#TBD](../release_criteria.md#TBD) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + +## Description +Add a short description here for this Testcase. + +## Setup +1. Add steps for setup for this Testcase. + +## How to test +1. Do this first... +2. Then do this... + +## Expected Results +1. This is what you should see/verify. +2. You should also see/verify this. + +{% include 'testing/qa_testcase_bottom.md' %} diff --git a/QA/index.md b/QA/index.md new file mode 100644 index 0000000..d1a62bd --- /dev/null +++ b/QA/index.md @@ -0,0 +1,61 @@ +--- +title: QA: Test Cases +revision_date: 2022-04-20 +--- + +This page lists all test cases in work and who is working on them... + +## Initialization Requirements + +| Requirement | Test Case | Assignee | Status | +| --------------------------------------------------- | ------------------------------------------------------------------------ | ----------------------- | --------------------------------------- | +| [Release-blocking images must boot](release_criteria.md#release-blocking-images-must-boot) | [QA:Testcase Boot Methods Boot ISO](Testcase_Boot_Methods_Boot_Iso.md) | @tcooper | template exists, openQA covered (ref) | +| [Release-blocking images must boot](release_criteria.md#release-blocking-images-must-boot) | [QA:Testcase Boot Methods DVD](Testcase_Boot_Methods_Dvd.md) | @tcooper | template exists, openQA covered (ref) | +| [Basic Graphics Mode behaviors](release_criteria.md#basic-graphics-mode-behaviors) | [QA:Testcase Basic Graphics Mode](Testcase_Basic_Graphics_Mode.md) | @tcooper | manual and/or new openQA TestCase | +| [No Broken Packages](release_criteria.md#no-broken-packages) | [QA:Testcase Media Repoclosure](Testcase_Media_Repoclosure.md)
[QA:Testcase Media File Conflicts](Testcase_Media_File_Conflicts.md) | @tcooper | manual using scripts or automated in CI | +| [Repositories Must Match Upstream](release_criteria.md#repositories-must-match-upstream) | [QA:Testcase repocompare](Testcase_Repo_Compare.md) | @tcooper | manual using Skip's repocompare | +| [Debranding](release_criteria.md#debranding) | [QA:Testcase Debranding Analysis](Testcase_Debranding.md) | @tcooper | process TBD | + + +## Installer Requirements + +| Requirement | Test Case | Assignee | Status | +| --------------------------------------------------- | ------------------------------------------------------------------------ | ----------------------- | --------------------------------------- | +| Media Consistency Verification | [QA:Testcase TBD](Testcase_Template.md) | @tbd | | +| Packages and Installer Sources | [QA:Testcase TBD](Testcase_Template.md) | @tbd | | +| NAS (Network Attached Storage) | [QA:Testcase TBD](Testcase_Template.md) | @tbd | | +| Installation Interfaces | [QA:Testcase TBD](Testcase_Template.md) | @tbd | | +| Minimal Installation | [QA:Testcase TBD](Testcase_Template.md) | @tbd | | +| Kickstart Installation | [QA:Testcase TBD](Testcase_Template.md) | @tbd | | +| Disk Layouts | [QA:Testcase TBD](Testcase_Template.md) | @tbd | | +| Firmware RAID | [QA:Testcase TBD](Testcase_Template.md) | @tbd | | +| Bootloader Disk Selection | [QA:Testcase TBD](Testcase_Template.md) | @tbd | | +| Storage Volume Resize | [QA:Testcase TBD](Testcase_Template.md) | @tbd | | +| Update Image | [QA:Testcase TBD](Testcase_Template.md) | @tbd | | +| Installer Help | [QA:Testcase TBD](Testcase_Template.md) | @tbd | | +| Installer Translations | [QA:Testcase TBD](Testcase_Template.md) | @tbd | | + + +## Cloud Image Requirements + +| Requirement | Test Case | Assignee | Status | +| --------------------------------------------------- | ------------------------------------------------------------------------ | ----------------------- | --------------------------------------- | +| Images Published to Cloud Providers | [QA:Testcase TBD](Testcase_Template.md) | @tbd | | + + +## Post-Installation Requirements + +| Requirement | Test Case | Assignee | Status | +| --------------------------------------------------- | ------------------------------------------------------------------------ | ----------------------- | --------------------------------------- | +| System Services | [QA:Testcase TBD](Testcase_Template.md) | @tbd | | +| Keyboard Layout | [QA:Testcase TBD](Testcase_Template.md) | @tbd | | +| SELinux Errors (Server) | [QA:Testcase TBD](Testcase_Template.md) | @tbd | | +| SELinux and Crash Notifications (Desktop Only) | [QA:Testcase TBD](Testcase_Template.md) | @tbd | | +| Default Application Functionality (Desktop Only) | [QA:Testcase TBD](Testcase_Template.md) | @tbd | | +| Default Panel Functionality (Desktop Only) | [QA:Testcase TBD](Testcase_Template.md) | @tbd | | +| Dual Monitor Setup (Desktop Only) | [QA:Testcase TBD](Testcase_Template.md) | @tbd | | +| Artwork and Assets (Server and Desktop) | [QA:Testcase TBD](Testcase_Template.md) | @tbd | | +| Packages and Module Installation | [QA:Testcase TBD](Testcase_Template.md) | @tbd | | + + +{% include 'content_bottom.md' %} diff --git a/index.md b/index.md index 65cdfd7..4c2f562 100644 --- a/index.md +++ b/index.md @@ -1,12 +1,18 @@ --- title: Testing +revision_date: 2022-04-01 --- This section is primarily for documentation for the Testing Team. The testing team handles testing and QA for Rocky Linux. ## Members -| Role | Name | Email | Mattermost Name | IRC Name | -|------------------------------------: |------------------------------- |--------------------- |--- | --- | -| Testing Lead | Trevor Cooper | tcooper@rockylinux.org | @tcooper | | -| Testing Lead | Chris Stackpole | stack@rockylinux.org | @stack | | + +| Role | Name | Email | Mattermost Name | IRC Name | +| -------------- | --------------- | ----------------------- | ------------------ | -------- | +| Testing Lead | Trevor Cooper | tcooper@rockylinux.org | @tcooper | | +| Testing Lead | Chris Stackpole | stack@rockylinux.org | @stack | | +| Testing Team | Al | | @raktajino | | +| Testing Team | Lukas Magauer | | @lumarel | | + +{% include 'content_bottom.md' %} diff --git a/release_criteria.md b/release_criteria.md new file mode 100644 index 0000000..4a339aa --- /dev/null +++ b/release_criteria.md @@ -0,0 +1,247 @@ +--- +title: Rocky Linux 8 Release Criteria +author: Trevor Cooper +revision_date: 2022-04-01 +rc: + prod: Rocky Linux + ver: 8 + level: Final +--- + +# {{ rc.level }} Release Objectives + +{% include 'testing/rc_content_example_only.md' %} + +The objective of a release (major or minor) is to provide a solid Enterprise Linux release that is suitable to: + +- Meet the needs of end users +- Meet the needs of enterprises big or small + +## {{ rc.level }} Release Requirements + +In order for {{ rc.prod }} to be released to the general public, a compose must be able to meet all the following criteria as provided in this document. This is allows the decision process to be straightforward and as clear as possible. This document only contains “hard requirement” items. Optional/nice to have items are not to be included in this list. + +There may cases where a requirement cannot be met but only in particular configurations. In these types of cases, the Release Engineering Team should use their judgement to determine whether or not the issue should be considered to block the release. They should consider the number of users likely to be affected by said issue, the severity of the case, if the issue can be avoided with ease (by both informed and uninformed users), and if the problem exists upstream in the current Red Hat Enterprise Linux that the release is based on. + +!!! info "Release-blocking Server" + ...means bugs as it pertains to server functionality can be considered to block a release. This applies to any packages that provide a service such as httpd, nginx, etc. All architectures apply. + +!!! info "Release-blocking Desktop" + ...means bugs as it pertains to desktop functionality (GNOME) can be considered to block a release. This applies to both x86_64 and aarch64. Additional desktops (as provided by EPEL or a SIG) are not considered blockers. + +!!! info "Release-blocking Image" + ...means bugs as it pertains to the images built that can block a release. This applies to the DVD, minimal, and boot images on all architectures. + +### Initialization Requirements + +#### Release-blocking images must boot + +Release-blocking installer images must boot when written to optical media or USB flash drive of appropriate sizes (if applicable) via officially supported methods. It is not the testing team’s responsibility to test optical media, but they can and report back. If a bug is found, it is considered a blocker. + +??? tldr "Optical Media Requirements" + Release-blocking images must boot when written to optical media of an appropriate size. Current size requirements are: boot.iso = 789M, minimal.iso = 2.0G and dvd.iso = 10G. + +??? tldr "Officially supported USB flash drive writing methods" + The following methods of writing USB flash drives are officially support: dd
+ The following methods of writing USB flash drives are _**not**_ supported: rufus + +??? tldr "References" + - Test cases: + - [QA:Testcase Boot Methods Boot ISO](Testcase_Boot_Methods_Boot_Iso.md) + - [QA:Testcase Boot Methods DVD](Testcase_Boot_Methods_Dvd.md) + - [QA:Testcase Media USB dd](Testcase_Media_USB_dd.md) + +#### Basic Graphics Mode behaviors +The generic video driver option (“basic graphics mode”) on all release-blocking installers must function as intended. This means launching the installer or desktop and attempting to use a generic driver. There must be no bugs that prevent the installer from being reached in this configuration on all systems and classes of hardware supported by the enterprise linux kernel. +??? tldr "References" + - Test cases: + - [QA:Testcase Basic Graphics Mode](Testcase_Basic_Graphics_Mode.md) + +#### No Broken Packages +Critical errors, such as undeclared conflicts, unresolved dependencies, or modules relying on packages from another stream will be considered an automatic blocker. There are potential exceptions to this (eg, freeradius cannot be installed on an older perl stream, this is a known issue upstream). +??? tldr "References" + - Test cases: + - [QA:Testcase Media Repoclosure](Testcase_Media_Repoclosure.md) + - [QA:Testcase Media File Conflicts](Testcase_Media_File_Conflicts.md) + +#### Repositories Must Match Upstream +Repositories and the packages within them should match upstream as closely as possible. Notable exceptions would be kmods, kpatch, or what is deemed as “spyware” like insights. Packages that are available from upstream should not have hard requirements on RHSM and packages that have it default built in should be patched out. +??? tldr "References" + - Test cases: + - [QA:Testcase Repo Compare](Testcase_Repo_Compare.md) + - [QA:Testcase Packages No Insights](Testcase_Packages_No_Insights.md) + - [QA:Testcase Packages No RHSM](Testcase_Packages_No_RHSM.md) + +#### Debranding +Assets and functionality that are Red Hat specific should not be included. If they are not patched out, it will be considered an automatic blocker. +??? tldr "References" + - Test cases: + - [QA:Testcase Debranding](Testcase_Debranding.md) + +### Installer Requirements + +#### Media Consistency Verification +This means that the installer’s mechanism for verifying the install medium is intact and must complete successfully, with the assumption that the medium was correctly written. It should return a failure message if this not the case. +??? tldr "References" + - Test cases: + - [QA:Testcase TBD](Testcase_Template.md) + +#### Packages and Installer Sources +The installer must be able to use all supported local/remote packages and installer sources. +??? tldr "References" + - Test cases: + - [QA:Testcase TBD](Testcase_Template.md) + +#### NAS (Network Attached Storage)" +The installer must be able to detect and install to supported NAS devices (if possible and supported by the kernel). +??? tldr "References" + - Test cases: + - [QA:Testcase TBD](Testcase_Template.md) + +#### Installation Interfaces +The installer must be able to complete an installation using all supported spokes. +??? tldr "References" + - Test cases: + - [QA:Testcase TBD](Testcase_Template.md) + +#### Minimal Installation +A minimal installation (via network) must be able to install the minimal package set. +??? tldr "References" + - Test cases: + - [QA:Testcase TBD](Testcase_Template.md) + +#### Kickstart Installation +A kickstart installation should succeed, whether from optical/USB media or via the network. +??? tldr "References" + - Test cases: + - [QA:Testcase TBD](Testcase_Template.md) + +#### Disk Layouts +The installer must be able to create and install to any workable partition layout using any file system or format combination offered or supported by the installer. File systems that are not supported by the EL kernel is not tested here (this means btrfs, zfs, both of wish are not supported). +??? tldr "References" + - Test cases: + - [QA:Testcase TBD](Testcase_Template.md) + +#### Firmware RAID +The installer must be able to detect and install to firmware RAID devices. Note that system-specific bugs do not count as blockers. It is likely that some hardware support might be broken or not available at all. DUDs (driver update disks) are not considered for this criteria. +??? tldr "References" + - Test cases: + - [QA:Testcase TBD](Testcase_Template.md) + +#### Bootloader Disk Selection +The installer must allow the user to choose which disk the bootloader will be installed to or, if the user so chooses, not to install a bootloader. +??? tldr "References" + - Test cases: + - [QA:Testcase TBD](Testcase_Template.md) + +#### Storage Volume Resize +Any installer mechanism for resizing storage volumes must correctly attempt the requested operation. This means that if the installer offers a way to resize storage volumes, then it must use the correct resizing tool with the correct parameters. However, it does not require the installer to disallow resizing of unformatted or volumes with an unknown filesystem type. +??? tldr "References" + - Test cases: + - [QA:Testcase TBD](Testcase_Template.md) + +#### Update Image +The installer must be able to use an installer update image retrieved from removable media or a remote package source. This includes DUDs (driver update disks). +??? tldr "References" + - Test cases: + - [QA:Testcase TBD](Testcase_Template.md) + +#### Installer Help +Any element in the installer which contains a “help” text must display the appropriate help documentation when selected. +??? tldr "References" + - Test cases: + - [QA:Testcase TBD](Testcase_Template.md) + +#### Installer Translations +The installer must correctly display all complete translations that are available for use. +??? tldr "References" + - Test cases: + - [QA:Testcase TBD](Testcase_Template.md) + +### Cloud Image Requirements +#### Images Published to Cloud Providers +Release-blocking cloud disk images must be published to appropriate cloud providers (such as Amazon) and they must successfully boot. This also applies to KVM based instances, such as x86 and aarch64 systems. +??? tldr "References" + - Test cases: + - [QA:Testcase TBD](Testcase_Template.md) + +### Post-Installation Requirements +#### System Services +All system services present after installation must start properly, with the exception of services that require hardware which is not present. Examples of such services would be: + - sshd + - firewalld + - auditd + - chronyd +??? tldr "References" + - Test cases: + - [QA:Testcase TBD](Testcase_Template.md) + +#### Keyboard Layout +If a particular keyboard layout has been configured for the system, that layout must be used: + - When unlocking storage volumes (encrypted by LUKS) + - When logging in at a TTY console + - When logging in via GDM + - After logging into a GNOME desktop system, if the user does not have their own layout configuration set. + +??? tldr "References" + - Test cases: + - [QA:Testcase TBD](Testcase_Template.md) + +#### SELinux Errors (Server) +There must be no SELinux denial logs in /var/log/audit/audit.log + +??? tldr "References" + - Test cases: + - [QA:Testcase TBD](Testcase_Template.md) + +#### SELinux and Crash Notifications (Desktop Only) +There must be no SELinux denial notifications or crash notifications on boot, during installation, or during first login. + +??? tldr "References" + - Test cases: + - [QA:Testcase TBD](Testcase_Template.md) + +#### Default Application Functionality (Desktop Only) +Applications that can be launched within GNOME or on the command line must start successfully and withstand basic functionality tests. This includes: + - Web browser + - File manager + - Package manager + - Image/Document Viewers + - Text editors (gedit, vim) + - Archive manager + - Terminal Emulator (gnome terminal) + - Problem Reporter + - Help Viewer + - System Settings + +??? tldr "References" + - Test cases: + - [QA:Testcase TBD](Testcase_Template.md) + +#### Default Panel Functionality (Desktop Only) +All elements of GNOME should function properly in regular use. +??? tldr "References" + - Test cases: + - [QA:Testcase TBD](Testcase_Template.md) + +#### Dual Monitor Setup (Desktop Only) +Computers using two monitors, the graphical output is correctly shown on both monitors. +??? tldr "References" + - Test cases: + - [QA:Testcase TBD](Testcase_Template.md) + +#### Artwork and Assets (Server and Desktop) +Proposed final artwork (such as wallpapers and other assets) must be included. A wallpaper from this package should show up as a default for GDM and GNOME. +??? tldr "References" + - Test cases: + - [QA:Testcase TBD](Testcase_Template.md) + +#### Packages and Module Installation +Packages (non-module) should be able to be installed without conflicts or dependent on repositories outside of {{ rc.prod }}. + - Default modules (as listed in dnf module list) should be installed without requiring them to be enabled. + - Module streams should be able to be switched and those packages should be able to be installed without errors or unresolved dependencies. +??? tldr "References" + - Test cases: + - [QA:Testcase TBD](Testcase_Template.md) + +{% include 'testing/rc_content_bottom.md' %} From 97a619ec0a19361714c79d4d166d9243b1ee5587 Mon Sep 17 00:00:00 2001 From: Trevor Cooper Date: Mon, 25 Apr 2022 16:22:04 -0700 Subject: [PATCH 03/45] remove content example header --- release_criteria.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/release_criteria.md b/release_criteria.md index 4a339aa..40b3be3 100644 --- a/release_criteria.md +++ b/release_criteria.md @@ -10,8 +10,6 @@ rc: # {{ rc.level }} Release Objectives -{% include 'testing/rc_content_example_only.md' %} - The objective of a release (major or minor) is to provide a solid Enterprise Linux release that is suitable to: - Meet the needs of end users From 84eec72867808edf665d10077007974dc986e1eb Mon Sep 17 00:00:00 2001 From: lumarel Date: Fri, 29 Apr 2022 02:14:41 +0200 Subject: [PATCH 04/45] Add documentation for spinning up a documentation devbox --- wiki_development_boxes.md | 39 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+) create mode 100644 wiki_development_boxes.md diff --git a/wiki_development_boxes.md b/wiki_development_boxes.md new file mode 100644 index 0000000..9c8ff59 --- /dev/null +++ b/wiki_development_boxes.md @@ -0,0 +1,39 @@ +--- +title: Create live devbox for wiki.rockylinux.org +author: Lukas Magauer +revision_date: 2022-04-28 +--- + +# How to create a live system to work on the documentation + +There are several ways how to setup your development environment, here are the currently used once by the testing team: + +## Vagrant + +For now here is the link to [Trevor's vagrant box setup](https://github.com/tcooper/rocky-linux-wikibox), this might be merged here in the future! + +## WSL + +- Make sure the Windows Subsystem for Linux is enabled on your machine +- Download the [Cloud image](https://github.com/rocky-linux/sig-cloud-instance-images/actions/workflows/build.yml) +- Get the actual disk out of the image (extract rocky-8.5-docker.tar) +- Run `wsl --import --version 2` to create the machine +- Run `wsl -d ` to initially login to the machine +- Run `dnf -y update` to update the system +- Run `dnf -y install git python39-pip` to install Python 3.9 and pip +- Run `python3.9 -m pip install -U pip` to update pip +- Clone the repo `git clone ` +- And get into the folder of the repo `cd ` +- Sometimes you will need to switch the branch here +- Install all the requirements of the repo `python3.9 -m pip install -r requirements.txt` +- If you just want to look at the output run `mkdocs serve 2>&1 | tee ./mkdocs.serve.log` + +To develop then, the easiest way is to use VS Code with the [Remote - WSL](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-wsl), where you have to open the repo in the container (click on the Remote symbol on the lower left, 'Open folder in WSL...'). + +And finally run `mkdocs serve 2>&1 | tee ./mkdocs.serve.log` in the terminal of this VS Code session. Then you are ready to start changing stuff! + +## Docker + +TODO + +{% include 'testing/rc_content_bottom.md' %} From 8d21f4c475badb2883d5807988430f2dae96594b Mon Sep 17 00:00:00 2001 From: Trevor Cooper Date: Fri, 6 May 2022 20:07:03 -0700 Subject: [PATCH 05/45] update lumarel and add alangm to team --- index.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/index.md b/index.md index 4c2f562..8b262d7 100644 --- a/index.md +++ b/index.md @@ -13,6 +13,7 @@ This section is primarily for documentation for the Testing Team. The testing te | Testing Lead | Trevor Cooper | tcooper@rockylinux.org | @tcooper | | | Testing Lead | Chris Stackpole | stack@rockylinux.org | @stack | | | Testing Team | Al | | @raktajino | | -| Testing Team | Lukas Magauer | | @lumarel | | +| Testing Team | Lukas Magauer | lukas@magauer.eu | @lumarel | | +| Testing Team | Alan Marshall | | @alangm | alangm | {% include 'content_bottom.md' %} From ef5392a22c60a00e4cb278967330318e99d027ad Mon Sep 17 00:00:00 2001 From: Trevor Cooper Date: Fri, 6 May 2022 20:15:18 -0700 Subject: [PATCH 06/45] remove space in page title which causes rendering error --- QA/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/QA/index.md b/QA/index.md index d1a62bd..b1dbbb0 100644 --- a/QA/index.md +++ b/QA/index.md @@ -1,5 +1,5 @@ --- -title: QA: Test Cases +title: QA:Test Cases revision_date: 2022-04-20 --- From abce15d753cb6490008fa81eb1a8f386ecfc1b16 Mon Sep 17 00:00:00 2001 From: Trevor Cooper Date: Fri, 6 May 2022 20:21:54 -0700 Subject: [PATCH 07/45] make QA Test Cases work tracking page visible and add to left nav --- .pages | 1 + QA/index.md => qa_test_cases.md | 0 2 files changed, 1 insertion(+) rename QA/index.md => qa_test_cases.md (100%) diff --git a/.pages b/.pages index d0662e6..b49f367 100644 --- a/.pages +++ b/.pages @@ -2,3 +2,4 @@ nav: - Team Members: index.md - Release Criteria: release_criteria.md + - QA:Test Cases: qa_test_cases.md diff --git a/QA/index.md b/qa_test_cases.md similarity index 100% rename from QA/index.md rename to qa_test_cases.md From 8f605e596202141048d4c1b2b4ba92b7d0d06746 Mon Sep 17 00:00:00 2001 From: Trevor Cooper Date: Fri, 6 May 2022 20:49:54 -0700 Subject: [PATCH 08/45] update assignees and reserve testcase filenames --- qa_test_cases.md | 42 +++++++++++++++++++++--------------------- 1 file changed, 21 insertions(+), 21 deletions(-) diff --git a/qa_test_cases.md b/qa_test_cases.md index b1dbbb0..2b48db8 100644 --- a/qa_test_cases.md +++ b/qa_test_cases.md @@ -1,6 +1,6 @@ --- title: QA:Test Cases -revision_date: 2022-04-20 +revision_date: 2022-05-06 --- This page lists all test cases in work and who is working on them... @@ -22,18 +22,18 @@ This page lists all test cases in work and who is working on them... | Requirement | Test Case | Assignee | Status | | --------------------------------------------------- | ------------------------------------------------------------------------ | ----------------------- | --------------------------------------- | | Media Consistency Verification | [QA:Testcase TBD](Testcase_Template.md) | @tbd | | -| Packages and Installer Sources | [QA:Testcase TBD](Testcase_Template.md) | @tbd | | +| Packages and Installer Sources | [QA:Testcase TBD](Testcase_Template.md) | @tbd | Implemented in openQA, document | | NAS (Network Attached Storage) | [QA:Testcase TBD](Testcase_Template.md) | @tbd | | -| Installation Interfaces | [QA:Testcase TBD](Testcase_Template.md) | @tbd | | -| Minimal Installation | [QA:Testcase TBD](Testcase_Template.md) | @tbd | | -| Kickstart Installation | [QA:Testcase TBD](Testcase_Template.md) | @tbd | | -| Disk Layouts | [QA:Testcase TBD](Testcase_Template.md) | @tbd | | +| Installation Interfaces | [QA:Testcase TBD](Testcase_Template.md) | @tbd | Implemented in openQA, document | +| Minimal Installation | [QA:Testcase TBD](Testcase_Template.md) | @tbd | Implemented in openQA, document | +| Kickstart Installation | [QA:Testcase TBD](Testcase_Template.md) | @tbd | Implemented in openQA, document | +| Disk Layouts | [QA:Testcase TBD](Testcase_Template.md) | @tbd | Implemented in openQA, document | | Firmware RAID | [QA:Testcase TBD](Testcase_Template.md) | @tbd | | | Bootloader Disk Selection | [QA:Testcase TBD](Testcase_Template.md) | @tbd | | -| Storage Volume Resize | [QA:Testcase TBD](Testcase_Template.md) | @tbd | | -| Update Image | [QA:Testcase TBD](Testcase_Template.md) | @tbd | | -| Installer Help | [QA:Testcase TBD](Testcase_Template.md) | @tbd | | -| Installer Translations | [QA:Testcase TBD](Testcase_Template.md) | @tbd | | +| Storage Volume Resize | [QA:Testcase TBD](Testcase_Template.md) | @tbd | Implemented in openQA, document | +| Update Image | [QA:Testcase TBD](Testcase_Template.md) | @tbd | Implemented in openQA, document | +| Installer Help | [QA:Testcase TBD](Testcase_Template.md) | @tbd | Implemented in openQA, document | +| Installer Translations | [QA:Testcase TBD](Testcase_Template.md) | @tbd | Implemented in openQA, document | ## Cloud Image Requirements @@ -45,17 +45,17 @@ This page lists all test cases in work and who is working on them... ## Post-Installation Requirements -| Requirement | Test Case | Assignee | Status | -| --------------------------------------------------- | ------------------------------------------------------------------------ | ----------------------- | --------------------------------------- | -| System Services | [QA:Testcase TBD](Testcase_Template.md) | @tbd | | -| Keyboard Layout | [QA:Testcase TBD](Testcase_Template.md) | @tbd | | -| SELinux Errors (Server) | [QA:Testcase TBD](Testcase_Template.md) | @tbd | | -| SELinux and Crash Notifications (Desktop Only) | [QA:Testcase TBD](Testcase_Template.md) | @tbd | | -| Default Application Functionality (Desktop Only) | [QA:Testcase TBD](Testcase_Template.md) | @tbd | | -| Default Panel Functionality (Desktop Only) | [QA:Testcase TBD](Testcase_Template.md) | @tbd | | -| Dual Monitor Setup (Desktop Only) | [QA:Testcase TBD](Testcase_Template.md) | @tbd | | -| Artwork and Assets (Server and Desktop) | [QA:Testcase TBD](Testcase_Template.md) | @tbd | | -| Packages and Module Installation | [QA:Testcase TBD](Testcase_Template.md) | @tbd | | +| Requirement | Test Case | Assignee | Status | +| --------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------- | ----------------------- | --------------------------------------- | +| System Services | [QA:Testcase System Services](Testcase_Template.md) | @lumarel | testcase filename reserved | +| Keyboard Layout | [QA:Testcase Keyboard Layout](Testcase_Template.md) | @lumarel | testcase filename reserved | +| SELinux Errors (Server) | [QA:Testcase SELinux Errors on Server installations](Testcase_Template.md) | @lumarel, @raktajino | testcase filename reserved | +| SELinux and Crash Notifications (Desktop Only) | [QA:Testcase SELinux Errors on Desktop clients](Testcase_Template.md) | @lumarel, @raktajino | testcase filename reserved | +| Default Application Functionality (Desktop Only) | [QA:Testcase Application Functionality](Testcase_Template.md) | @lumarel | testcase filename reserved | +| Default Panel Functionality (Desktop Only) | [QA:Testcase GNOME UI Functionality](Testcase_Template.md) | @lumarel | testcase filename reserved | +| Dual Monitor Setup (Desktop Only) | [QA:Testcase Multimonitor Setup](Testcase_Template.md) | @lumarel | testcase filename reserved | +| Artwork and Assets (Server and Desktop) | [QA:Testcase Artwork and Assets](Testcase_Template.md) | @lumarel | testcase filename reserved | +| Packages and Module Installation | [QA:Testcase Basic Package installs](Testcase_Template.md)
[QA:Testcase Module Streams](Testcase_Template.md) | @lumarel | | {% include 'content_bottom.md' %} From f56d347960926f185f9f878c7de91a6be86f1e06 Mon Sep 17 00:00:00 2001 From: Al Bowles Date: Sat, 7 May 2022 21:21:11 -0500 Subject: [PATCH 09/45] Update dev box documentation - Add Docker documentation to the wiki development boxes page - Dockerfile will now install git at buildtime --- wiki_development_boxes.md | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/wiki_development_boxes.md b/wiki_development_boxes.md index 9c8ff59..eee9564 100644 --- a/wiki_development_boxes.md +++ b/wiki_development_boxes.md @@ -1,7 +1,9 @@ --- title: Create live devbox for wiki.rockylinux.org -author: Lukas Magauer -revision_date: 2022-04-28 +author: + - Lukas Magauer + - Al Bowles +revision_date: 2022-05-07 --- # How to create a live system to work on the documentation @@ -34,6 +36,10 @@ And finally run `mkdocs serve 2>&1 | tee ./mkdocs.serve.log` in the terminal of ## Docker -TODO +From the root of this repository on a machine with Docker installed, run + + docker-compose up + +When the container finishes starting up, you can access the documentation in your web browser at [http://localhost:8000](http://localhost:8000). {% include 'testing/rc_content_bottom.md' %} From 766a67614480bfc7ddbb706332d396be9087d122 Mon Sep 17 00:00:00 2001 From: Al Bowles Date: Sat, 7 May 2022 21:24:08 -0500 Subject: [PATCH 10/45] Update team roster --- index.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/index.md b/index.md index 8b262d7..8c31f9f 100644 --- a/index.md +++ b/index.md @@ -8,12 +8,12 @@ This section is primarily for documentation for the Testing Team. The testing te ## Members -| Role | Name | Email | Mattermost Name | IRC Name | -| -------------- | --------------- | ----------------------- | ------------------ | -------- | -| Testing Lead | Trevor Cooper | tcooper@rockylinux.org | @tcooper | | -| Testing Lead | Chris Stackpole | stack@rockylinux.org | @stack | | -| Testing Team | Al | | @raktajino | | -| Testing Team | Lukas Magauer | lukas@magauer.eu | @lumarel | | -| Testing Team | Alan Marshall | | @alangm | alangm | +| Role | Name | Email | Mattermost Name | IRC Name | +| -------------- | --------------- | ----------------------- | ------------------ | --------- | +| Testing Lead | Trevor Cooper | tcooper@rockylinux.org | @tcooper | | +| Testing Lead | Chris Stackpole | stack@rockylinux.org | @stack | | +| Testing Team | Al Bowles | | @raktajino | raktajino | +| Testing Team | Lukas Magauer | lukas@magauer.eu | @lumarel | | +| Testing Team | Alan Marshall | | @alangm | alangm | {% include 'content_bottom.md' %} From f8b9c89eee212b52072e10489d8161780ff0abe8 Mon Sep 17 00:00:00 2001 From: Brian Clemens Date: Sun, 8 May 2022 21:46:19 +0900 Subject: [PATCH 11/45] feat: pre-commit --- QA/.pages | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/QA/.pages b/QA/.pages index 9dc1966..840af76 100644 --- a/QA/.pages +++ b/QA/.pages @@ -1,2 +1,2 @@ --- -hide: true \ No newline at end of file +hide: true From a61afd053c4c9b85235010635bea8c6c7d25f085 Mon Sep 17 00:00:00 2001 From: Trevor Cooper Date: Wed, 18 May 2022 23:25:33 -0700 Subject: [PATCH 12/45] release criteria are now versioned for 8 and 9 --- release_criteria/.pages | 5 + release_criteria/index.md | 19 ++ .../r8/8_release_criteria.md | 4 +- release_criteria/r9/9_release_criteria.md | 245 ++++++++++++++++++ 4 files changed, 271 insertions(+), 2 deletions(-) create mode 100644 release_criteria/.pages create mode 100644 release_criteria/index.md rename release_criteria.md => release_criteria/r8/8_release_criteria.md (98%) create mode 100644 release_criteria/r9/9_release_criteria.md diff --git a/release_criteria/.pages b/release_criteria/.pages new file mode 100644 index 0000000..6086810 --- /dev/null +++ b/release_criteria/.pages @@ -0,0 +1,5 @@ +--- +nav: + - ... | index.md + - Rocky Linux 8: r8 + - Rocky Linux 9: r9 diff --git a/release_criteria/index.md b/release_criteria/index.md new file mode 100644 index 0000000..3c00274 --- /dev/null +++ b/release_criteria/index.md @@ -0,0 +1,19 @@ +--- +title: Rocky Linux Release Criteria & Status +author: Trevor Cooper +revision_date: 2022-05-18 +rc: + prod: Rocky Linux +--- + +# {{ rc.prod }} Release Status + +The QA and Testing efforts during releases are tracked in online shared documents. After release the status track and go/no-go documents are published here. + +| Rocky Linux Version | Release Criteria | QA and Testing Status | QA and Testing GO / NO-GO Summary | Official Release Date | +| -------------------- | --------------------------------------- |--------------------------------------- | ---------------------------------------- | --------------------------- | +| {{ rc.prod }} 8.5 | not available | not available | not available | NOVEMBER 15, 2021 | +| {{ rc.prod }} 8.6 | [AVAILABLE](9.0_qa_testing_summary.md) | [AVAILABLE](8.6_qa_testing_summary.md) | [AVAILABLE](8.6_qa_testing_go_no_go.md) | MAY 16, 2022 | +| {{ rc.prod }} 9.0 | [AVAILABLE](9.0_qa_testing_summary.md) | [AVAILABLE](9.0_qa_testing_summary.md) | [AVAILABLE](9.0_qa_testing_go_no_go.md) | TBD | + +{% include 'content_bottom.md' %} diff --git a/release_criteria.md b/release_criteria/r8/8_release_criteria.md similarity index 98% rename from release_criteria.md rename to release_criteria/r8/8_release_criteria.md index 40b3be3..2bace3f 100644 --- a/release_criteria.md +++ b/release_criteria/r8/8_release_criteria.md @@ -8,14 +8,14 @@ rc: level: Final --- -# {{ rc.level }} Release Objectives +# {{ rc.prod }} {{ rc.ver }} {{ rc.level }} Release Objectives The objective of a release (major or minor) is to provide a solid Enterprise Linux release that is suitable to: - Meet the needs of end users - Meet the needs of enterprises big or small -## {{ rc.level }} Release Requirements +## {{ rc.prod }} {{ rc.ver }} {{ rc.level }} Release Requirements In order for {{ rc.prod }} to be released to the general public, a compose must be able to meet all the following criteria as provided in this document. This is allows the decision process to be straightforward and as clear as possible. This document only contains “hard requirement” items. Optional/nice to have items are not to be included in this list. diff --git a/release_criteria/r9/9_release_criteria.md b/release_criteria/r9/9_release_criteria.md new file mode 100644 index 0000000..2603c86 --- /dev/null +++ b/release_criteria/r9/9_release_criteria.md @@ -0,0 +1,245 @@ +--- +title: Rocky Linux 9 Release Criteria +author: Trevor Cooper +revision_date: 2022-04-01 +rc: + prod: Rocky Linux + ver: 9 + level: Final +--- + +# {{ rc.prod }} {{ rc.ver }} {{ rc.level }} Release Objectives + +The objective of a release (major or minor) is to provide a solid Enterprise Linux release that is suitable to: + +- Meet the needs of end users +- Meet the needs of enterprises big or small + +## {{ rc.prod }} {{ rc.ver }} {{ rc.level }} Release Requirements + +In order for {{ rc.prod }} to be released to the general public, a compose must be able to meet all the following criteria as provided in this document. This is allows the decision process to be straightforward and as clear as possible. This document only contains “hard requirement” items. Optional/nice to have items are not to be included in this list. + +There may cases where a requirement cannot be met but only in particular configurations. In these types of cases, the Release Engineering Team should use their judgement to determine whether or not the issue should be considered to block the release. They should consider the number of users likely to be affected by said issue, the severity of the case, if the issue can be avoided with ease (by both informed and uninformed users), and if the problem exists upstream in the current Red Hat Enterprise Linux that the release is based on. + +!!! info "Release-blocking Server" + ...means bugs as it pertains to server functionality can be considered to block a release. This applies to any packages that provide a service such as httpd, nginx, etc. All architectures apply. + +!!! info "Release-blocking Desktop" + ...means bugs as it pertains to desktop functionality (GNOME) can be considered to block a release. This applies to both x86_64 and aarch64. Additional desktops (as provided by EPEL or a SIG) are not considered blockers. + +!!! info "Release-blocking Image" + ...means bugs as it pertains to the images built that can block a release. This applies to the DVD, minimal, and boot images on all architectures. + +### Initialization Requirements + +#### Release-blocking images must boot + +Release-blocking installer images must boot when written to optical media or USB flash drive of appropriate sizes (if applicable) via officially supported methods. It is not the testing team’s responsibility to test optical media, but they can and report back. If a bug is found, it is considered a blocker. + +??? tldr "Optical Media Requirements" + Release-blocking images must boot when written to optical media of an appropriate size. Current size requirements are: boot.iso = 789M, minimal.iso = 2.0G and dvd.iso = 10G. + +??? tldr "Officially supported USB flash drive writing methods" + The following methods of writing USB flash drives are officially support: dd
+ The following methods of writing USB flash drives are _**not**_ supported: rufus + +??? tldr "References" + - Test cases: + - [QA:Testcase Boot Methods Boot ISO](Testcase_Boot_Methods_Boot_Iso.md) + - [QA:Testcase Boot Methods DVD](Testcase_Boot_Methods_Dvd.md) + - [QA:Testcase Media USB dd](Testcase_Media_USB_dd.md) + +#### Basic Graphics Mode behaviors +The generic video driver option (“basic graphics mode”) on all release-blocking installers must function as intended. This means launching the installer or desktop and attempting to use a generic driver. There must be no bugs that prevent the installer from being reached in this configuration on all systems and classes of hardware supported by the enterprise linux kernel. +??? tldr "References" + - Test cases: + - [QA:Testcase Basic Graphics Mode](Testcase_Basic_Graphics_Mode.md) + +#### No Broken Packages +Critical errors, such as undeclared conflicts, unresolved dependencies, or modules relying on packages from another stream will be considered an automatic blocker. There are potential exceptions to this (eg, freeradius cannot be installed on an older perl stream, this is a known issue upstream). +??? tldr "References" + - Test cases: + - [QA:Testcase Media Repoclosure](Testcase_Media_Repoclosure.md) + - [QA:Testcase Media File Conflicts](Testcase_Media_File_Conflicts.md) + +#### Repositories Must Match Upstream +Repositories and the packages within them should match upstream as closely as possible. Notable exceptions would be kmods, kpatch, or what is deemed as “spyware” like insights. Packages that are available from upstream should not have hard requirements on RHSM and packages that have it default built in should be patched out. +??? tldr "References" + - Test cases: + - [QA:Testcase Repo Compare](Testcase_Repo_Compare.md) + - [QA:Testcase Packages No Insights](Testcase_Packages_No_Insights.md) + - [QA:Testcase Packages No RHSM](Testcase_Packages_No_RHSM.md) + +#### Debranding +Assets and functionality that are Red Hat specific should not be included. If they are not patched out, it will be considered an automatic blocker. +??? tldr "References" + - Test cases: + - [QA:Testcase Debranding](Testcase_Debranding.md) + +### Installer Requirements + +#### Media Consistency Verification +This means that the installer’s mechanism for verifying the install medium is intact and must complete successfully, with the assumption that the medium was correctly written. It should return a failure message if this not the case. +??? tldr "References" + - Test cases: + - [QA:Testcase TBD](Testcase_Template.md) + +#### Packages and Installer Sources +The installer must be able to use all supported local/remote packages and installer sources. +??? tldr "References" + - Test cases: + - [QA:Testcase TBD](Testcase_Template.md) + +#### NAS (Network Attached Storage)" +The installer must be able to detect and install to supported NAS devices (if possible and supported by the kernel). +??? tldr "References" + - Test cases: + - [QA:Testcase TBD](Testcase_Template.md) + +#### Installation Interfaces +The installer must be able to complete an installation using all supported spokes. +??? tldr "References" + - Test cases: + - [QA:Testcase TBD](Testcase_Template.md) + +#### Minimal Installation +A minimal installation (via network) must be able to install the minimal package set. +??? tldr "References" + - Test cases: + - [QA:Testcase TBD](Testcase_Template.md) + +#### Kickstart Installation +A kickstart installation should succeed, whether from optical/USB media or via the network. +??? tldr "References" + - Test cases: + - [QA:Testcase TBD](Testcase_Template.md) + +#### Disk Layouts +The installer must be able to create and install to any workable partition layout using any file system or format combination offered or supported by the installer. File systems that are not supported by the EL kernel is not tested here (this means btrfs, zfs, both of wish are not supported). +??? tldr "References" + - Test cases: + - [QA:Testcase TBD](Testcase_Template.md) + +#### Firmware RAID +The installer must be able to detect and install to firmware RAID devices. Note that system-specific bugs do not count as blockers. It is likely that some hardware support might be broken or not available at all. DUDs (driver update disks) are not considered for this criteria. +??? tldr "References" + - Test cases: + - [QA:Testcase TBD](Testcase_Template.md) + +#### Bootloader Disk Selection +The installer must allow the user to choose which disk the bootloader will be installed to or, if the user so chooses, not to install a bootloader. +??? tldr "References" + - Test cases: + - [QA:Testcase TBD](Testcase_Template.md) + +#### Storage Volume Resize +Any installer mechanism for resizing storage volumes must correctly attempt the requested operation. This means that if the installer offers a way to resize storage volumes, then it must use the correct resizing tool with the correct parameters. However, it does not require the installer to disallow resizing of unformatted or volumes with an unknown filesystem type. +??? tldr "References" + - Test cases: + - [QA:Testcase TBD](Testcase_Template.md) + +#### Update Image +The installer must be able to use an installer update image retrieved from removable media or a remote package source. This includes DUDs (driver update disks). +??? tldr "References" + - Test cases: + - [QA:Testcase TBD](Testcase_Template.md) + +#### Installer Help +Any element in the installer which contains a “help” text must display the appropriate help documentation when selected. +??? tldr "References" + - Test cases: + - [QA:Testcase TBD](Testcase_Template.md) + +#### Installer Translations +The installer must correctly display all complete translations that are available for use. +??? tldr "References" + - Test cases: + - [QA:Testcase TBD](Testcase_Template.md) + +### Cloud Image Requirements +#### Images Published to Cloud Providers +Release-blocking cloud disk images must be published to appropriate cloud providers (such as Amazon) and they must successfully boot. This also applies to KVM based instances, such as x86 and aarch64 systems. +??? tldr "References" + - Test cases: + - [QA:Testcase TBD](Testcase_Template.md) + +### Post-Installation Requirements +#### System Services +All system services present after installation must start properly, with the exception of services that require hardware which is not present. Examples of such services would be: + - sshd + - firewalld + - auditd + - chronyd +??? tldr "References" + - Test cases: + - [QA:Testcase TBD](Testcase_Template.md) + +#### Keyboard Layout +If a particular keyboard layout has been configured for the system, that layout must be used: + - When unlocking storage volumes (encrypted by LUKS) + - When logging in at a TTY console + - When logging in via GDM + - After logging into a GNOME desktop system, if the user does not have their own layout configuration set. + +??? tldr "References" + - Test cases: + - [QA:Testcase TBD](Testcase_Template.md) + +#### SELinux Errors (Server) +There must be no SELinux denial logs in /var/log/audit/audit.log + +??? tldr "References" + - Test cases: + - [QA:Testcase TBD](Testcase_Template.md) + +#### SELinux and Crash Notifications (Desktop Only) +There must be no SELinux denial notifications or crash notifications on boot, during installation, or during first login. + +??? tldr "References" + - Test cases: + - [QA:Testcase TBD](Testcase_Template.md) + +#### Default Application Functionality (Desktop Only) +Applications that can be launched within GNOME or on the command line must start successfully and withstand basic functionality tests. This includes: + - Web browser + - File manager + - Package manager + - Image/Document Viewers + - Text editors (gedit, vim) + - Archive manager + - Terminal Emulator (gnome terminal) + - Problem Reporter + - Help Viewer + - System Settings + +??? tldr "References" + - Test cases: + - [QA:Testcase TBD](Testcase_Template.md) + +#### Default Panel Functionality (Desktop Only) +All elements of GNOME should function properly in regular use. +??? tldr "References" + - Test cases: + - [QA:Testcase TBD](Testcase_Template.md) + +#### Dual Monitor Setup (Desktop Only) +Computers using two monitors, the graphical output is correctly shown on both monitors. +??? tldr "References" + - Test cases: + - [QA:Testcase TBD](Testcase_Template.md) + +#### Artwork and Assets (Server and Desktop) +Proposed final artwork (such as wallpapers and other assets) must be included. A wallpaper from this package should show up as a default for GDM and GNOME. +??? tldr "References" + - Test cases: + - [QA:Testcase TBD](Testcase_Template.md) + +#### Packages and Module Installation +Packages (non-module) should be able to be installed without conflicts or dependent on repositories outside of {{ rc.prod }}. + - Default modules (as listed in dnf module list) should be installed without requiring them to be enabled. + - Module streams should be able to be switched and those packages should be able to be installed without errors or unresolved dependencies. +??? tldr "References" + - Test cases: + - [QA:Testcase TBD](Testcase_Template.md) + +{% include 'testing/rc_content_bottom.md' %} From ab736ee034a0af9d5c2764581d5b037d2cf2c757 Mon Sep 17 00:00:00 2001 From: Trevor Cooper Date: Wed, 18 May 2022 23:26:40 -0700 Subject: [PATCH 13/45] add 8.5 QA:Testing Summary and Go/NoGo Status and templates --- .../.qa_testing_summary_template.md | 67 ++++++++++ .../.ver_qa_testing_go_no_go_template.md | 89 +++++++++++++ release_criteria/r8/.pages | 6 + .../r8/8.6_qa_testing_go_no_go.md | 90 +++++++++++++ release_criteria/r8/8.6_qa_testing_summary.md | 121 ++++++++++++++++++ 5 files changed, 373 insertions(+) create mode 100644 release_criteria/.qa_testing_summary_template.md create mode 100644 release_criteria/.ver_qa_testing_go_no_go_template.md create mode 100644 release_criteria/r8/.pages create mode 100644 release_criteria/r8/8.6_qa_testing_go_no_go.md create mode 100644 release_criteria/r8/8.6_qa_testing_summary.md diff --git a/release_criteria/.qa_testing_summary_template.md b/release_criteria/.qa_testing_summary_template.md new file mode 100644 index 0000000..28abe0b --- /dev/null +++ b/release_criteria/.qa_testing_summary_template.md @@ -0,0 +1,67 @@ +--- +title: Release Status Template +author: Trevor Cooper +revision_date: 2022-05-18 +rc: + prod: Rocky Linux + ver: 9 + level: Final +--- + +# {{ rc.prod }} {{ rc.ver }} QA and Testing Summary +Last updated: + +## Scope +This document will record a summary of all QA and Testing results for {{ rc.prod }} {{ rc.9 }} release. It is only a record of success and/or failure. Solution discussion should take place elsewhere. + +## Reference +- Please check RHEL 9 Release Notes **BEFORE** marking issue here as **FAIL**. + +## SOP +- Please include **PASS**, **FAIL**, **NOTABUG**, **INVESTIGATE** or **UPSTREAM** as appropriate in all entries. +- Please only provide brief summary. Details should go to Rocky Pastebin, links here is OK. +- Please leave your MM @handle on all items you have done or are working on so we can talk to you to get resolution. +- If the item you have reported is related to a QA:Testcase please mention it. +- If you think the item you have reported should be a QA:Testcase, even if it's not a current requirement, suggest a title and create an issue in the wiki repository so we can add it. + +## INVESTIGATE +This is a list of items that are being INVESTIGATEd further before being assigned a PASS, FAIL, NOTABUG or UPSTREAM status. +PLEASE add your MM handle if you are working on this item to minimize duplication of work. More than one handle is allowed but please communicate. + +- QA:Testcase Basic Graphics Mode - [INVESTIGATE] - @tcooper +- QA:Testcase Boot Methods Boot Iso - [INVESTIGATE] - @neil +- QA:Testcase Boot Methods DVD - [INVESTIGATE] - @neil +- QA:Testcase Debranding - [INVESTIGATE] - @tcooper +- QA:Testcase Media Consistency Verification - [INVESTIGATE] - @tcooper +- QA:Testcase Media File Conflicts - [INVESTIGATE] - @tcooper +- QA:Testcase Media File Conflicts - [INVESTIGATE] - @tcooper +- QA:Testcase Media File Conflicts - [INVESTIGATE] - @tcooper +- QA:Testcase Media File Conflicts - [INVESTIGATE] - @tcooper +- QA:Testcase Media Repoclosure - [INVESTIGATE] - @tcooper +- QA:Testcase Storage Volume Resize - [INVESTIGATE] - @raktajino +- QA:Testcase Update Image - [INVESTIGATE] - @raktajino +- QA:Testcase boot/install minimal x86_64 over DVD/Bluray - [INVESTIGATE] - @atomicturtle +- QA:Testcase_Mediacheck - [INVESTIGATE] - @tcooper + +## UPSTREAM +This is a list of items that have been verified to be replicated UPSTREAM in RHEL {{ rc.9 }} and/or are described clearly in the RHEL 9 Release Notes. + +- QA:Testcase_Some_Testcase - [UPSTREAM] - @your_mm_handle - + +## FAIL +This is a list of items that have been verified to FAIL the QA:Testcase. In addition to recording who did the test please indicate if the item is BLOCKING release or not. + +- QA:Testcase_Some_Testcase - [FAIL] - @your_mm_handle - + +## NOTABUG +This is a list of items that have been verified as less than optimal but are expected and NOTABUG. + +- QA:Testcase_Some_Testcase - [NOTABUG] - @your_mm_handle - + +## PASS +This is a list of item that have been verified as PASSing the QA:Testcase named (or proposed). + +- QA:Testcase_Some_Testcase - [PASS] - @your_mm_handle - + +## OTHER NOTABLE ITEMS + diff --git a/release_criteria/.ver_qa_testing_go_no_go_template.md b/release_criteria/.ver_qa_testing_go_no_go_template.md new file mode 100644 index 0000000..8590d13 --- /dev/null +++ b/release_criteria/.ver_qa_testing_go_no_go_template.md @@ -0,0 +1,89 @@ +--- +title: Rocky Linux TBD QA and Testing GO / NO GO Status +author: Trevor Cooper +revision_date: 2022-05-18 +rc: + prod: Rocky Linux + ver: TBD +--- + +# {{ rc.prod }} {{ rc.ver }} QA and Testing : GO / NO GO Status +This document summarizes the GO / NO GO Status for Release of Rocky Linux 8.6 from perspective of the QA / Testing efforts. It is based largely on the Release Criteria (https://wiki.rockylinux.org/team/testing/release_criteria/) as was started as an import of that document. If there are differences between the official Release Critieria document and this document the official Release Criteria document will override and this document shall be updated. + +As a reminder, the objective of a release (major or minor) is to provide a solid Enterprise Linux release that is suitable to: +- Meet the needs of end users +- Meet the needs of enterprises big or small + + +## SUMMARY + +| Category | Proportion | Remaining Items | +| ----------------- | ----------------- | --------------- | +| TO_CONFIRM | 0 / 29 (0%) | | +| PASS | 0 / 29 (0%) | | +| FAIL_NON_BLOCKING | 0 / 29 (0%) | | +| FAIL_BLOCKING | 0 / 29 (0%) | | + + +## SOP +In this document each requirement is described and status is specified in the title. + +Current choices are... + +### TO_CONFIRM +- this means the item may be INCOMPLETE, PASS, FAIL_NON_BLOCKING or FAIL_BLOCKING and must be verified + +### PASS +- this means that the release criteria has been met and is not a blocker + +### FAIL_NON_BLOCKING +- this means the release criter has not been met but is non-blocking + +### FAIL_BLOCKING +- this means the release criteria has not been met and is blocking + +In this document criteria status should include who completed the item and generally how it was complete. + +Examples... + +- PASS - (@tcooper, virt only, manual) +- PASS - (@lumarel, @raktajino, @tcooper, semi-automatic , openQA) + + +## Initialization Requirements + +- Release-blocking images must boot - PASS - (@neil, @atomicturtle) +- Optical Media Requirements - PASS - (@atomicturtle) +- Basic Graphics Mode behaviors - PASS - (@tcooper, virt only, manual) +- No Broken Packages - PASS - (@tcooper, scripted, manual) +- Repositories Must Match Upstream - TO_CONFIRM - (@tcooper, manual) +- Debranding - PASS - (@tcooper, scripted, manual) +- Media Consistency Verification - PASS - (@tcooper, scripted, manual) +- Packages and Installer Sources - PASS - (@lumarel, semi-automatic, openQA test) +- NAS (Network Attached Storage) - TO_CONFIRM - (@lumarel?, semi-automatic, openQA dual-host test) +- Installation Interfaces - PASS - (@lumarel, @raktajino, @tcooper, semi-automatic, openQA, @atomicturtle, manual?, SCAP) +- Minimal Installation - PASS - (@lumarel, @raktajino, @tcooper, semi-automatic , openQA) +- Kickstart Installation - PASS - (@label, @tcooper, manual, createhdds) +- Disk Layouts - PASS - (@lumarel, @raktajino, @tcooper, semi-automatic, openQA) +- Firmware RAID - TO_CONFIRM - (@tbd, missing hardware support?) +- Bootloader Disk Selection - PASS - (@raktajino, manual) +- Storage Volume Resize - PASS - (@lumarel, @raktajino, @tcooper, semi-automatic, openQA) +- Update Image - PASS - (@raktajino,@tcooper, semi-automatic, openQA) +- Installer Help - PASS - (@lumarel, @raktajino, @tcooper, semi-automatic, openQA) +- Installer Translations - PASS - (@lumarel, @raktajino, @tcooper, semi-automatic, openQA) + + +## Cloud Image Requirements +- Images Published to Cloud Providers - FAIL_NON_BLOCKING - (@neil) + + +## Post-Installation Requirements +- System Services - PASS - (@lumarel, semi-automatic, openQA) +- Keyboard Layout - PASS - (@lumarel, semi-automatic, openQA) +- SELinux Errors (Server) - PASS - (@lumarel, semi-automatic, openQA) +- SELinux and Crash Notifications (Desktop Only) - PASS - (@lumarel, semi-automatic, openQA) +- Default Application Functionality (Desktop Only) - PASS - (@lumarel, semi-automatic, openQA) +- Default Panel Functionality (Desktop Only) - PASS - (@lumarel, semi-automatic, openQA) +- Dual Monitor Setup (Desktop Only) - PASS - (@lumarel, semi-automatic, openQA) +- Artwork and Assets (Server and Desktop) - PASS - (@lumarel, semi-automatic, openQA) +- Packages and Module Installation - PASS - (@lumarel, semi-automatic, openQA) diff --git a/release_criteria/r8/.pages b/release_criteria/r8/.pages new file mode 100644 index 0000000..cff21a5 --- /dev/null +++ b/release_criteria/r8/.pages @@ -0,0 +1,6 @@ +--- +nav: + - ... | index.md + - Rocky Linux 8 Release Criteria: 8_release_criteria.md + - Rocky Linux 8.6 QA and Testing Summary: 8.6_qa_testing_summary.md + - Rocky Linux 8.6 GO / NO-GO Status: 8.6_qa_testing_go_no_go.md diff --git a/release_criteria/r8/8.6_qa_testing_go_no_go.md b/release_criteria/r8/8.6_qa_testing_go_no_go.md new file mode 100644 index 0000000..26ca71c --- /dev/null +++ b/release_criteria/r8/8.6_qa_testing_go_no_go.md @@ -0,0 +1,90 @@ +--- +title: Rocky Linux 8.6 QA and Testing GO / NO GO Status +author: Trevor Cooper +revision_date: 2022-05-18 +rc: + prod: Rocky Linux + ver: 8.6 + level: Final +--- + +# Rocky Linux 8.6 QA and Testing : GO / NO GO Status +This document summarizes the GO / NO GO Status for Release of Rocky Linux 8.6 from perspective of the QA / Testing efforts. It is based largely on the Release Criteria (https://wiki.rockylinux.org/team/testing/release_criteria/) as was started as an import of that document. If there are differences between the official Release Critieria document and this document the official Release Criteria document will override and this document shall be updated. + +As a reminder, the objective of a release (major or minor) is to provide a solid Enterprise Linux release that is suitable to: +- Meet the needs of end users +- Meet the needs of enterprises big or small + + +## SUMMARY + +| Category | Proportion | Remaining Items | +| --------------------- | ----------------- | --------------- | +| **TO_CONFIRM** | 3 / 29 (10%) | | +| **PASS** | 25 / 29 (86%) | | +| **FAIL_NON_BLOCKING** | 1 / 29 (3%) | cloud-images | +| FAIL_BLOCKING | 0 / 29 (0%) | | + + +## SOP +In this document each requirement is described and status is specified in the title. + +Current choices are... + +### TO_CONFIRM +- this means the item may be **[INCOMPLETE]**, **[PASS]**, **[FAIL_NON_BLOCKING[]** or **[FAIL_BLOCKING]** and must be verified + +### PASS +- this means that the release criteria has been met and is not a blocker + +### FAIL_NON_BLOCKING +- this means the release criter has not been met but is non-blocking + +### FAIL_BLOCKING +- this means the release criteria has not been met and is blocking + +In this document criteria status should include who completed the item and generally how it was complete. + +Examples... + +- **[PASS]** - (@tcooper, virt only, manual) +- **[PASS]** - (@lumarel, @raktajino, @tcooper, semi-automatic , openQA) + + +## Initialization Requirements + +- Release-blocking images must boot - **[PASS]** - (@neil, @atomicturtle) +- Optical Media Requirements - **[PASS]** - (@atomicturtle) +- Basic Graphics Mode behaviors - **[PASS]** - (@tcooper, virt only, manual) +- No Broken Packages - **[PASS]** - (@tcooper, scripted, manual) +- Repositories Must Match Upstream -**TO_CONFIRM**- (@tcooper, manual) +- Debranding - **[PASS]** - (@tcooper, scripted, manual) +- Media Consistency Verification - **[PASS]** - (@tcooper, scripted, manual) +- Packages and Installer Sources - **[PASS]** - (@lumarel, semi-automatic, openQA test) +- NAS (Network Attached Storage) -**TO_CONFIRM**- (@lumarel?, semi-automatic, openQA dual-host test) +- Installation Interfaces - **[PASS]** - (@lumarel, @raktajino, @tcooper, semi-automatic, openQA, @atomicturtle, manual?, SCAP) +- Minimal Installation - **[PASS]** - (@lumarel, @raktajino, @tcooper, semi-automatic , openQA) +- Kickstart Installation - **[PASS]** - (@label, @tcooper, manual, createhdds) +- Disk Layouts - **[PASS]** - (@lumarel, @raktajino, @tcooper, semi-automatic, openQA) +- Firmware RAID -**TO_CONFIRM**- (@tbd, missing hardware support?) +- Bootloader Disk Selection - **[PASS]** - (@raktajino, manual) +- Storage Volume Resize - **[PASS]** - (@lumarel, @raktajino, @tcooper, semi-automatic, openQA) +- Update Image - **[PASS]** - (@raktajino,@tcooper, semi-automatic, openQA) +- Installer Help - **[PASS]** - (@lumarel, @raktajino, @tcooper, semi-automatic, openQA) +- Installer Translations - **[PASS]** - (@lumarel, @raktajino, @tcooper, semi-automatic, openQA) + + +## Cloud Image Requirements +- Images Published to Cloud Providers - **FAIL_NON_BLOCKING**- (@neil) + + +## Post-Installation Requirements +- System Services - **[PASS]** - (@lumarel, semi-automatic, openQA) +- Keyboard Layout - **[PASS]** - (@lumarel, semi-automatic, openQA) +- SELinux Errors (Server) - **[PASS]** - (@lumarel, semi-automatic, openQA) +- SELinux and Crash Notifications (Desktop Only) - **[PASS]** - (@lumarel, semi-automatic, openQA) +- Default Application Functionality (Desktop Only) - **[PASS]** - (@lumarel, semi-automatic, openQA) +- Default Panel Functionality (Desktop Only) - **[PASS]** - (@lumarel, semi-automatic, openQA) +- Dual Monitor Setup (Desktop Only) - **[PASS]** - (@lumarel, semi-automatic, openQA) +- Artwork and Assets (Server and Desktop) - **[PASS]** - (@lumarel, semi-automatic, openQA) +- Packages and Module Installation - **[PASS]** - (@lumarel, semi-automatic, openQA) diff --git a/release_criteria/r8/8.6_qa_testing_summary.md b/release_criteria/r8/8.6_qa_testing_summary.md new file mode 100644 index 0000000..31b69ac --- /dev/null +++ b/release_criteria/r8/8.6_qa_testing_summary.md @@ -0,0 +1,121 @@ +--- +title: Rocky Linux 8.6 QA and Testing Summary +author: Trevor Cooper +revision_date: 2022-05-18 +rc: + prod: Rocky Linux + ver: 8.6 + level: Final +--- + +# Rocky Linux 8.6 QA and Testing Summary +**_Last updated: Fri May 13 17:36:41 UTC 2022_** + + +## Scope +This document will record a summary of all QA and Testing results for Rocky Linux 8.6 release. It is only a record of success and/or failure. Solution discussion should take place elsewhere. + + +## SOP +- Please include PASS, FAIL, NOTABUG, INVESTIGATE or UPSTREAM as appropriate in all entries. +- Please only provide brief summary. Details should go to Rocky Pastebin, links here is OK. +- Especially for an negative result please leave your MM @handle so we can talk to you to get resolution. +- If the item you are reported is related to a QA:Testcase please mention it. If it should be a QA:Testcase, even if it's not a current requirement, suggest a title and create an issue in the wiki repository so we can add it. + + +## INVESTIGATE +- errors in all tests in openQA - INVESTIGATE - see openQA section below. +- INVESTIGATE whether kdump issue affects qcows with encrypted partitions in createhdds. These are pre-reqs for openQA multi-machine tests. NOTE: This is addressed below in section - @tcooper +- KDE and XFCE Life images are broken - INVESTIGATE - @label + + +## UPSTREAM +- Anaconda error when specific steps get executed in the right order (configure network -> disable kdump -> select some os install group -> configure default storage -> configure the storage a second time, but this time with encryption enabled, confirmed via several openqa test suites and manual installation on ESXi) - retested in RC1_2 - UPSTREAM - @lumarel +- Issue repeated on RHEL8.6 - @atomicturtle +- Issue reported to RH https://bugzilla.redhat.com/show_bug.cgi?id=2085321 - @stack + + +## FAIL +- ESXi secureboot (x86_64) still FAILing, but expected - @lumarel + + +## NOTABUG +- Minimal: Selecting a SCAP profile with dependencies not available (aide, etc), selecting Ignore dependency during installation will crash anaconda at the final oscap check. NOTABUG, this is for documentation - @atomicturtle +- Minimal ISO: is missing the source for rsyslog again, and somehow also doesn't pull it in when installing, which means it is missing it after the install (doesn't happen for boot ISO and dvd1 ISO) - NOTABUG (per @label) - @lumarel +- Minimal ISO: if the server base environment is installed with the minimal iso and cockpit is enabled after the installation, the SELinux submenu shows an error "semanage: command not found" (doesn't happen with boot/dvd-iso) - also marked as expected - NOTABUG (per @label) - @lumarel + + +## Manual success reported in MM +- minimal install from minimal ISO fine - PASS - @Luna Jernberg +- workstation (x86_64) install with applications fine - retested in RC1_2 - PASS - @lumarel +- all repos are available with the exact naming as they are in the rocky-repos package (nfv needs fix for that) - retested in RC1_2 - PASS - @lumarel +- packer build for 8.6 worked flawlessly - retested by @neil in RC1_2 - PASS - @gmazrael +- security profiles look good in anaconda UI - PASS - @atomicturtle (need openQA testing) +- minimal and dvd recognized as Rocky Linux 8 in KVM - PASS - @atomicturtle +- CIS profiles confirmed good in lvl1/2 in anaconda - PASS - @atomicturtle +- DISA profiles confirmed good in anaconda - PASS - @atomicturtle +- DVD: libvirt correctly boots when Rocky Linux 8 profile is selected - PASS - @atomicturtle +- SELinux checks on Server (x86_64) (letting it run for an hour and run sealert -a /var/log/audit/audit.log) - everything okay - retested in RC1_2 - PASS - @lumarel +- SELinux checks on Desktop (x86_64) (start the GNOME SETroubleshooter after some minutes of running) - everything okay - retested in RC1_2 - PASS - @lumarel +- DVD: Anaconda manual network configuration, and PCI-DSS SCAP profile selected confirmed good - PASS - @atomicturtle +- QA:Testcase_Mediacheck - PASS for all x86_64 ISOs - @tcooper +- QA:Testcase_Mediacheck - PASS for all aarch64 ISOs - @tcooper +- QA:Testcase Media Repoclosure - PASS for minimal & dvd1 for x86_64 & aarch64 (confirms RelEng results) - @tcooper +- QA:Testcase Media File Conflicts - PASS for minimal for x86_64 & aarch64 (0 file conflicts found and 0 package conflicts found) - @tcooper +- QA:Testcase Basic Graphics Mode - PASS - verified manually for Rocky-8.6-x86_64-dvd1.iso in VirtualBox on macOS X - @tcooper +- DVD: Anaconda install with 3rd party repo, encrypted filesystem, HIPAA SCAP profile selected, confirmed good - PASS - @atomicturtle +- Upgrade tests on several test machines from 8.5 to 8.6, no issues no SELinux alerts - PASS - @lumarel +- All module streams except perl:5.32 and log4j:2 correctly have the dependencies set and packages look to be built correctly - PASS - @lumarel +- log4j module stream was broken, (should be able to hook against java-8 and 11) - got fixed now in RC1_2 - PASS - @lumarel +- Anything perl 5.32 (module stream) was broken - got fixed in RC1_2 - PASS - @nazunalika +- Greenbone appliance installation test (https://rpa.st/DQNA) - PASS - @atomicturtle +- QA:Testcase Debranding for RC2 content from koji (srpms, kernel-rt and pcs are not all on the dvd1) - 46/47 PASS , 1 FALSE PASS - https://rpa.st/raw/QK3A - @tcooper +- QA:Testcase Media Consistency Verification (not written yet) for all RC2 ISOs x86_64, aarch64 - PASS - @tcooper +- QA:Testcase Media File Conflicts - EXPECTED(per @label) for Rocky-8.6-x86_64-dvd1.iso (4 file conflicts found and 13 package conflicts found, these appear to be same as 8.5 conflict between mariadb and mysql packages/files, full results - https://rpa.st/raw/ZWPQ) - @tcooper +- QA:Testcase Media File Conflicts - EXPECTED(per @label) for Rocky-8.6-aarch64-dvd1.iso (modular dependency problems, 3 file conflicts found 4 package conflicts found, full results - https://rpa.st/raw/KOFQ) - @tcooper +- QA:Testcase Media File Conflicts for both x86_64 (https://rpa.st/raw/NLGA) and aarch64 (https://rpa.st/raw/4SFQ) are essentially unchanged and remain - EXPECTED(per @label) with RC1_2 ISOs. - @tcooper +- OpenQA tests @lumarel - there are errors from the test cases, but everything image and repo related looks good - PASS - @lumarel +- the dvd1 iso of aarch64 doesn't show an workstation base environment - it doesn't have an workstation environment - PASS - @lumarel +- Installs of aarch64 systems of all 3 isos look good and installs with all base environments work as expected from these - PASS - @lumarel +- Live Image Workstation and Workstation Lite looks good - PASS - @lumarel +- QA:Testcase Boot Methods Boot Iso - PASS - @neil +- QA:Testcase Boot Methods DVD - PASS - @neil +- QA: Testcase boot/install minimal x86_64 over DVD/Bluray (burned with fedora mediawriter) on G752 ASUS laptop - PASS - @atomicturtle +- Container images for x86_64 and aarch64 work as expected in Docker, Podman and WSL - PASS - @lumarel +- QA: Testcase Storage Volume Resize - PASS - @raktajino https://rpa.st/MQSA +- QA: Testcase Update Image - PASS - @raktajino (manually checked against Fedora's testcase (https://fedoraproject.org/wiki/QA:Testcase_Anaconda_updates.img_via_URL), needles also still match in openQA) + + +## openQA summary +- errors in all tests in openQA - INVESTIGATE +- RC1_1 run 1: + - @lumarel: https://rpa.st/CCPQ + - @raktajino: https://rpa.st/5RVA +- RC1_1 run 2 + - @lumarel: https://rpa.st/FWTQ + - @raktajino: +- RC1_2 run 1: + - @lumarel: https://rpa.st/EOGQ +- @raktajino:https://rpa.st/VHLQ + - RC1_2 run 2: +- @raktajino: https://rpa.st/DKCQ +- Upgrade F35 -> F36 needs postgresql-setup --upgrade to convert openqa databse to new format - @alangm +- Per discussion in Testing Team meeting we have 4-8 (ish) issues to fix in openQA (needles and code) to be able to complete all tests. @lumarel has created issues our openQA repo (https://github.com/rocky-linux/os-autoinst-distri-rocky) and we'll pick up and resolve ASAP. + + +##createhdds kickstart file test summary + +Test method: Used packer to build VM. Booted VM. Verified root login. Shutdown VM. + +- UEFI Testing: + - desktop.ks - PASS - Note: resulting image asks for EULA acceptance when booted due to `firstboot --enable` (unsure if that is desired behavior) + - desktopencrypt.ks - PASS - Note: resulting image asks for EULA acceptance when booted due to `firstboot --enable` (unsure if that is desired behavior) + - minimal-uefi.ks - PASS + - server.ks - PASS + - support.ks - PASS +- BIOS Testing: + - desktop.ks - PASS - Note: resulting image asks for EULA acceptance when booted due to `firstboot --enable` (unsure if that is desired behavior) + - desktopencrypt.ks - PASS - Note: resulting image asks for EULA acceptance when booted due to `firstboot --enable` (unsure if that is desired behavior) + - minimal.ks - PASS + - server.ks - PASS + - support.ks - PASS \ No newline at end of file From f82431b09c958d8d6751d9eaca8011b7c96e9b2f Mon Sep 17 00:00:00 2001 From: Trevor Cooper Date: Wed, 18 May 2022 23:27:43 -0700 Subject: [PATCH 14/45] add 9.0 QA:Testing Summary and Go/NoGo Status skeleton docs --- release_criteria/r9/.pages | 6 ++ .../r9/9.0_qa_testing_go_no_go.md | 89 +++++++++++++++++++ release_criteria/r9/9.0_qa_testing_summary.md | 67 ++++++++++++++ 3 files changed, 162 insertions(+) create mode 100644 release_criteria/r9/.pages create mode 100644 release_criteria/r9/9.0_qa_testing_go_no_go.md create mode 100644 release_criteria/r9/9.0_qa_testing_summary.md diff --git a/release_criteria/r9/.pages b/release_criteria/r9/.pages new file mode 100644 index 0000000..a69df71 --- /dev/null +++ b/release_criteria/r9/.pages @@ -0,0 +1,6 @@ +--- +nav: + - ... | index.md + - Rocky Linux 9 Release Criteria: 9_release_criteria.md + - Rocky Linux 9.0 QA and Testing Summary: 9.0_qa_testing_summary.md + - Rocky Linux 9.0 GO / NO-GO Status: 9.0_qa_testing_go_no_go.md diff --git a/release_criteria/r9/9.0_qa_testing_go_no_go.md b/release_criteria/r9/9.0_qa_testing_go_no_go.md new file mode 100644 index 0000000..1b15c9a --- /dev/null +++ b/release_criteria/r9/9.0_qa_testing_go_no_go.md @@ -0,0 +1,89 @@ +--- +title: Rocky Linux 9.0 QA and Testing GO / NO GO Status +author: Trevor Cooper +revision_date: 2022-05-18 +rc: + prod: Rocky Linux + ver: 9 +--- + +# {{ rc.prod }} {{ rc.ver }} QA and Testing : GO / NO GO Status +This document summarizes the GO / NO GO Status for Release of Rocky Linux 8.6 from perspective of the QA / Testing efforts. It is based largely on the Release Criteria (https://wiki.rockylinux.org/team/testing/release_criteria/) as was started as an import of that document. If there are differences between the official Release Critieria document and this document the official Release Criteria document will override and this document shall be updated. + +As a reminder, the objective of a release (major or minor) is to provide a solid Enterprise Linux release that is suitable to: +- Meet the needs of end users +- Meet the needs of enterprises big or small + + +## SUMMARY + +| Category | Proportion | Remaining Items | +| ----------------- | ----------------- | --------------- | +| TO_CONFIRM | 0 / 29 (0%) | | +| PASS | 0 / 29 (0%) | | +| FAIL_NON_BLOCKING | 0 / 29 (0%) | | +| FAIL_BLOCKING | 0 / 29 (0%) | | + + +## SOP +In this document each requirement is described and status is specified in the title. + +Current choices are... + +### TO_CONFIRM +- this means the item may be INCOMPLETE, PASS, FAIL_NON_BLOCKING or FAIL_BLOCKING and must be verified + +### PASS +- this means that the release criteria has been met and is not a blocker + +### FAIL_NON_BLOCKING +- this means the release criter has not been met but is non-blocking + +### FAIL_BLOCKING +- this means the release criteria has not been met and is blocking + +In this document criteria status should include who completed the item and generally how it was complete. + +Examples... + +- PASS - (@tcooper, virt only, manual) +- PASS - (@lumarel, @raktajino, @tcooper, semi-automatic , openQA) + + +## Initialization Requirements + +- Release-blocking images must boot - PASS - (@neil, @atomicturtle) +- Optical Media Requirements - PASS - (@atomicturtle) +- Basic Graphics Mode behaviors - PASS - (@tcooper, virt only, manual) +- No Broken Packages - PASS - (@tcooper, scripted, manual) +- Repositories Must Match Upstream - TO_CONFIRM - (@tcooper, manual) +- Debranding - PASS - (@tcooper, scripted, manual) +- Media Consistency Verification - PASS - (@tcooper, scripted, manual) +- Packages and Installer Sources - PASS - (@lumarel, semi-automatic, openQA test) +- NAS (Network Attached Storage) - TO_CONFIRM - (@lumarel?, semi-automatic, openQA dual-host test) +- Installation Interfaces - PASS - (@lumarel, @raktajino, @tcooper, semi-automatic, openQA, @atomicturtle, manual?, SCAP) +- Minimal Installation - PASS - (@lumarel, @raktajino, @tcooper, semi-automatic , openQA) +- Kickstart Installation - PASS - (@label, @tcooper, manual, createhdds) +- Disk Layouts - PASS - (@lumarel, @raktajino, @tcooper, semi-automatic, openQA) +- Firmware RAID - TO_CONFIRM - (@tbd, missing hardware support?) +- Bootloader Disk Selection - PASS - (@raktajino, manual) +- Storage Volume Resize - PASS - (@lumarel, @raktajino, @tcooper, semi-automatic, openQA) +- Update Image - PASS - (@raktajino,@tcooper, semi-automatic, openQA) +- Installer Help - PASS - (@lumarel, @raktajino, @tcooper, semi-automatic, openQA) +- Installer Translations - PASS - (@lumarel, @raktajino, @tcooper, semi-automatic, openQA) + + +## Cloud Image Requirements +- Images Published to Cloud Providers - FAIL_NON_BLOCKING - (@neil) + + +## Post-Installation Requirements +- System Services - PASS - (@lumarel, semi-automatic, openQA) +- Keyboard Layout - PASS - (@lumarel, semi-automatic, openQA) +- SELinux Errors (Server) - PASS - (@lumarel, semi-automatic, openQA) +- SELinux and Crash Notifications (Desktop Only) - PASS - (@lumarel, semi-automatic, openQA) +- Default Application Functionality (Desktop Only) - PASS - (@lumarel, semi-automatic, openQA) +- Default Panel Functionality (Desktop Only) - PASS - (@lumarel, semi-automatic, openQA) +- Dual Monitor Setup (Desktop Only) - PASS - (@lumarel, semi-automatic, openQA) +- Artwork and Assets (Server and Desktop) - PASS - (@lumarel, semi-automatic, openQA) +- Packages and Module Installation - PASS - (@lumarel, semi-automatic, openQA) diff --git a/release_criteria/r9/9.0_qa_testing_summary.md b/release_criteria/r9/9.0_qa_testing_summary.md new file mode 100644 index 0000000..e6deff0 --- /dev/null +++ b/release_criteria/r9/9.0_qa_testing_summary.md @@ -0,0 +1,67 @@ +--- +title: Rocky Linux 9.0 QA and Testing Summary +author: Trevor Cooper +revision_date: 2022-05-18 +rc: + prod: Rocky Linux + ver: 9 + level: Final +--- + +# {{ rc.prod }} {{ rc.ver }} QA and Testing Summary +Last updated: + +## Scope +This document will record a summary of all QA and Testing results for {{ rc.prod }} {{ rc.9 }} release. It is only a record of success and/or failure. Solution discussion should take place elsewhere. + +## Reference +- Please check RHEL 9 Release Notes **BEFORE** marking issue here as **FAIL**. + +## SOP +- Please include **PASS**, **FAIL**, **NOTABUG**, **INVESTIGATE** or **UPSTREAM** as appropriate in all entries. +- Please only provide brief summary. Details should go to Rocky Pastebin, links here is OK. +- Please leave your MM @handle on all items you have done or are working on so we can talk to you to get resolution. +- If the item you have reported is related to a QA:Testcase please mention it. +- If you think the item you have reported should be a QA:Testcase, even if it's not a current requirement, suggest a title and create an issue in the wiki repository so we can add it. + +## INVESTIGATE +This is a list of items that are being INVESTIGATEd further before being assigned a PASS, FAIL, NOTABUG or UPSTREAM status. +PLEASE add your MM handle if you are working on this item to minimize duplication of work. More than one handle is allowed but please communicate. + +- QA:Testcase Basic Graphics Mode - [INVESTIGATE] - @tcooper +- QA:Testcase Boot Methods Boot Iso - [INVESTIGATE] - @neil +- QA:Testcase Boot Methods DVD - [INVESTIGATE] - @neil +- QA:Testcase Debranding - [INVESTIGATE] - @tcooper +- QA:Testcase Media Consistency Verification - [INVESTIGATE] - @tcooper +- QA:Testcase Media File Conflicts - [INVESTIGATE] - @tcooper +- QA:Testcase Media File Conflicts - [INVESTIGATE] - @tcooper +- QA:Testcase Media File Conflicts - [INVESTIGATE] - @tcooper +- QA:Testcase Media File Conflicts - [INVESTIGATE] - @tcooper +- QA:Testcase Media Repoclosure - [INVESTIGATE] - @tcooper +- QA:Testcase Storage Volume Resize - [INVESTIGATE] - @raktajino +- QA:Testcase Update Image - [INVESTIGATE] - @raktajino +- QA:Testcase boot/install minimal x86_64 over DVD/Bluray - [INVESTIGATE] - @atomicturtle +- QA:Testcase_Mediacheck - [INVESTIGATE] - @tcooper + +## UPSTREAM +This is a list of items that have been verified to be replicated UPSTREAM in RHEL {{ rc.9 }} and/or are described clearly in the RHEL 9 Release Notes. + +- QA:Testcase_Some_Testcase - [UPSTREAM] - @your_mm_handle - + +## FAIL +This is a list of items that have been verified to FAIL the QA:Testcase. In addition to recording who did the test please indicate if the item is BLOCKING release or not. + +- QA:Testcase_Some_Testcase - [FAIL] - @your_mm_handle - + +## NOTABUG +This is a list of items that have been verified as less than optimal but are expected and NOTABUG. + +- QA:Testcase_Some_Testcase - [NOTABUG] - @your_mm_handle - + +## PASS +This is a list of item that have been verified as PASSing the QA:Testcase named (or proposed). + +- QA:Testcase_Some_Testcase - [PASS] - @your_mm_handle - + +## OTHER NOTABLE ITEMS + From 84e3e46ea24032c66565256bc4479e56abe5a3a9 Mon Sep 17 00:00:00 2001 From: Trevor Cooper Date: Wed, 18 May 2022 23:28:49 -0700 Subject: [PATCH 15/45] adjust nav in Testing Team area --- .pages | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.pages b/.pages index b49f367..c6fae80 100644 --- a/.pages +++ b/.pages @@ -1,5 +1,5 @@ --- nav: - Team Members: index.md - - Release Criteria: release_criteria.md - QA:Test Cases: qa_test_cases.md + - Release Criteria & Status: release_criteria From 3ff38c8ce16a17d8400ef746e02dd6d8b63c304c Mon Sep 17 00:00:00 2001 From: Trevor Cooper Date: Wed, 18 May 2022 23:54:36 -0700 Subject: [PATCH 16/45] let Autolinks plugin do it's thing - don't use relative links --- QA/Testcase_Basic_Graphics_Mode.md | 6 +++--- QA/Testcase_Boot_Methods_Boot_Iso.md | 4 ++-- QA/Testcase_Boot_Methods_Dvd.md | 4 ++-- QA/Testcase_Debranding.md | 4 ++-- QA/Testcase_Media_File_Conflicts.md | 4 ++-- QA/Testcase_Media_Repoclosure.md | 4 ++-- QA/Testcase_Media_USB_dd.md | 4 ++-- QA/Testcase_Packages_No_Insights.md | 4 ++-- QA/Testcase_Packages_No_RHSM.md | 4 ++-- QA/Testcase_Repo_Compare.md | 4 ++-- QA/Testcase_Template.md | 4 ++-- release_criteria/.pages | 2 +- release_criteria/{index.md => release_criteria.md} | 0 13 files changed, 24 insertions(+), 24 deletions(-) rename release_criteria/{index.md => release_criteria.md} (100%) diff --git a/QA/Testcase_Basic_Graphics_Mode.md b/QA/Testcase_Basic_Graphics_Mode.md index 7207935..a65e8f2 100644 --- a/QA/Testcase_Basic_Graphics_Mode.md +++ b/QA/Testcase_Basic_Graphics_Mode.md @@ -1,7 +1,7 @@ --- title: QA:Testcase Basic Graphics Mode author: Trevor Cooper -revision_date: 2022-04-24 +revision_date: 2022-05-18 rc: prod: Rocky Linux ver: 8 @@ -9,10 +9,10 @@ rc: --- !!! error "REFERENCED RELEASE CRITERIA IS OVERLY GENERAL AND UNTESTABLE" - The associated release criteria, [Release_Criteria#basic-graphics-mode-behaviors](../release_criteria.md#basic-graphics-mode-behaviors), for this test case is overly general and **must** be modified to specific enough to be testable. + The associated release criteria, [Release_Criteria#basic-graphics-mode-behaviors](release_criteria.md#basic-graphics-mode-behaviors), for this test case is overly general and **must** be modified to specific enough to be testable. !!! info "Associated release criterion" - This test case is associated with the [Release_Criteria#basic-graphics-mode-behaviors](../release_criteria.md#basic-graphics-mode-behaviors) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + This test case is associated with the [Release_Criteria#basic-graphics-mode-behaviors](release_criteria.md#basic-graphics-mode-behaviors) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. ## Description This test case will verify that release-blocking installers function as intended using the generic video driver option (“basic graphics mode”) on supported systems and classes of hardware. diff --git a/QA/Testcase_Boot_Methods_Boot_Iso.md b/QA/Testcase_Boot_Methods_Boot_Iso.md index 5b2662a..97d6417 100644 --- a/QA/Testcase_Boot_Methods_Boot_Iso.md +++ b/QA/Testcase_Boot_Methods_Boot_Iso.md @@ -1,7 +1,7 @@ --- title: QA:Testcase Boot Methods Boot Iso author: Trevor Cooper -revision_date: 2021-04-21 +revision_date: 2022-05-18 rc: prod: Rocky Linux ver: 8 @@ -9,7 +9,7 @@ rc: --- !!! info "Associated release criterion" - This test case is associated with the [Release_Criteria#initialization-requirements](../release_criteria.md#initialization-requirements) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + This test case is associated with the [Release_Criteria#initialization-requirements](release_criteria.md#initialization-requirements) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. ## Description This is to verify that the Anaconda installer starts correctly when booting from the Rocky Linux boot.iso. diff --git a/QA/Testcase_Boot_Methods_Dvd.md b/QA/Testcase_Boot_Methods_Dvd.md index 422886a..51af408 100644 --- a/QA/Testcase_Boot_Methods_Dvd.md +++ b/QA/Testcase_Boot_Methods_Dvd.md @@ -1,7 +1,7 @@ --- title: QA:Testcase Boot Methods DVD author: Trevor Cooper -revision_date: 2021-04-21 +revision_date: 2022-05-18 rc: prod: Rocky Linux ver: 8 @@ -9,7 +9,7 @@ rc: --- !!! info "Associated release criterion" - This test case is associated with the [Release_Criteria#initialization-requirements](../release_criteria.md#initialization-requirements) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + This test case is associated with the [Release_Criteria#initialization-requirements](release_criteria.md#initialization-requirements) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. ## Description This is to verify that the Anaconda installer starts correctly when booting from DVD.iso. diff --git a/QA/Testcase_Debranding.md b/QA/Testcase_Debranding.md index dcb69e5..360fccb 100644 --- a/QA/Testcase_Debranding.md +++ b/QA/Testcase_Debranding.md @@ -1,7 +1,7 @@ --- title: QA:Testcase Debranding author: Trevor Cooper -revision_date: 2022-04-20 +revision_date: 2022-05-18 rc: prod: Rocky Linux ver: 8 @@ -9,7 +9,7 @@ rc: --- !!! info "Associated release criterion" - This test case is associated with the [Release_Criteria - Debranding](../release_criteria.md#debranding) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + This test case is associated with the [Release_Criteria - Debranding](release_criteria.md#debranding) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. ## Description Add a short description here for this Testcase. diff --git a/QA/Testcase_Media_File_Conflicts.md b/QA/Testcase_Media_File_Conflicts.md index eded5ca..ddf2075 100644 --- a/QA/Testcase_Media_File_Conflicts.md +++ b/QA/Testcase_Media_File_Conflicts.md @@ -1,7 +1,7 @@ --- title: QA:Testcase Media File Conflicts author: Trevor Cooper -revision_date: 2022-04-24 +revision_date: 2022-05-18 rc: prod: Rocky Linux ver: 8 @@ -9,7 +9,7 @@ rc: --- !!! info "Associated release criterion" - This test case is associated with the [Release_Criteria#no-broken-packages](../release_criteria.md#no-broken-packages) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + This test case is associated with the [Release_Criteria#no-broken-packages](release_criteria.md#no-broken-packages) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. ## Description This testcase will verify that the offline repository included on release blocking images will not contain any file conflicts between packages without explicit `Conflicts:` tag in the package metadata. diff --git a/QA/Testcase_Media_Repoclosure.md b/QA/Testcase_Media_Repoclosure.md index 379cb97..91e4dc0 100644 --- a/QA/Testcase_Media_Repoclosure.md +++ b/QA/Testcase_Media_Repoclosure.md @@ -1,7 +1,7 @@ --- title: QA:Testcase Media Repoclosure author: Trevor Cooper -revision_date: 2022-04-24 +revision_date: 2022-05-18 rc: prod: Rocky Linux ver: 8 @@ -9,7 +9,7 @@ rc: --- !!! info "Associated release criterion" - This test case is associated with the [Release_Criteria#no-broken-packages](../release_criteria.md#no-broken-packages) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + This test case is associated with the [Release_Criteria#no-broken-packages](release_criteria.md#no-broken-packages) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. ## Description This testcase will verify that the offline repository included on release blocking images will not contain broken dependencies. diff --git a/QA/Testcase_Media_USB_dd.md b/QA/Testcase_Media_USB_dd.md index 0428200..73b3733 100644 --- a/QA/Testcase_Media_USB_dd.md +++ b/QA/Testcase_Media_USB_dd.md @@ -1,7 +1,7 @@ --- title: QA:Testcase Media USB dd author: Trevor Cooper -revision_date: 2021-04-21 +revision_date: 2022-05-18 rc: prod: Rocky Linux ver: 8 @@ -9,7 +9,7 @@ rc: --- !!! info "Associated release criterion" - This test case is associated with the [Release_Criteria#initialization-requirements](../release_criteria.md#initialization-requirements) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + This test case is associated with the [Release_Criteria#initialization-requirements](release_criteria.md#initialization-requirements) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. ## Description This verifies that Rocky Linux ISO image can be written to USB media using `dd` command, and the resulting USB media successfully boots to the Anaconda Installer. diff --git a/QA/Testcase_Packages_No_Insights.md b/QA/Testcase_Packages_No_Insights.md index 68858cc..80e9b9a 100644 --- a/QA/Testcase_Packages_No_Insights.md +++ b/QA/Testcase_Packages_No_Insights.md @@ -1,7 +1,7 @@ --- title: QA:Testcase Packages No Insights author: Trevor Cooper -revision_date: 2022-04-24 +revision_date: 2022-05-18 rc: prod: Rocky Linux ver: 8 @@ -9,7 +9,7 @@ rc: --- !!! info "Associated release criterion" - This test case is associated with the [Release_Criteria#repositories-must-match-upstream](../release_criteria.md#repositories-must-match-upstream) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + This test case is associated with the [Release_Criteria#repositories-must-match-upstream](release_criteria.md#repositories-must-match-upstream) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. ## Description This test will verify that `insights-client` package is not declared be installed as part of a package group. diff --git a/QA/Testcase_Packages_No_RHSM.md b/QA/Testcase_Packages_No_RHSM.md index 6fae64e..0e0ce31 100644 --- a/QA/Testcase_Packages_No_RHSM.md +++ b/QA/Testcase_Packages_No_RHSM.md @@ -1,7 +1,7 @@ --- title: QA:Testcase Packages No RHSM author: Trevor Cooper -revision_date: 2022-04-24 +revision_date: 2022-05-18 rc: prod: Rocky Linux ver: 8 @@ -9,7 +9,7 @@ rc: --- !!! info "Associated release criterion" - This test case is associated with the [Release_Criteria#repositories-must-match-upstream](../release_criteria.md#repositories-must-match-upstream) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + This test case is associated with the [Release_Criteria#repositories-must-match-upstream](release_criteria.md#repositories-must-match-upstream) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. ## Description This test will verify that packages that are availble from upstream do not have hard requirements on `subscription-manager` (RHSM). diff --git a/QA/Testcase_Repo_Compare.md b/QA/Testcase_Repo_Compare.md index e7c76e8..2b4a30e 100644 --- a/QA/Testcase_Repo_Compare.md +++ b/QA/Testcase_Repo_Compare.md @@ -1,7 +1,7 @@ --- title: QA:Testcase Media Repo Compare author: Trevor Cooper -revision_date: 2022-04-24 +revision_date: 2022-05-18 rc: prod: Rocky Linux ver: 8 @@ -9,7 +9,7 @@ rc: --- !!! info "Associated release criterion" - This test case is associated with the [Release_Criteria#repositories-must-match-upstream](../release_criteria.md#repositories-must-match-upstream) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + This test case is associated with the [Release_Criteria#repositories-must-match-upstream](release_criteria.md#repositories-must-match-upstream) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. ## Description This test case will verify that repositories and the packages within them match upstream as closely as possible. diff --git a/QA/Testcase_Template.md b/QA/Testcase_Template.md index 8fa1acf..9af0cdc 100644 --- a/QA/Testcase_Template.md +++ b/QA/Testcase_Template.md @@ -1,7 +1,7 @@ --- title: QA:Testcase Template author: Trevor Cooper -revision_date: 2022-04-01 +revision_date: 2022-05-18 rc: prod: Rocky Linux ver: 8 @@ -9,7 +9,7 @@ rc: --- !!! info "Associated release criterion" - This test case is associated with the [Release_Criteria#TBD](../release_criteria.md#TBD) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + This test case is associated with the [Release_Criteria#TBD](release_criteria.md#TBD) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. ## Description Add a short description here for this Testcase. diff --git a/release_criteria/.pages b/release_criteria/.pages index 6086810..313559f 100644 --- a/release_criteria/.pages +++ b/release_criteria/.pages @@ -1,5 +1,5 @@ --- nav: - - ... | index.md + - ... | release_criteria.md - Rocky Linux 8: r8 - Rocky Linux 9: r9 diff --git a/release_criteria/index.md b/release_criteria/release_criteria.md similarity index 100% rename from release_criteria/index.md rename to release_criteria/release_criteria.md From 822c6dcf1a913bd4a73dc6f2fbf7ecebfef9fa38 Mon Sep 17 00:00:00 2001 From: Trevor Cooper Date: Thu, 19 May 2022 00:08:02 -0700 Subject: [PATCH 17/45] point at versioned release criteria for latest release --- QA/Testcase_Basic_Graphics_Mode.md | 4 ++-- QA/Testcase_Boot_Methods_Boot_Iso.md | 2 +- QA/Testcase_Boot_Methods_Dvd.md | 2 +- QA/Testcase_Debranding.md | 2 +- QA/Testcase_Media_File_Conflicts.md | 2 +- QA/Testcase_Media_Repoclosure.md | 2 +- QA/Testcase_Media_USB_dd.md | 2 +- QA/Testcase_Packages_No_Insights.md | 2 +- QA/Testcase_Packages_No_RHSM.md | 2 +- QA/Testcase_Repo_Compare.md | 2 +- QA/Testcase_Template.md | 2 +- qa_test_cases.md | 12 ++++++------ 12 files changed, 18 insertions(+), 18 deletions(-) diff --git a/QA/Testcase_Basic_Graphics_Mode.md b/QA/Testcase_Basic_Graphics_Mode.md index a65e8f2..e344032 100644 --- a/QA/Testcase_Basic_Graphics_Mode.md +++ b/QA/Testcase_Basic_Graphics_Mode.md @@ -9,10 +9,10 @@ rc: --- !!! error "REFERENCED RELEASE CRITERIA IS OVERLY GENERAL AND UNTESTABLE" - The associated release criteria, [Release_Criteria#basic-graphics-mode-behaviors](release_criteria.md#basic-graphics-mode-behaviors), for this test case is overly general and **must** be modified to specific enough to be testable. + The associated release criteria, [Release_Criteria#basic-graphics-mode-behaviors](9_release_criteria.md#basic-graphics-mode-behaviors), for this test case is overly general and **must** be modified to specific enough to be testable. !!! info "Associated release criterion" - This test case is associated with the [Release_Criteria#basic-graphics-mode-behaviors](release_criteria.md#basic-graphics-mode-behaviors) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + This test case is associated with the [Release_Criteria#basic-graphics-mode-behaviors](9_release_criteria.md#basic-graphics-mode-behaviors) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. ## Description This test case will verify that release-blocking installers function as intended using the generic video driver option (“basic graphics mode”) on supported systems and classes of hardware. diff --git a/QA/Testcase_Boot_Methods_Boot_Iso.md b/QA/Testcase_Boot_Methods_Boot_Iso.md index 97d6417..77441ab 100644 --- a/QA/Testcase_Boot_Methods_Boot_Iso.md +++ b/QA/Testcase_Boot_Methods_Boot_Iso.md @@ -9,7 +9,7 @@ rc: --- !!! info "Associated release criterion" - This test case is associated with the [Release_Criteria#initialization-requirements](release_criteria.md#initialization-requirements) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + This test case is associated with the [Release_Criteria#initialization-requirements](9_release_criteria.md#initialization-requirements) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. ## Description This is to verify that the Anaconda installer starts correctly when booting from the Rocky Linux boot.iso. diff --git a/QA/Testcase_Boot_Methods_Dvd.md b/QA/Testcase_Boot_Methods_Dvd.md index 51af408..f242746 100644 --- a/QA/Testcase_Boot_Methods_Dvd.md +++ b/QA/Testcase_Boot_Methods_Dvd.md @@ -9,7 +9,7 @@ rc: --- !!! info "Associated release criterion" - This test case is associated with the [Release_Criteria#initialization-requirements](release_criteria.md#initialization-requirements) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + This test case is associated with the [Release_Criteria#initialization-requirements](9_release_criteria.md#initialization-requirements) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. ## Description This is to verify that the Anaconda installer starts correctly when booting from DVD.iso. diff --git a/QA/Testcase_Debranding.md b/QA/Testcase_Debranding.md index 360fccb..2474b67 100644 --- a/QA/Testcase_Debranding.md +++ b/QA/Testcase_Debranding.md @@ -9,7 +9,7 @@ rc: --- !!! info "Associated release criterion" - This test case is associated with the [Release_Criteria - Debranding](release_criteria.md#debranding) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + This test case is associated with the [Release_Criteria - Debranding](9_release_criteria.md#debranding) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. ## Description Add a short description here for this Testcase. diff --git a/QA/Testcase_Media_File_Conflicts.md b/QA/Testcase_Media_File_Conflicts.md index ddf2075..8164285 100644 --- a/QA/Testcase_Media_File_Conflicts.md +++ b/QA/Testcase_Media_File_Conflicts.md @@ -9,7 +9,7 @@ rc: --- !!! info "Associated release criterion" - This test case is associated with the [Release_Criteria#no-broken-packages](release_criteria.md#no-broken-packages) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + This test case is associated with the [Release_Criteria#no-broken-packages](9_release_criteria.md#no-broken-packages) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. ## Description This testcase will verify that the offline repository included on release blocking images will not contain any file conflicts between packages without explicit `Conflicts:` tag in the package metadata. diff --git a/QA/Testcase_Media_Repoclosure.md b/QA/Testcase_Media_Repoclosure.md index 91e4dc0..8f7224b 100644 --- a/QA/Testcase_Media_Repoclosure.md +++ b/QA/Testcase_Media_Repoclosure.md @@ -9,7 +9,7 @@ rc: --- !!! info "Associated release criterion" - This test case is associated with the [Release_Criteria#no-broken-packages](release_criteria.md#no-broken-packages) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + This test case is associated with the [Release_Criteria#no-broken-packages](9_release_criteria.md#no-broken-packages) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. ## Description This testcase will verify that the offline repository included on release blocking images will not contain broken dependencies. diff --git a/QA/Testcase_Media_USB_dd.md b/QA/Testcase_Media_USB_dd.md index 73b3733..77d2e32 100644 --- a/QA/Testcase_Media_USB_dd.md +++ b/QA/Testcase_Media_USB_dd.md @@ -9,7 +9,7 @@ rc: --- !!! info "Associated release criterion" - This test case is associated with the [Release_Criteria#initialization-requirements](release_criteria.md#initialization-requirements) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + This test case is associated with the [Release_Criteria#initialization-requirements](9_release_criteria.md#initialization-requirements) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. ## Description This verifies that Rocky Linux ISO image can be written to USB media using `dd` command, and the resulting USB media successfully boots to the Anaconda Installer. diff --git a/QA/Testcase_Packages_No_Insights.md b/QA/Testcase_Packages_No_Insights.md index 80e9b9a..a59fe93 100644 --- a/QA/Testcase_Packages_No_Insights.md +++ b/QA/Testcase_Packages_No_Insights.md @@ -9,7 +9,7 @@ rc: --- !!! info "Associated release criterion" - This test case is associated with the [Release_Criteria#repositories-must-match-upstream](release_criteria.md#repositories-must-match-upstream) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + This test case is associated with the [Release_Criteria#repositories-must-match-upstream](9_release_criteria.md#repositories-must-match-upstream) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. ## Description This test will verify that `insights-client` package is not declared be installed as part of a package group. diff --git a/QA/Testcase_Packages_No_RHSM.md b/QA/Testcase_Packages_No_RHSM.md index 0e0ce31..3d66e1c 100644 --- a/QA/Testcase_Packages_No_RHSM.md +++ b/QA/Testcase_Packages_No_RHSM.md @@ -9,7 +9,7 @@ rc: --- !!! info "Associated release criterion" - This test case is associated with the [Release_Criteria#repositories-must-match-upstream](release_criteria.md#repositories-must-match-upstream) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + This test case is associated with the [Release_Criteria#repositories-must-match-upstream](9_release_criteria.md#repositories-must-match-upstream) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. ## Description This test will verify that packages that are availble from upstream do not have hard requirements on `subscription-manager` (RHSM). diff --git a/QA/Testcase_Repo_Compare.md b/QA/Testcase_Repo_Compare.md index 2b4a30e..4136a8c 100644 --- a/QA/Testcase_Repo_Compare.md +++ b/QA/Testcase_Repo_Compare.md @@ -9,7 +9,7 @@ rc: --- !!! info "Associated release criterion" - This test case is associated with the [Release_Criteria#repositories-must-match-upstream](release_criteria.md#repositories-must-match-upstream) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + This test case is associated with the [Release_Criteria#repositories-must-match-upstream](9_release_criteria.md#repositories-must-match-upstream) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. ## Description This test case will verify that repositories and the packages within them match upstream as closely as possible. diff --git a/QA/Testcase_Template.md b/QA/Testcase_Template.md index 9af0cdc..4e65294 100644 --- a/QA/Testcase_Template.md +++ b/QA/Testcase_Template.md @@ -9,7 +9,7 @@ rc: --- !!! info "Associated release criterion" - This test case is associated with the [Release_Criteria#TBD](release_criteria.md#TBD) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + This test case is associated with the [Release_Criteria#TBD](9_release_criteria.md#TBD) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. ## Description Add a short description here for this Testcase. diff --git a/qa_test_cases.md b/qa_test_cases.md index 2b48db8..fb7dfb8 100644 --- a/qa_test_cases.md +++ b/qa_test_cases.md @@ -9,12 +9,12 @@ This page lists all test cases in work and who is working on them... | Requirement | Test Case | Assignee | Status | | --------------------------------------------------- | ------------------------------------------------------------------------ | ----------------------- | --------------------------------------- | -| [Release-blocking images must boot](release_criteria.md#release-blocking-images-must-boot) | [QA:Testcase Boot Methods Boot ISO](Testcase_Boot_Methods_Boot_Iso.md) | @tcooper | template exists, openQA covered (ref) | -| [Release-blocking images must boot](release_criteria.md#release-blocking-images-must-boot) | [QA:Testcase Boot Methods DVD](Testcase_Boot_Methods_Dvd.md) | @tcooper | template exists, openQA covered (ref) | -| [Basic Graphics Mode behaviors](release_criteria.md#basic-graphics-mode-behaviors) | [QA:Testcase Basic Graphics Mode](Testcase_Basic_Graphics_Mode.md) | @tcooper | manual and/or new openQA TestCase | -| [No Broken Packages](release_criteria.md#no-broken-packages) | [QA:Testcase Media Repoclosure](Testcase_Media_Repoclosure.md)
[QA:Testcase Media File Conflicts](Testcase_Media_File_Conflicts.md) | @tcooper | manual using scripts or automated in CI | -| [Repositories Must Match Upstream](release_criteria.md#repositories-must-match-upstream) | [QA:Testcase repocompare](Testcase_Repo_Compare.md) | @tcooper | manual using Skip's repocompare | -| [Debranding](release_criteria.md#debranding) | [QA:Testcase Debranding Analysis](Testcase_Debranding.md) | @tcooper | process TBD | +| [Release-blocking images must boot](9_release_criteria.md#release-blocking-images-must-boot) | [QA:Testcase Boot Methods Boot ISO](Testcase_Boot_Methods_Boot_Iso.md) | @tcooper | template exists, openQA covered (ref) | +| [Release-blocking images must boot](9_release_criteria.md#release-blocking-images-must-boot) | [QA:Testcase Boot Methods DVD](Testcase_Boot_Methods_Dvd.md) | @tcooper | template exists, openQA covered (ref) | +| [Basic Graphics Mode behaviors](9_release_criteria.md#basic-graphics-mode-behaviors) | [QA:Testcase Basic Graphics Mode](Testcase_Basic_Graphics_Mode.md) | @tcooper | manual and/or new openQA TestCase | +| [No Broken Packages](9_release_criteria.md#no-broken-packages) | [QA:Testcase Media Repoclosure](Testcase_Media_Repoclosure.md)
[QA:Testcase Media File Conflicts](Testcase_Media_File_Conflicts.md) | @tcooper | manual using scripts or automated in CI | +| [Repositories Must Match Upstream](9_release_criteria.md#repositories-must-match-upstream) | [QA:Testcase repocompare](Testcase_Repo_Compare.md) | @tcooper | manual using Skip's repocompare | +| [Debranding](9_release_criteria.md#debranding) | [QA:Testcase Debranding Analysis](Testcase_Debranding.md) | @tcooper | process TBD | ## Installer Requirements From 2d10e13f3ef3f96a46dc8f79509f8358525fae97 Mon Sep 17 00:00:00 2001 From: Trevor Cooper Date: Thu, 19 May 2022 00:12:19 -0700 Subject: [PATCH 18/45] minor changes to the Testing Team landing page --- .pages | 2 +- index.md | 13 ++++++++----- 2 files changed, 9 insertions(+), 6 deletions(-) diff --git a/.pages b/.pages index c6fae80..5e36c63 100644 --- a/.pages +++ b/.pages @@ -1,5 +1,5 @@ --- nav: - - Team Members: index.md + - Testing Team: index.md - QA:Test Cases: qa_test_cases.md - Release Criteria & Status: release_criteria diff --git a/index.md b/index.md index 8c31f9f..722a421 100644 --- a/index.md +++ b/index.md @@ -1,12 +1,11 @@ --- -title: Testing +title: QA Testing Team revision_date: 2022-04-01 +rc: + prod: Rocky Linux --- -This section is primarily for documentation for the Testing Team. The testing team handles testing and QA for Rocky Linux. - -## Members - +## Team Members | Role | Name | Email | Mattermost Name | IRC Name | | -------------- | --------------- | ----------------------- | ------------------ | --------- | @@ -16,4 +15,8 @@ This section is primarily for documentation for the Testing Team. The testing te | Testing Team | Lukas Magauer | lukas@magauer.eu | @lumarel | | | Testing Team | Alan Marshall | | @alangm | alangm | +## Responsibilities + +The Testing Team handles testing and QA for Rocky Linux. + {% include 'content_bottom.md' %} From c17f120d80cacf006b0d2376d1e7142481b97fb6 Mon Sep 17 00:00:00 2001 From: Trevor Cooper Date: Thu, 19 May 2022 19:43:51 -0700 Subject: [PATCH 19/45] docs: update testcase_debranding (#30) * general testcase_debranding process * minor wording update Authors: @tcooper, @akatch --- QA/Testcase_Debranding.md | 186 ++++++++++++++++++++++++++++++++++++-- 1 file changed, 179 insertions(+), 7 deletions(-) diff --git a/QA/Testcase_Debranding.md b/QA/Testcase_Debranding.md index dcb69e5..f42fadd 100644 --- a/QA/Testcase_Debranding.md +++ b/QA/Testcase_Debranding.md @@ -1,7 +1,7 @@ --- title: QA:Testcase Debranding author: Trevor Cooper -revision_date: 2022-04-20 +revision_date: 2022-05-11 rc: prod: Rocky Linux ver: 8 @@ -12,17 +12,189 @@ rc: This test case is associated with the [Release_Criteria - Debranding](../release_criteria.md#debranding) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. ## Description -Add a short description here for this Testcase. +The Rocky Linux [Release Engineering Team](/team/release_engineering/) builds and maintains tools to manage the debranding of packages received from the upstream vendor. They have published a comprehensive [debranding guide](/team/release_engineering/debranding/) and maintain a [list of packages](https://git.rockylinux.org/rocky/metadata/-/blob/main/patch.yml) requiring debranding patches. + +This testcase will verify that all packages available on released media that Rocky Linux Release Engineering has identified as requiring debranding are debranded successfully per their specification. ## Setup -1. Add steps for setup for this Testcase. +1. Obtain access to an environment with the `dnf`, and `koji` commands and access to [Rocky Linux Gitlab](https://git.rockylinux.org) and [Rocky Linux Koji](https://koji.rockylinux.org) +2. Download the ISO to be tested to test machine. +3. Configure `/etc/koji.conf` to access the [Rocky Linux Koji](https://koji.rockylinux.org). +4. Download a recent copy the [patch.yml](https://git.rockylinux.org/rocky/metadata/-/blob/main/patch.yml) from [Rocky Linux Gitlab](https://git.rockylinux.org). + +!!! info "patch.yml" + Packages listed in `patch.yml` are names of source RPMs. Binary RPMs containing content produced by building the patched source RPMs need to be validated. The easiest way to get the list of all possible binary RPMs for a particular package and arch is to ask obtain that information in koji. ## How to test -1. Do this first... -2. Then do this... +1. Mount the ISO to be tested locally. + - Example: + ``` + $ mount -o loop Rocky-8.5-x86_64-dvd1.iso /media + ``` +2. Determine the path(s) to the `repodata` directory(ies) on the ISO. + - Example: + ``` + $ find /media -name repodata + ``` +3. For each package to be validated get the names of the `noarch` and `` specific packages created from it. + - Example: + ``` + $ koji --quiet latest-build --arch=x86_64 dist-rocky8-compose + $ koji --quiet latest-build --arch=noarch dist-rocky8-compose + ``` +4. Use `dnf` to obtain the paths to the binary packages requiring debranding. + - Example: + ``` + $ dnf download --urls --repofrompath BaseOS,/media/BaseOS --repo BaseOS \ + --repofrompath Minimal,/media/Minimal --repo Minimal \ + + ``` +5. Copy the `` from the media and examine it's metadata and/or contents to determine if it has obviously been patched. + - Example: + ``` + $ rpm -q --changelog -p | head | \ + grep "Release Engineering " -C2 | \ + grep -Eq "" + + $ rpm2cpio | + cpio --quiet --extract --to-stdout . | \ + grep -Eq "" + ``` + + !!! info "NOTE" + Note all debranding patches will patch files directly and leave very obvious traces, some patches don't even add changelog messages to use as an indicator that the package has been patched or debranded. Sometimes the only solution is to extract the binary package and examine the contents directly to find something to test. + +6. Unmount the ISO. + - Example: + ``` + $ umount /media + ``` ## Expected Results -1. This is what you should see/verify. -2. You should also see/verify this. +1. Packages tracked by Release Engineering as requiring debranding and published on installation media are, in fact, debranded per their specification. + +

Sample Output

+ +=== "Success" + + ``` + $ sudo mount -o loop Rocky-8.5-aarch64-minimal.iso /media + mount: /media: WARNING: device write-protected, mounted read-only. + + $ find /media -name repodata + /media/BaseOS/repodata + /media/Minimal/repodata + + $ curl -LOR https://git.rockylinux.org/rocky/metadata/-/raw/main/patch.yml + % Total % Received % Xferd Average Speed Time Time Time Current + Dload Upload Total Spent Left Speed + 100 3410 100 3410 0 0 20419 0 --:--:-- --:--:-- --:--:-- 20419 + + $ yq .debrand.all[] patch.yml | column -x -c 100 -o " " + abrt anaconda anaconda-user-help chrony + cloud-init cockpit crash dhcp + dnf firefox fwupd gcc + gcc-toolset-9-gcc gcc-toolset-10-gcc gcc-toolset-11-gcc gcc-toolset-12-gcc + gnome-settings-daemon grub2 httpd initial-setup + kernel kernel-rt libdnf libreoffice + libreport nginx opa-ff opa-fm + openscap pesign PackageKit python-pip + python3 redhat-rpm-config scap-security-guide shim + shim-unsigned-x64 shim-unsigned-aarch64 sos subscription-manager + systemd thunderbird WALinuxAgent + + $ ./yq .debrand.r8[] patch.yml | column -x -c 100 -o " " + dotnet3.0 fwupdate gnome-boxes libguestfs pcs plymouth + python2 + + NOTE: Only a single package will be shown in this Example. + + $ koji --quiet latest-build --arch=x86_64 dist-rocky8-compose sos + + $ koji --quiet latest-build --arch=noarch dist-rocky8-compose sos + sos-4.1-9.el8_5.rocky.3.noarch + sos-audit-4.1-9.el8_5.rocky.3.noarch + + $ dnf download --urls --repofrompath BaseOS,/media/BaseOS --repo BaseOS \ + --repofrompath Minimal,/media/Minimal --repo Minimal \ + sos sos-audit | grep -E "^file" + file:///media/BaseOS/Packages/s/sos-4.1-5.el8.noarch.rpm + file:///media/BaseOS/Packages/s/sos-audit-4.1-5.el8.noarch.rpm + + $ rpm -q --changelog -p /media/BaseOS/Packages/s/sos-4.1-5.el8.noarch.rpm | \ + head | grep "Release Engineering " -C2 + * Mon Oct 18 2021 Release Engineering - 4.1-5 + - Remove Red Hat branding from sos + $ echo $? + 0 + + $ rpm -q --changelog -p /media/BaseOS/Packages/s/sos-audit-4.1-5.el8.noarch.rpm | \ + head | grep "Release Engineering " -C2 + * Mon Oct 18 2021 Release Engineering - 4.1-5 + - Remove Red Hat branding from sos + $ echo $? + 0 + + $ umount /media + ``` + +=== "Failure" + + ``` + $ sudo mount -o loop Rocky-8.5-aarch64-minimal.iso /media + mount: /media: WARNING: device write-protected, mounted read-only. + + $ find /media -name repodata + /media/BaseOS/repodata + /media/Minimal/repodata + + $ curl -LOR https://git.rockylinux.org/rocky/metadata/-/raw/main/patch.yml + % Total % Received % Xferd Average Speed Time Time Time Current + Dload Upload Total Spent Left Speed + 100 3410 100 3410 0 0 20419 0 --:--:-- --:--:-- --:--:-- 20419 + + $ yq .debrand.all[] patch.yml | column -x -c 100 -o " " + abrt anaconda anaconda-user-help chrony + cloud-init cockpit crash dhcp + dnf firefox fwupd gcc + gcc-toolset-9-gcc gcc-toolset-10-gcc gcc-toolset-11-gcc gcc-toolset-12-gcc + gnome-settings-daemon grub2 httpd initial-setup + kernel kernel-rt libdnf libreoffice + libreport nginx opa-ff opa-fm + openscap pesign PackageKit python-pip + python3 redhat-rpm-config scap-security-guide shim + shim-unsigned-x64 shim-unsigned-aarch64 sos subscription-manager + systemd thunderbird WALinuxAgent + + $ ./yq .debrand.r8[] patch.yml | column -x -c 100 -o " " + dotnet3.0 fwupdate gnome-boxes libguestfs pcs plymouth + python2 + + NOTE: Only a single package will be shown in this Example. + + $ koji --quiet latest-build --arch=x86_64 dist-rocky8-compose sos + + $ koji --quiet latest-build --arch=noarch dist-rocky8-compose sos + sos-4.1-9.el8_5.rocky.3.noarch + sos-audit-4.1-9.el8_5.rocky.3.noarch + + $ dnf download --urls --repofrompath BaseOS,/media/BaseOS --repo BaseOS \ + --repofrompath Minimal,/media/Minimal --repo Minimal \ + sos sos-audit | grep -E "^file" + file:///media/BaseOS/Packages/s/sos-4.1-5.el8.noarch.rpm + file:///media/BaseOS/Packages/s/sos-audit-4.1-5.el8.noarch.rpm + + $ rpm -q --changelog -p /media/BaseOS/Packages/s/sos-4.1-5.el8.noarch.rpm | \ + head | grep "Release Engineering " -C2 + $ echo $? + 1 + + $ rpm -q --changelog -p /media/BaseOS/Packages/s/sos-audit-4.1-5.el8.noarch.rpm | \ + head | grep "Release Engineering " -C2 + $ echo $? + 1 + + $ umount /media + ``` {% include 'testing/qa_testcase_bottom.md' %} From 0fdf9eef3136de8ae5551c8d18679ca61c3e3d5c Mon Sep 17 00:00:00 2001 From: Trevor Cooper Date: Thu, 19 May 2022 20:41:59 -0700 Subject: [PATCH 20/45] feat: linkage to both r8 and r9 release criteria - Requirements may exist for more than one concurrently supported version of Rocky Linux - Prepare for possibility of macro-tization of the generation of Requirement linkage via TBD mechanism --- qa_test_cases.md | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/qa_test_cases.md b/qa_test_cases.md index fb7dfb8..1d0782e 100644 --- a/qa_test_cases.md +++ b/qa_test_cases.md @@ -1,6 +1,8 @@ --- title: QA:Test Cases -revision_date: 2022-05-06 +revision_date: 2022-05-19 +rc: + prod: Rocky Linux --- This page lists all test cases in work and who is working on them... @@ -9,12 +11,12 @@ This page lists all test cases in work and who is working on them... | Requirement | Test Case | Assignee | Status | | --------------------------------------------------- | ------------------------------------------------------------------------ | ----------------------- | --------------------------------------- | -| [Release-blocking images must boot](9_release_criteria.md#release-blocking-images-must-boot) | [QA:Testcase Boot Methods Boot ISO](Testcase_Boot_Methods_Boot_Iso.md) | @tcooper | template exists, openQA covered (ref) | -| [Release-blocking images must boot](9_release_criteria.md#release-blocking-images-must-boot) | [QA:Testcase Boot Methods DVD](Testcase_Boot_Methods_Dvd.md) | @tcooper | template exists, openQA covered (ref) | -| [Basic Graphics Mode behaviors](9_release_criteria.md#basic-graphics-mode-behaviors) | [QA:Testcase Basic Graphics Mode](Testcase_Basic_Graphics_Mode.md) | @tcooper | manual and/or new openQA TestCase | -| [No Broken Packages](9_release_criteria.md#no-broken-packages) | [QA:Testcase Media Repoclosure](Testcase_Media_Repoclosure.md)
[QA:Testcase Media File Conflicts](Testcase_Media_File_Conflicts.md) | @tcooper | manual using scripts or automated in CI | -| [Repositories Must Match Upstream](9_release_criteria.md#repositories-must-match-upstream) | [QA:Testcase repocompare](Testcase_Repo_Compare.md) | @tcooper | manual using Skip's repocompare | -| [Debranding](9_release_criteria.md#debranding) | [QA:Testcase Debranding Analysis](Testcase_Debranding.md) | @tcooper | process TBD | +| Release-blocking images must boot
[{{ rc.prod }} 8](8_release_criteria.md#release-blocking-images-must-boot) [{{ rc.prod }} 9](9_release_criteria.md#release-blocking-images-must-boot) | [QA:Testcase Boot Methods Boot ISO](Testcase_Boot_Methods_Boot_Iso.md) | @tcooper | template exists, openQA covered (ref) | +| Release-blocking images must boot
[{{ rc.prod }} 8](8_release_criteria.md#release-blocking-images-must-boot) [{{ rc.prod }} 9](9_release_criteria.md#release-blocking-images-must-boot) | [QA:Testcase Boot Methods DVD](Testcase_Boot_Methods_Dvd.md) | @tcooper | template exists, openQA covered (ref) | +| Basic Graphics Mode behaviors
[{{ rc.prod }} 8](8_release_criteria.md#basic-graphics-mode-behaviors) [{{ rc.prod }} 9](9_release_criteria.md#basic-graphics-mode-behaviors) | [QA:Testcase Basic Graphics Mode](Testcase_Basic_Graphics_Mode.md) | @tcooper | manual and/or new openQA TestCase | +| No Broken Packages
[{{ rc.prod }} 8](8_release_criteria.md#no-broken-packages) [{{ rc.prod }} 9](9_release_criteria.md#no-broken-packages) | [QA:Testcase Media Repoclosure](Testcase_Media_Repoclosure.md)
[QA:Testcase Media File Conflicts](Testcase_Media_File_Conflicts.md) | @tcooper | manual using scripts or automated in CI | +| Repositories Must Match Upstream
[{{ rc.prod }} 8](8_release_criteria.md#repositories-must-match-upstream) [{{ rc.prod }} 9 ](9_release_criteria.md#repositories-must-match-upstream) | [QA:Testcase repocompare](Testcase_Repo_Compare.md) | @tcooper | manual using Skip's repocompare | +| Debranding
[{{ rc.prod }} 8](8_release_criteria.md#debranding) [{{ rc.prod }} 9](9_release_criteria.md#debranding) | [QA:Testcase Debranding Analysis](Testcase_Debranding.md) | @tcooper | process TBD | ## Installer Requirements From f18d831ee4802560cc0a61f946732504c7b78c8f Mon Sep 17 00:00:00 2001 From: Trevor Cooper Date: Sun, 22 May 2022 12:29:45 -0700 Subject: [PATCH 21/45] fix link to wrong page Thanks @akatch --- release_criteria/release_criteria.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/release_criteria/release_criteria.md b/release_criteria/release_criteria.md index 3c00274..f9401ad 100644 --- a/release_criteria/release_criteria.md +++ b/release_criteria/release_criteria.md @@ -1,7 +1,7 @@ --- title: Rocky Linux Release Criteria & Status author: Trevor Cooper -revision_date: 2022-05-18 +revision_date: 2022-05-22 rc: prod: Rocky Linux --- @@ -13,7 +13,7 @@ The QA and Testing efforts during releases are tracked in online shared document | Rocky Linux Version | Release Criteria | QA and Testing Status | QA and Testing GO / NO-GO Summary | Official Release Date | | -------------------- | --------------------------------------- |--------------------------------------- | ---------------------------------------- | --------------------------- | | {{ rc.prod }} 8.5 | not available | not available | not available | NOVEMBER 15, 2021 | -| {{ rc.prod }} 8.6 | [AVAILABLE](9.0_qa_testing_summary.md) | [AVAILABLE](8.6_qa_testing_summary.md) | [AVAILABLE](8.6_qa_testing_go_no_go.md) | MAY 16, 2022 | +| {{ rc.prod }} 8.6 | [AVAILABLE](8.6_qa_testing_summary.md) | [AVAILABLE](8.6_qa_testing_summary.md) | [AVAILABLE](8.6_qa_testing_go_no_go.md) | MAY 16, 2022 | | {{ rc.prod }} 9.0 | [AVAILABLE](9.0_qa_testing_summary.md) | [AVAILABLE](9.0_qa_testing_summary.md) | [AVAILABLE](9.0_qa_testing_go_no_go.md) | TBD | {% include 'content_bottom.md' %} From b4c0ad62dd87f8d85c43a764495a5530b03ffb34 Mon Sep 17 00:00:00 2001 From: Trevor Cooper Date: Sun, 22 May 2022 12:30:13 -0700 Subject: [PATCH 22/45] fix copy/paste error Thanks @akatch --- release_criteria/.qa_testing_summary_template.md | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/release_criteria/.qa_testing_summary_template.md b/release_criteria/.qa_testing_summary_template.md index 28abe0b..de13a8f 100644 --- a/release_criteria/.qa_testing_summary_template.md +++ b/release_criteria/.qa_testing_summary_template.md @@ -1,7 +1,7 @@ --- title: Release Status Template author: Trevor Cooper -revision_date: 2022-05-18 +revision_date: 2022-05-22 rc: prod: Rocky Linux ver: 9 @@ -34,9 +34,6 @@ PLEASE add your MM handle if you are working on this item to minimize duplicatio - QA:Testcase Debranding - [INVESTIGATE] - @tcooper - QA:Testcase Media Consistency Verification - [INVESTIGATE] - @tcooper - QA:Testcase Media File Conflicts - [INVESTIGATE] - @tcooper -- QA:Testcase Media File Conflicts - [INVESTIGATE] - @tcooper -- QA:Testcase Media File Conflicts - [INVESTIGATE] - @tcooper -- QA:Testcase Media File Conflicts - [INVESTIGATE] - @tcooper - QA:Testcase Media Repoclosure - [INVESTIGATE] - @tcooper - QA:Testcase Storage Volume Resize - [INVESTIGATE] - @raktajino - QA:Testcase Update Image - [INVESTIGATE] - @raktajino From 02a06ef2b5d1e109344314e9b8be969cecb0fb73 Mon Sep 17 00:00:00 2001 From: Al Bowles Date: Mon, 13 Jun 2022 17:42:04 -0500 Subject: [PATCH 23/45] feat: GPG keypair generation and signing documentation --- .pages | 1 + dev_guides/.pages | 4 + dev_guides/commit_signing.md | 99 +++++++++++++++++++ .../wiki_development_boxes.md | 0 4 files changed, 104 insertions(+) create mode 100644 dev_guides/.pages create mode 100644 dev_guides/commit_signing.md rename wiki_development_boxes.md => dev_guides/wiki_development_boxes.md (100%) diff --git a/.pages b/.pages index 5e36c63..8a0ca5f 100644 --- a/.pages +++ b/.pages @@ -3,3 +3,4 @@ nav: - Testing Team: index.md - QA:Test Cases: qa_test_cases.md - Release Criteria & Status: release_criteria + - Wiki Development Guides: dev_guides diff --git a/dev_guides/.pages b/dev_guides/.pages new file mode 100644 index 0000000..52fe553 --- /dev/null +++ b/dev_guides/.pages @@ -0,0 +1,4 @@ +--- +nav: + - Development Box Setup: development_boxes.md + - Git Commit Signing: commit_signing.md diff --git a/dev_guides/commit_signing.md b/dev_guides/commit_signing.md new file mode 100644 index 0000000..eef8be8 --- /dev/null +++ b/dev_guides/commit_signing.md @@ -0,0 +1,99 @@ +--- +title: Signing Commits with GPG +author: Al Bowles +revision_date: 2022-06-13 +rc: + prod: Rocky Linux + ver: 8 + level: Final +--- +# Creating your primary keypair +Create a new gpg keypair, ideally set to expire in <= 1y + + gpg --full-generate-key --expert + +Select eddsa [ECC] and set a validity period +Specify real name and email address to associate with this keypair +Type a passphrase + +# Create a signing keypair +Add a signing subkey + + gpg --edit-key my@email.addr + gpg> addkey + [ passphrase ] + +Select [ECC] (sign / authenticate / encrypt?) for kind of key, 4096 bits, valid for 180d + + gpg> save + +Create revocation certificate + + gpg --output \.gpg-revocation-certificate --gen-revoke my@email.addr + +# Back up your keypair +Export the *primary keypair* (put these somewhere very safe along with revocation certificate) + + gpg --export-secret-keys --armor my@email.addr > \.private.gpg-key + gpg --export --armor my@email.addr > \.public.gpg-key + +# Remove the *primary keypair* from your keyring +Export all subkeys from the new keypair to a file - use ramfs instead of tmpfs/ or /dev/shm/ because ramfs doesn't write to swap + + mkdir /tmp/gpg + sudo mount -t ramfs -o size=1M ramfs /tmp/gpg + sudo chown $(logname):$(logname) /tmp/gpg + gpg --export-secret-subkeys my@email.addr > /tmp/gpg/subkeys + +Delete original signing subkey from keypair in our keyring + + gpg --delete-secret-key my@email.addr + +Re-import the previously exported keys + + gpg --import /tmp/gpg/subkeys + sudo umount /tmp/gpg + rmdir /tmp/gpg + +Look for `sec#` instead of `sec` in the output - pound sign means signing subkey is *not* in the keypair located in the keyring + gpg --list-secret-keys $HOME/.gnupg/secring.gpg + +# Revoking a *signing keypair* +Find the *primary keypair* and import it (preferably into an ephemeral system like a liveUSB) + + gpg --import /path/to/\.public.gpg-key /path/to/\.private.gpg-key + gpg --edit-key my@email.addr + gpg> revkey + [ passphrase twice ] + gpg> save + + +# Renew an expired or expiring keypair + + gpg --edit-key my@email.addr + [select a key] + gpg> expire + [specify an expiration] + gpg> save + +# Create a single signed git commit + + git commit -S -m "my awesome signed commit" + +# Configure git to always sign commits with a specified key + + $ gpg --list-secret-keys --keyid-format=long # grab the fingerprint from the 'sec' line + git config [--global] commit.gpgsign true + git config [--global] user.signingkey DEADB33FBAD1D3A + +# Configure VSCode to sign commits + + # User or workspace setting + "git.enableCommitSigning": true + +# References +[OpenPGP Best Practices](https://riseup.net/en/security/message-security/openpgp/best-practices#key-configuration)
+[Github: Signing Commits](https://docs.github.com/en/enterprise-server@3.5/authentication/managing-commit-signature-verification/signing-commits)
+[Braincoke's Log: Create a GPG Key](https://blog.braincoke.fr/security/create-a-gpg-key/)
+[Creating the Perfect GPG Keypair](https://alexcabal.com/creating-the-perfect-gpg-keypair)
+[Digital Neanderthal: Generate GPG Keys With Curve Ed25519](https://www.digitalneanderthal.com/post/gpg/)
diff --git a/wiki_development_boxes.md b/dev_guides/wiki_development_boxes.md similarity index 100% rename from wiki_development_boxes.md rename to dev_guides/wiki_development_boxes.md From 0cf5e3c9400907dcb2df3592352448a7bfdbca0a Mon Sep 17 00:00:00 2001 From: Al Bowles Date: Mon, 13 Jun 2022 17:55:39 -0500 Subject: [PATCH 24/45] Clarify documentation --- dev_guides/commit_signing.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/dev_guides/commit_signing.md b/dev_guides/commit_signing.md index eef8be8..39a90f0 100644 --- a/dev_guides/commit_signing.md +++ b/dev_guides/commit_signing.md @@ -8,13 +8,15 @@ rc: level: Final --- # Creating your primary keypair -Create a new gpg keypair, ideally set to expire in <= 1y +Initiate the keypair generation wizard gpg --full-generate-key --expert -Select eddsa [ECC] and set a validity period +Select option `(9) ECC and ECC` for the key type +Select option `(1) Curve 25519` for the elliptic curve +Set a validity period of your choice, ideally less than 1 year Specify real name and email address to associate with this keypair -Type a passphrase +Type a passphrase (twice) # Create a signing keypair Add a signing subkey From e3585ee29f4e382870d31701edce660a60a4a008 Mon Sep 17 00:00:00 2001 From: Al Bowles Date: Mon, 13 Jun 2022 17:57:25 -0500 Subject: [PATCH 25/45] Specifics around email address --- dev_guides/commit_signing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/dev_guides/commit_signing.md b/dev_guides/commit_signing.md index 39a90f0..0f58886 100644 --- a/dev_guides/commit_signing.md +++ b/dev_guides/commit_signing.md @@ -15,7 +15,7 @@ Initiate the keypair generation wizard Select option `(9) ECC and ECC` for the key type Select option `(1) Curve 25519` for the elliptic curve Set a validity period of your choice, ideally less than 1 year -Specify real name and email address to associate with this keypair +Specify real name and email address to associate with this keypair. The email address must match your verified Github email address or be set to `your-github-username@users.noreply.github.com`. Type a passphrase (twice) # Create a signing keypair From 66911894b4a301b3ce4022721adefd886e3a867f Mon Sep 17 00:00:00 2001 From: Al Bowles Date: Mon, 13 Jun 2022 18:02:06 -0500 Subject: [PATCH 26/45] Fix formatting --- dev_guides/commit_signing.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/dev_guides/commit_signing.md b/dev_guides/commit_signing.md index 0f58886..58b7973 100644 --- a/dev_guides/commit_signing.md +++ b/dev_guides/commit_signing.md @@ -8,15 +8,15 @@ rc: level: Final --- # Creating your primary keypair -Initiate the keypair generation wizard +1. Initiate the keypair generation wizard - gpg --full-generate-key --expert + gpg --full-generate-key --expert -Select option `(9) ECC and ECC` for the key type -Select option `(1) Curve 25519` for the elliptic curve -Set a validity period of your choice, ideally less than 1 year -Specify real name and email address to associate with this keypair. The email address must match your verified Github email address or be set to `your-github-username@users.noreply.github.com`. -Type a passphrase (twice) +1. Select option `(9) ECC and ECC` for the key type +1. Select option `(1) Curve 25519` for the elliptic curve +1. Set a validity period of your choice, ideally less than 1 year +1. Specify real name and email address to associate with this keypair. The email address must match your verified Github email address or be set to `your-github-username@users.noreply.github.com`. +1. Type a passphrase (twice) # Create a signing keypair Add a signing subkey From b1be795d047e9aa7513b2a03c21cd2d44338c89d Mon Sep 17 00:00:00 2001 From: Al Bowles Date: Mon, 13 Jun 2022 18:24:55 -0500 Subject: [PATCH 27/45] Clarify instructions for signing subkey --- dev_guides/commit_signing.md | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/dev_guides/commit_signing.md b/dev_guides/commit_signing.md index 58b7973..5f94ee6 100644 --- a/dev_guides/commit_signing.md +++ b/dev_guides/commit_signing.md @@ -19,17 +19,20 @@ rc: 1. Type a passphrase (twice) # Create a signing keypair -Add a signing subkey +1. Add a signing subkey - gpg --edit-key my@email.addr - gpg> addkey - [ passphrase ] + gpg --expert --edit-key my@email.addr + gpg> addkey -Select [ECC] (sign / authenticate / encrypt?) for kind of key, 4096 bits, valid for 180d +1. Select option `(10) ECC (sign only)` for the key type +1. Select option `(1) Curve 25519` for the elliptic curve +1. Set a validity period of your choice, ideally less than 1 year +1. Accept the prompts and type a passphrase (twice) +1. Save and exit - gpg> save + gpg> save -Create revocation certificate +# Create revocation certificate gpg --output \.gpg-revocation-certificate --gen-revoke my@email.addr From ef6a54afa54d79cbc3faddbb6ec1738039a7ecb5 Mon Sep 17 00:00:00 2001 From: Al Bowles Date: Mon, 13 Jun 2022 18:30:24 -0500 Subject: [PATCH 28/45] fix dev box link --- dev_guides/.pages | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/dev_guides/.pages b/dev_guides/.pages index 52fe553..6ecf067 100644 --- a/dev_guides/.pages +++ b/dev_guides/.pages @@ -1,4 +1,4 @@ --- nav: - - Development Box Setup: development_boxes.md + - Development Box Setup: wiki_development_boxes.md - Git Commit Signing: commit_signing.md From cfde1e626519820ca31e4c8b8d931073d5661fd9 Mon Sep 17 00:00:00 2001 From: Al Bowles Date: Mon, 13 Jun 2022 18:37:53 -0500 Subject: [PATCH 29/45] Rename example files --- dev_guides/commit_signing.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/dev_guides/commit_signing.md b/dev_guides/commit_signing.md index 5f94ee6..70f744b 100644 --- a/dev_guides/commit_signing.md +++ b/dev_guides/commit_signing.md @@ -34,13 +34,13 @@ rc: # Create revocation certificate - gpg --output \.gpg-revocation-certificate --gen-revoke my@email.addr + gpg --output my_email_addr.gpg-revocation-certificate --gen-revoke my@email.addr # Back up your keypair Export the *primary keypair* (put these somewhere very safe along with revocation certificate) - gpg --export-secret-keys --armor my@email.addr > \.private.gpg-key - gpg --export --armor my@email.addr > \.public.gpg-key + gpg --export-secret-keys --armor my@email.addr > my_email_addr.private.gpg-key + gpg --export --armor my@email.addr > my_email_addr.public.gpg-key # Remove the *primary keypair* from your keyring Export all subkeys from the new keypair to a file - use ramfs instead of tmpfs/ or /dev/shm/ because ramfs doesn't write to swap @@ -66,7 +66,7 @@ Look for `sec#` instead of `sec` in the output - pound sign means signing subkey # Revoking a *signing keypair* Find the *primary keypair* and import it (preferably into an ephemeral system like a liveUSB) - gpg --import /path/to/\.public.gpg-key /path/to/\.private.gpg-key + gpg --import /path/to/my_email_addr.public.gpg-key /path/to/my_email_addr.private.gpg-key gpg --edit-key my@email.addr gpg> revkey [ passphrase twice ] From 13b503f79c4d387850ba0bab0fbcfb8b3e7d1428 Mon Sep 17 00:00:00 2001 From: Al Bowles Date: Mon, 13 Jun 2022 18:49:07 -0500 Subject: [PATCH 30/45] Simplify subkey export/import process --- dev_guides/commit_signing.md | 22 +++++++++------------- 1 file changed, 9 insertions(+), 13 deletions(-) diff --git a/dev_guides/commit_signing.md b/dev_guides/commit_signing.md index 70f744b..4850280 100644 --- a/dev_guides/commit_signing.md +++ b/dev_guides/commit_signing.md @@ -43,25 +43,21 @@ Export the *primary keypair* (put these somewhere very safe along with revocatio gpg --export --armor my@email.addr > my_email_addr.public.gpg-key # Remove the *primary keypair* from your keyring -Export all subkeys from the new keypair to a file - use ramfs instead of tmpfs/ or /dev/shm/ because ramfs doesn't write to swap +1. Export all subkeys from the new keypair to a file - mkdir /tmp/gpg - sudo mount -t ramfs -o size=1M ramfs /tmp/gpg - sudo chown $(logname):$(logname) /tmp/gpg - gpg --export-secret-subkeys my@email.addr > /tmp/gpg/subkeys + gpg --export-secret-subkeys my@email.addr > $HOME/.gnupg/subkeys -Delete original signing subkey from keypair in our keyring +1. Delete primary key from keyring - *BE SURE TO BACK UP YOUR PRIMARY KEYPAIR FIRST!* - gpg --delete-secret-key my@email.addr + gpg --delete-secret-key my@email.addr -Re-import the previously exported keys +1. Re-import the previously exported keys - gpg --import /tmp/gpg/subkeys - sudo umount /tmp/gpg - rmdir /tmp/gpg + gpg --import $HOME/.gnupg/subkeys -Look for `sec#` instead of `sec` in the output - pound sign means signing subkey is *not* in the keypair located in the keyring - gpg --list-secret-keys $HOME/.gnupg/secring.gpg +1. Look for `sec#` instead of `sec` in the output - pound sign means signing subkey is *not* in the keypair located in the keyring + + gpg --list-secret-keys $HOME/.gnupg/secring.gpg # Revoking a *signing keypair* Find the *primary keypair* and import it (preferably into an ephemeral system like a liveUSB) From 08c6cad3b917297e1316ad7e9b16356c06b8051b Mon Sep 17 00:00:00 2001 From: Trevor Cooper Date: Fri, 1 Jul 2022 22:18:40 -0700 Subject: [PATCH 31/45] Testing: Add Testcase_VNC_Graphics_Mode (#37) * add Testcase_VNC_Graphics_Mode * suggest but do not require specific VNC clients * complete description and add examples * update QA testcases index --- QA/Testcase_Basic_Graphics_Mode.md | 6 +-- QA/Testcase_Custom_Boot_Methods_Boot_Iso.md | 32 ++++++++++++++ QA/Testcase_VNC_Graphics_Mode.md | 46 +++++++++++++++++++++ qa_test_cases.md | 7 ++-- release_criteria/r9/9_release_criteria.md | 6 +-- 5 files changed, 88 insertions(+), 9 deletions(-) create mode 100644 QA/Testcase_Custom_Boot_Methods_Boot_Iso.md create mode 100644 QA/Testcase_VNC_Graphics_Mode.md diff --git a/QA/Testcase_Basic_Graphics_Mode.md b/QA/Testcase_Basic_Graphics_Mode.md index e344032..15499d1 100644 --- a/QA/Testcase_Basic_Graphics_Mode.md +++ b/QA/Testcase_Basic_Graphics_Mode.md @@ -1,7 +1,7 @@ --- title: QA:Testcase Basic Graphics Mode author: Trevor Cooper -revision_date: 2022-05-18 +revision_date: 2022-06-23 rc: prod: Rocky Linux ver: 8 @@ -9,10 +9,10 @@ rc: --- !!! error "REFERENCED RELEASE CRITERIA IS OVERLY GENERAL AND UNTESTABLE" - The associated release criteria, [Release_Criteria#basic-graphics-mode-behaviors](9_release_criteria.md#basic-graphics-mode-behaviors), for this test case is overly general and **must** be modified to specific enough to be testable. + The associated release criteria, [Release_Criteria#basic-graphics-mode-behaviors](8_release_criteria.md#basic-graphics-mode-behaviors), for this test case is overly general and **must** be modified to specific enough to be testable. !!! info "Associated release criterion" - This test case is associated with the [Release_Criteria#basic-graphics-mode-behaviors](9_release_criteria.md#basic-graphics-mode-behaviors) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + This test case is associated with the [Release_Criteria#basic-graphics-mode-behaviors](8_release_criteria.md#basic-graphics-mode-behaviors) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. ## Description This test case will verify that release-blocking installers function as intended using the generic video driver option (“basic graphics mode”) on supported systems and classes of hardware. diff --git a/QA/Testcase_Custom_Boot_Methods_Boot_Iso.md b/QA/Testcase_Custom_Boot_Methods_Boot_Iso.md new file mode 100644 index 0000000..b5fb18d --- /dev/null +++ b/QA/Testcase_Custom_Boot_Methods_Boot_Iso.md @@ -0,0 +1,32 @@ +--- +title: QA:Testcase Custom Boot Methods Boot Iso +author: Trevor Cooper +revision_date: 2022-06-23 +rc: + prod: Rocky Linux + ver: 9 + level: Final +--- + +!!! info "Associated release criterion" + This test case is associated with the [Release_Criteria#vnc-graphics-mode-behaviors](9_release_criteria.md#vnc-graphics-mode-behaviors) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + +## Description +This is to verify that the Anaconda installer starts correctly when booting from the Rocky Linux boot.iso using a custom kernel command line. + +## Setup +1. Prepare your system for booting the boot.iso image. This may involve writing the image to a USB key or burning it to an optical disk. Additionally, attaching the boot.iso to a virtual machine instance as a Virtual Optical Disk or mounting the boot.iso to server via baseboard management controller virtual media attach should be possible but is not expressly required. + +## How to test +1. Boot the system from the prepared optical, USB media or virtual device attachment. +2. In the boot menu select the appropriate option to boot the installer. +3. Interrupt the normal boot and edit the kernel command line. +4. Add appropriate/required options to the kernel command line and resume booting into the installer. + - Example: For network install from an alternate repository add `--inst.url=http:///` and (optionally) `--inst.repo=AppStream,http:///` to the kernel command line. + - Example: For VNC install in **Direct Mode** add `--inst.vnc` to the kernel command line. For VNC install in **Connect Mode** add `--inst.vnc` and `--inst.vncserver=:` to the kernel command line. + +## Expected Results +1. Boot menu is displayed for users to select install options. Navigating the menu and selecting entries must work. Editing the boot command line must be possible. If no option is selected, the installer should load after a reasonable timeout. +2. System boots into the Anaconda installer and any command line options specified are utilized. + +{% include 'testing/qa_testcase_bottom.md' %} diff --git a/QA/Testcase_VNC_Graphics_Mode.md b/QA/Testcase_VNC_Graphics_Mode.md new file mode 100644 index 0000000..8dab859 --- /dev/null +++ b/QA/Testcase_VNC_Graphics_Mode.md @@ -0,0 +1,46 @@ +--- +title: QA:Testcase VNC Graphics Mode +author: Trevor Cooper +revision_date: 2022-06-23 +rc: + prod: Rocky Linux + ver: 9 + level: Final +--- + +!!! info "Associated release criterion" + This test case is associated with the [Release_Criteria#vnc-graphics-mode-behaviors](9_release_criteria.md#vnc-graphics-mode-behaviors) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + +## Description +This test case will verify that release-blocking installers function as intended using the VNC installation method on supported systems and classes of hardware. + +{% include 'testing/qa_testcase_supported_systems.md' %} + +## Setup +1. Obtain access to supported system and hardware class to be installed. +2. Prepare appropriate media for the selected ISO to be tested. + - Example: [QA:Testcase Media USB dd](Testcase_Media_USB_dd.md) +3. Obtain access to remote system with a VNC client installed to use for VNC connection. + + +!!! info "Suggested VNC Clients" + Both [`tigervnc`](https://tigervnc.org) and [`vinagre`](https://wiki.gnome.org/Apps/Vinagre) are VNC clients provided in Rocky Linux but any VNC client may be used. + +## How to test +1. Boot the system from the prepared optical, USB media or virtual device attachment. + - Examples: [QA:Testcase Custom Boot Methods Boot ISO](Testcase_Custom_Boot_Methods_Boot_Iso.md) +2. Interrupt the kernel boot and specify the appropriate VNC installation option on the boot command line. +3. Proceed with installation on the test system.
**Depending on installer choices this MAY destroy all the data on the test system.** +4. Depending on the choice or `direct` or `connect` mode connect inbound to the system under test or wait for it to connect to your listening VNC client. + +!!! error "DATA LOSS" + If you choose to complete the installation of the test system any/all data on the system may be lost. Please do not install on a system whose contents you need to keep. + +## Expected Results +1. Connection to (with direct mode) or from (in connect mode) to the Anaconda installer using VNC is possible. +2. Anaconda installer presents a usable graphical intallation environment. +3. System under test can be installed normally. +4. After reboot system boots into graphical environment. +5. After login user is able to operate the graphical environment. + +{% include 'testing/qa_testcase_bottom.md' %} diff --git a/qa_test_cases.md b/qa_test_cases.md index 1d0782e..0cdc519 100644 --- a/qa_test_cases.md +++ b/qa_test_cases.md @@ -1,6 +1,6 @@ --- title: QA:Test Cases -revision_date: 2022-05-19 +revision_date: 2022-06-23 rc: prod: Rocky Linux --- @@ -13,10 +13,11 @@ This page lists all test cases in work and who is working on them... | --------------------------------------------------- | ------------------------------------------------------------------------ | ----------------------- | --------------------------------------- | | Release-blocking images must boot
[{{ rc.prod }} 8](8_release_criteria.md#release-blocking-images-must-boot) [{{ rc.prod }} 9](9_release_criteria.md#release-blocking-images-must-boot) | [QA:Testcase Boot Methods Boot ISO](Testcase_Boot_Methods_Boot_Iso.md) | @tcooper | template exists, openQA covered (ref) | | Release-blocking images must boot
[{{ rc.prod }} 8](8_release_criteria.md#release-blocking-images-must-boot) [{{ rc.prod }} 9](9_release_criteria.md#release-blocking-images-must-boot) | [QA:Testcase Boot Methods DVD](Testcase_Boot_Methods_Dvd.md) | @tcooper | template exists, openQA covered (ref) | -| Basic Graphics Mode behaviors
[{{ rc.prod }} 8](8_release_criteria.md#basic-graphics-mode-behaviors) [{{ rc.prod }} 9](9_release_criteria.md#basic-graphics-mode-behaviors) | [QA:Testcase Basic Graphics Mode](Testcase_Basic_Graphics_Mode.md) | @tcooper | manual and/or new openQA TestCase | +| Basic Graphics Mode behaviors
[{{ rc.prod }} 8](8_release_criteria.md#basic-graphics-mode-behaviors) | [QA:Testcase Basic Graphics Mode](Testcase_Basic_Graphics_Mode.md) | @tcooper | openQA TestCase | +| VNC Graphics Mode behaviors
[{{ rc.prod }} 9](9_release_criteria.md#vnc-graphics-mode-behaviors) | [QA:Testcase VNC Graphics Mode](Testcase_VNC_Graphics_Mode.md) | @tcooper | openQA TestCase | | No Broken Packages
[{{ rc.prod }} 8](8_release_criteria.md#no-broken-packages) [{{ rc.prod }} 9](9_release_criteria.md#no-broken-packages) | [QA:Testcase Media Repoclosure](Testcase_Media_Repoclosure.md)
[QA:Testcase Media File Conflicts](Testcase_Media_File_Conflicts.md) | @tcooper | manual using scripts or automated in CI | | Repositories Must Match Upstream
[{{ rc.prod }} 8](8_release_criteria.md#repositories-must-match-upstream) [{{ rc.prod }} 9 ](9_release_criteria.md#repositories-must-match-upstream) | [QA:Testcase repocompare](Testcase_Repo_Compare.md) | @tcooper | manual using Skip's repocompare | -| Debranding
[{{ rc.prod }} 8](8_release_criteria.md#debranding) [{{ rc.prod }} 9](9_release_criteria.md#debranding) | [QA:Testcase Debranding Analysis](Testcase_Debranding.md) | @tcooper | process TBD | +| Debranding
[{{ rc.prod }} 8](8_release_criteria.md#debranding) [{{ rc.prod }} 9](9_release_criteria.md#debranding) | [QA:Testcase Debranding Analysis](Testcase_Debranding.md) | @tcooper | manual using scripts or automated in CI | ## Installer Requirements diff --git a/release_criteria/r9/9_release_criteria.md b/release_criteria/r9/9_release_criteria.md index 2603c86..18522f8 100644 --- a/release_criteria/r9/9_release_criteria.md +++ b/release_criteria/r9/9_release_criteria.md @@ -49,11 +49,11 @@ Release-blocking installer images must boot when written to optical media or USB - [QA:Testcase Boot Methods DVD](Testcase_Boot_Methods_Dvd.md) - [QA:Testcase Media USB dd](Testcase_Media_USB_dd.md) -#### Basic Graphics Mode behaviors -The generic video driver option (“basic graphics mode”) on all release-blocking installers must function as intended. This means launching the installer or desktop and attempting to use a generic driver. There must be no bugs that prevent the installer from being reached in this configuration on all systems and classes of hardware supported by the enterprise linux kernel. +#### VNC Graphics Mode behaviors +The VNC installation option on all release-blocking installers must function as intended. This means launching the installer or desktop and connecting with VNC to complete the installation. There must be no bugs that prevent the installer from being reached in this configuration on all systems and classes of hardware supported by the enterprise linux kernel. ??? tldr "References" - Test cases: - - [QA:Testcase Basic Graphics Mode](Testcase_Basic_Graphics_Mode.md) + - [QA:Testcase VNC Graphics Mode](Testcase_VNC_Graphics_Mode.md) #### No Broken Packages Critical errors, such as undeclared conflicts, unresolved dependencies, or modules relying on packages from another stream will be considered an automatic blocker. There are potential exceptions to this (eg, freeradius cannot be installed on an older perl stream, this is a known issue upstream). From 154e70963278b0d0dc3bf7066ca6762bb8c1995f Mon Sep 17 00:00:00 2001 From: Lukas Magauer <42647570+lumarel@users.noreply.github.com> Date: Tue, 5 Jul 2022 04:51:02 +0200 Subject: [PATCH 32/45] Testing: Post-Install test docs (#15) * Update Release Criteria and Test Cases * Testcase batch 1 * Update the post install release criteria for Rocky Linux 9 * Update the test case overview * Update text-formatting * Testcase batch 2 * Fix words and add some links * Missed one word * Make sure the authors are correct * Add text about the release relevance Co-authored-by: lumarel --- QA/Testcase_Post_Application_Functionality.md | 55 ++++++++++++ QA/Testcase_Post_Artwork_and_Assets.md | 44 ++++++++++ QA/Testcase_Post_GNOME_UI_Functionality.md | 45 ++++++++++ QA/Testcase_Post_Keyboard_Layout.md | 60 +++++++++++++ QA/Testcase_Post_Module_Streams.md | 49 +++++++++++ QA/Testcase_Post_Multimonitor_Setup.md | 36 ++++++++ QA/Testcase_Post_Package_installs.md | 55 ++++++++++++ QA/Testcase_Post_SELinux_Errors_Desktop.md | 38 +++++++++ QA/Testcase_Post_SELinux_Errors_Server.md | 40 +++++++++ QA/Testcase_Post_System_Services.md | 41 +++++++++ qa_test_cases.md | 22 ++--- release_criteria/r8/8_release_criteria.md | 84 ++++++++++++------- release_criteria/r9/9_release_criteria.md | 82 +++++++++++------- 13 files changed, 579 insertions(+), 72 deletions(-) create mode 100644 QA/Testcase_Post_Application_Functionality.md create mode 100644 QA/Testcase_Post_Artwork_and_Assets.md create mode 100644 QA/Testcase_Post_GNOME_UI_Functionality.md create mode 100644 QA/Testcase_Post_Keyboard_Layout.md create mode 100644 QA/Testcase_Post_Module_Streams.md create mode 100644 QA/Testcase_Post_Multimonitor_Setup.md create mode 100644 QA/Testcase_Post_Package_installs.md create mode 100644 QA/Testcase_Post_SELinux_Errors_Desktop.md create mode 100644 QA/Testcase_Post_SELinux_Errors_Server.md create mode 100644 QA/Testcase_Post_System_Services.md diff --git a/QA/Testcase_Post_Application_Functionality.md b/QA/Testcase_Post_Application_Functionality.md new file mode 100644 index 0000000..efc6af4 --- /dev/null +++ b/QA/Testcase_Post_Application_Functionality.md @@ -0,0 +1,55 @@ +--- +title: QA:Testcase Application Functionality +author: Lukas Magauer +revision_date: 2022-05-31 +rc: + prod: Rocky Linux + ver: + - 8 + - 9 + level: Final +--- + +!!! info "Release relevance" + This Testcase applies the following versions of {{ rc.prod }}: {% for version in rc.ver %}{{ version }}{% if not loop.last %}, {% endif %}{% endfor %} + +!!! info "Associated release criterion" + This test case is associated with the [Release_Criteria#default-application-functionality-desktop-only](9_release_criteria.md#default-application-functionality-desktop-only) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + +!!! error "REFERENCED RELEASE CRITERIA IS OVERLY GENERAL AND UNTESTABLE" + The associated release criteria, [Release_Criteria#default-application-functionality-desktop-only](9_release_criteria.md#default-application-functionality-desktop-only), for this test case is overly general and **must** be modified to specific enough to be testable. + +## Description + +This testcase handles all applications, considered as core applications of the desktop environment GNOME or user facing commandline applications. + +The following tasks apply in general to all of the following applications: + +- Firefox +- Files (Nautilus) +- GNOME Software +- (Image Viewer) +- (Document Viewer) +- Gedit (Text Editor) +- Archive Manager +- GNOME Terminal (Terminal Emulator) +- Problem Reporter +- Help Viewer +- System Settings +- vim (Console Text Editor) + +## Setup + +Obtain access to a suitable system with either a Workstation or a Graphical Server installation. + +## How to test + +1. Check that the application starts without any errors +2. Further check that the context menus for the correct function +3. Open files to test the functionality of the individual applications + +## Expected Results + +Make sure that the individual applications behave as they should. + +{% include 'testing/qa_testcase_bottom.md' %} diff --git a/QA/Testcase_Post_Artwork_and_Assets.md b/QA/Testcase_Post_Artwork_and_Assets.md new file mode 100644 index 0000000..5edb3a9 --- /dev/null +++ b/QA/Testcase_Post_Artwork_and_Assets.md @@ -0,0 +1,44 @@ +--- +title: QA:Testcase Artwork and Assets +author: Lukas Magauer +revision_date: 2022-05-31 +rc: + prod: Rocky Linux + ver: + - 8 + - 9 + level: Final +--- + +!!! info "Release relevance" + This Testcase applies the following versions of {{ rc.prod }}: {% for version in rc.ver %}{{ version }}{% if not loop.last %}, {% endif %}{% endfor %} + +!!! info "Associated release criterion" + This test case is associated with the [Release_Criteria#artwork-and-assets-server-and-desktop](9_release_criteria.md#artwork-and-assets-server-and-desktop) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + +## Description + +There are several brand artworks and assets throughout the whole OS, this testcase will take care of checking, that these are actually in place, and don't produce any UI errors. This is exclusively for installations with the default desktop environment GDM and GNOME. + +## Setup + +1. Acquire access to either a baremetal machine or a VM host, to install a new machine +2. Prepare appropriate media for the selected ISO to be tested. + - Example: [QA:Testcase Media USB dd](Testcase_Media_USB_dd.md) + +## How to test + +1. While booting the image check, that the correct logo is visible in the loading screen before Anaconda comes up +2. Look at the Anaconda images in the [rocky-logos repo](https://github.com/rocky-linux/rocky-logos/tree/r8-fedora/anaconda) and check if all assets are correctly applied in Anaconda (they will generally be visible right away while going through the install process) +3. Install the system with either the Workstation install set or Graphical Server +4. While the OS does its first boot, check that the correct logo is visible in the loading screen before the boot login screen shows up +5. Check the logo and background of the boot login screen +6. After the login check the desktop background and further all available options in the settings menu for the desktop background +7. Lock the screen and check the background visible in the flyover +8. At last check the logo and background of the login screen + +## Expected Results + +The tests during the process could be successfully finished. + +{% include 'testing/qa_testcase_bottom.md' %} diff --git a/QA/Testcase_Post_GNOME_UI_Functionality.md b/QA/Testcase_Post_GNOME_UI_Functionality.md new file mode 100644 index 0000000..14f560b --- /dev/null +++ b/QA/Testcase_Post_GNOME_UI_Functionality.md @@ -0,0 +1,45 @@ +--- +title: QA:Testcase GNOME UI Functionality +author: Lukas Magauer +revision_date: 2022-05-31 +rc: + prod: Rocky Linux + ver: + - 8 + - 9 + level: Final +--- + +!!! info "Release relevance" + This Testcase applies the following versions of {{ rc.prod }}: {% for version in rc.ver %}{{ version }}{% if not loop.last %}, {% endif %}{% endfor %} + +!!! error "REFERENCED RELEASE CRITERIA IS OVERLY GENERAL AND UNTESTABLE" + The associated release criteria, [Release_Criteria#default-panel-functionality-desktop-only](9_release_criteria.md#default-panel-functionality-desktop-only), for this test case is overly general and **must** be modified to specific enough to be testable. + +!!! info "Associated release criterion" + This test case is associated with the [Release_Criteria#default-panel-functionality-desktop-only](9_release_criteria.md#default-panel-functionality-desktop-only) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + +## Description + +This test collection takes care of the correct functionality of the GNOME UI. + +## Setup + +Obtain access to a suitable system with either a Workstation or a Graphical Server installation. + +## How to test + +1. Login to the Rocky Machine via the UI +2. Navigate through the GNOME UI + +## Expected Results + +1. After the login you should have landed on the desktop with the background and the top bar of GNOME visible +2. Clicking the the Activities button in the upper right should bring up the overview +3. Further there should be the favourite applications ribbon on the left +4. And after clicking the 9-dot-icon all applications should appear +5. Back on the desktop check the function of the system and clock panel on the upper right and middle + +It is also a good measure to do some more basic clicking through the GNOME UI, like opening applications, interacting with the application headerbar, moving applications to different desktops or changing settings in the System settings. + +{% include 'testing/qa_testcase_bottom.md' %} diff --git a/QA/Testcase_Post_Keyboard_Layout.md b/QA/Testcase_Post_Keyboard_Layout.md new file mode 100644 index 0000000..d57e72e --- /dev/null +++ b/QA/Testcase_Post_Keyboard_Layout.md @@ -0,0 +1,60 @@ +--- +title: QA:Testcase Keyboard Layout +author: Lukas Magauer +revision_date: 2022-05-31 +rc: + prod: Rocky Linux + ver: + - 8 + - 9 + level: Final +--- + +!!! info "Release relevance" + This Testcase applies the following versions of {{ rc.prod }}: {% for version in rc.ver %}{{ version }}{% if not loop.last %}, {% endif %}{% endfor %} + +!!! info "Associated release criterion" + This test case is associated with the [Release_Criteria#keyboard-layout](9_release_criteria.md#keyboard-layout) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + +## Description + +As there are a lot of different keyboard layouts available, it is necessary to test if the keyboard functionality works without any issues throughout the system. + +## Setup + +- Obtain access to a few different system configurations, especially with and without UI, and not to forget with disk encryption. +- Acquire access to either a baremetal machine or a VM host, to install a new machine + - Prepare appropriate media for the selected ISO to be tested. + - Example: [QA:Testcase Media USB dd](Testcase_Media_USB_dd.md) + +## How to test + +### Installer + +1. Bootup the installer +2. Choose a language +3. Make sure that the keyboard layout got chosen correctly corresponding to the language setting +4. Change the keyboard layout if needed to test +5. Enter text all over Anaconda to make sure the keyboard layout works correctly with the chosen keyboard layout + +### Disk Encryption + +1. Setup a system with disk encryption +2. Check that the password for the disk encryption works on bootup with graphical UI +3. Check that the password for the disk encryption works on bootup with text mode + +### Text mode + +Check that the chosen keyboard layout works correctly on text mode. + +### GNOME and Application + +1. Check the login, that the keyboard layout works correctly on the graphical UI login screen +2. Also check that the GNOME UI works correctly with the chosen keyboard layout +3. And finally check some applications, that the keyboard works as expected + +## Expected Results + +The tests during the process could be successfully finished. + +{% include 'testing/qa_testcase_bottom.md' %} diff --git a/QA/Testcase_Post_Module_Streams.md b/QA/Testcase_Post_Module_Streams.md new file mode 100644 index 0000000..97401c8 --- /dev/null +++ b/QA/Testcase_Post_Module_Streams.md @@ -0,0 +1,49 @@ +--- +title: QA:Testcase Module Streams +author: Lukas Magauer +revision_date: 2022-05-31 +rc: + prod: Rocky Linux + ver: + - 8 + - 9 + level: Final +--- + +!!! info "Release relevance" + This Testcase applies the following versions of {{ rc.prod }}: {% for version in rc.ver %}{{ version }}{% if not loop.last %}, {% endif %}{% endfor %} + +!!! info "Associated release criterion" + This test case is associated with the [Release_Criteria#packages-and-module-installation](9_release_criteria.md#packages-and-module-installation) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + +## Description + +This test case takes care of testing the module streams, that they are all installable, all available and working as expected. + +## Setup + +For this tests you will need to install every module stream on its own, so it's the best to setup a new system which gets snapshoted after the initial setup. After that it can be rolled back for every module install. + +It's enough to setup a system with the Minimal Install group. + +## How to test + +1. Login to the machine +2. Get a list of all module streams and compare it to the [stream list from RHEL](https://access.redhat.com/support/policy/updates/rhel-app-streams-life-cycle) and to the source in [Git source repo](https://git.rockylinux.org/rocky/rocky-module-defaults) +3. The easiest way to test all streams is to install the package groups in the individual streams, i.e. for postgresql: + +```bash +dnf module install postgresql +dnf module install postgresql:13 +dnf module install postgresql:13/client +``` + +Repeat step 3 as often as module streams and package groups are available. + +This could be automated with i.e. Ansible to do all the `install -> rollback -> install -> rollback -> ...` and emiting the output via Ansible. + +## Expected Results + +All module streams should be available and there shouldn't be any errors while installing any of the package groups of the individual streams. (some of the installs will show warnings though because they are incompatible with other streams) + +{% include 'testing/qa_testcase_bottom.md' %} diff --git a/QA/Testcase_Post_Multimonitor_Setup.md b/QA/Testcase_Post_Multimonitor_Setup.md new file mode 100644 index 0000000..053ab48 --- /dev/null +++ b/QA/Testcase_Post_Multimonitor_Setup.md @@ -0,0 +1,36 @@ +--- +title: QA:Testcase Multimonitor Setup +author: Lukas Magauer +revision_date: 2022-05-31 +rc: + prod: Rocky Linux + ver: + - 8 + - 9 + level: Final +--- + +!!! info "Release relevance" + This Testcase applies the following versions of {{ rc.prod }}: {% for version in rc.ver %}{{ version }}{% if not loop.last %}, {% endif %}{% endfor %} + +!!! info "Associated release criterion" + This test case is associated with the [Release_Criteria#dual-monitor-setup-desktop-only](9_release_criteria.md#dual-monitor-setup-desktop-only) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + +## Description + +This test covers the check if GNOME behaves as it should in multi-monitor setups. + +## Setup + +You will need either a machine which can be reinstalled with multiple screens, or a virtualization software which is capable of providing multiple screens (like VMware Workstation ([Pro](https://www.vmware.com/products/workstation-pro/workstation-pro-evaluation.html) or [Player](https://www.vmware.com/products/workstation-player/workstation-player-evaluation.html)) or [VMware Fusion](https://www.vmware.com/products/fusion/fusion-evaluation.html), but there is also a [hack](https://communities.vmware.com/t5/VMware-vSphere-Discussions/ESXi-6-7-Multiple-Monitors-for-VMs/td-p/2748906) for [VMware ESXi](https://customerconnect.vmware.com/en/web/vmware/evalcenter?p=vsphere-eval-7)) + +## How to test + +1. Run installer with multiple screens connected and install with either the Workstation or Graphical Server group +2. Login to the machine after the finished install + +## Expected Results + +There shouldn't be any graphical glitches, or scaling issues through the install and the usage. + +{% include 'testing/qa_testcase_bottom.md' %} diff --git a/QA/Testcase_Post_Package_installs.md b/QA/Testcase_Post_Package_installs.md new file mode 100644 index 0000000..adedd55 --- /dev/null +++ b/QA/Testcase_Post_Package_installs.md @@ -0,0 +1,55 @@ +--- +title: QA:Testcase Basic Package installs +author: Lukas Magauer +revision_date: 2022-05-31 +rc: + prod: Rocky Linux + ver: + - 8 + - 9 + level: Final +--- + +!!! info "Release relevance" + This Testcase applies the following versions of {{ rc.prod }}: {% for version in rc.ver %}{{ version }}{% if not loop.last %}, {% endif %}{% endfor %} + +!!! info "Associated release criterion" + This test case is associated with the [Release_Criteria#packages-and-module-installation](9_release_criteria.md#packages-and-module-installation) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + +!!! error "REFERENCED RELEASE CRITERIA IS OVERLY GENERAL AND UNTESTABLE" + The associated release criteria, [Release_Criteria#packages-and-module-installation](9_release_criteria.md#packages-and-module-installation), for this test case is overly general and **must** be modified to specific enough to be testable. + +## Description + +Installing several packages should work without any issues. + +Please also test these usecases (it's basically the fun of learning to install software, it's even good if it's done differently each other time): + +- httpd +- httpd with php and ssl +- nginx +- nginx with php and ssl +- mysql-server +- mysql-server with secure setup +- mariadb-server +- postgresql-server +- postgresql-server with secure setup +- compiling packages with: + - cmake + - g++ +- ipa-server +- ipa-server with dns + +## Setup + +Obtain access to a suitable system where any of the tested packages can be installed without any issues. + +## How to test + +1. Install a list of packages or usecases + +## Expected Results + +Installs work without any issues. + +{% include 'testing/qa_testcase_bottom.md' %} diff --git a/QA/Testcase_Post_SELinux_Errors_Desktop.md b/QA/Testcase_Post_SELinux_Errors_Desktop.md new file mode 100644 index 0000000..7eb01fe --- /dev/null +++ b/QA/Testcase_Post_SELinux_Errors_Desktop.md @@ -0,0 +1,38 @@ +--- +title: QA:Testcase SELinux Errors on Desktop clients +author: Lukas Magauer +revision_date: 2022-05-31 +rc: + prod: Rocky Linux + ver: + - 8 + - 9 + level: Final +--- + +!!! info "Release relevance" + This Testcase applies the following versions of {{ rc.prod }}: {% for version in rc.ver %}{{ version }}{% if not loop.last %}, {% endif %}{% endfor %} + +!!! info "Associated release criterion" + This test case is associated with the [Release_Criteria#selinux-and-crash-notifications-desktop-only](9_release_criteria.md#selinux-and-crash-notifications-desktop-only) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + +## Description + +Basically running a Workstation or Graphical Server install for a longer amount of time, while using it and then checking if there were any SELinux audit messages. + +## Setup + +Obtain access to a suitable system with either a Workstation or a Graphical Server installation. + +## How to test + +1. Setup new machine or get access to a installed machine +2. Click through the system and various applications, to mimic user behavior +3. Leave the system running for a few more minutes, if possible hours + +## Expected Results + +1. Open the SETroubleshoot Application and invoke the error summarization. +2. There must not be shown any errors / denials + +{% include 'testing/qa_testcase_bottom.md' %} diff --git a/QA/Testcase_Post_SELinux_Errors_Server.md b/QA/Testcase_Post_SELinux_Errors_Server.md new file mode 100644 index 0000000..72fde65 --- /dev/null +++ b/QA/Testcase_Post_SELinux_Errors_Server.md @@ -0,0 +1,40 @@ +--- +title: QA:Testcase SELinux Errors on Server installations +author: Lukas Magauer +revision_date: 2022-05-31 +rc: + prod: Rocky Linux + ver: + - 8 + - 9 + level: Final +--- + +!!! info "Release relevance" + This Testcase applies the following versions of {{ rc.prod }}: {% for version in rc.ver %}{{ version }}{% if not loop.last %}, {% endif %}{% endfor %} + +!!! info "Associated release criterion" + This test case is associated with the [Release_Criteria#selinux-errors-server](9_release_criteria.md#selinux-errors-server) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + +## Description + +Basically running a text mode installation for a longer amount of time, while using it and then checking if there were any SELinux audit messages. + +## Setup + +Obtain access to a suitable system with one of the text mode installation base groups. + +It might also be beneficial to run this test with other than the core installation, but that is a long term test and a bit out of scope of this test. + +## How to test + +1. Setup new machine or get access to a installed machine +2. As this test is mostly about the stability of the core system it is mostly only needed to let the system run for a few minutes, if possible hours + +## Expected Results + +1. Install `sealert` with `dnf install setroubleshoot-server` +2. Run `sealert -a /var/log/audit/audit.log` +3. There must not be shown any errors / denials + +{% include 'testing/qa_testcase_bottom.md' %} diff --git a/QA/Testcase_Post_System_Services.md b/QA/Testcase_Post_System_Services.md new file mode 100644 index 0000000..ddd3734 --- /dev/null +++ b/QA/Testcase_Post_System_Services.md @@ -0,0 +1,41 @@ +--- +title: QA:Testcase System Services +author: Lukas Magauer +revision_date: 2022-05-31 +rc: + prod: Rocky Linux + ver: + - 8 + - 9 + level: Final +--- + +!!! info "Release relevance" + This Testcase applies the following versions of {{ rc.prod }}: {% for version in rc.ver %}{{ version }}{% if not loop.last %}, {% endif %}{% endfor %} + +!!! info "Associated release criterion" + This test case is associated with the [Release_Criteria#system-services](9_release_criteria.md#system-services) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + +## Description + +This test covers the check, that all basic system service which are getting installed with the base groups are starting / running normally. + +## Setup + +1. Acquire access to either a baremetal machine or a VM host, to install a new machine +2. Prepare appropriate media for the selected ISO to be tested. + - Example: [QA:Testcase Media USB dd](Testcase_Media_USB_dd.md) + +## How to test + +Startup the system and check that all services are running without any failure: + +```bash +systemctl status +``` + +## Expected Results + +The tests during the process could be successfully finished. + +{% include 'testing/qa_testcase_bottom.md' %} diff --git a/qa_test_cases.md b/qa_test_cases.md index 0cdc519..1349b9d 100644 --- a/qa_test_cases.md +++ b/qa_test_cases.md @@ -48,17 +48,17 @@ This page lists all test cases in work and who is working on them... ## Post-Installation Requirements -| Requirement | Test Case | Assignee | Status | -| --------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------- | ----------------------- | --------------------------------------- | -| System Services | [QA:Testcase System Services](Testcase_Template.md) | @lumarel | testcase filename reserved | -| Keyboard Layout | [QA:Testcase Keyboard Layout](Testcase_Template.md) | @lumarel | testcase filename reserved | -| SELinux Errors (Server) | [QA:Testcase SELinux Errors on Server installations](Testcase_Template.md) | @lumarel, @raktajino | testcase filename reserved | -| SELinux and Crash Notifications (Desktop Only) | [QA:Testcase SELinux Errors on Desktop clients](Testcase_Template.md) | @lumarel, @raktajino | testcase filename reserved | -| Default Application Functionality (Desktop Only) | [QA:Testcase Application Functionality](Testcase_Template.md) | @lumarel | testcase filename reserved | -| Default Panel Functionality (Desktop Only) | [QA:Testcase GNOME UI Functionality](Testcase_Template.md) | @lumarel | testcase filename reserved | -| Dual Monitor Setup (Desktop Only) | [QA:Testcase Multimonitor Setup](Testcase_Template.md) | @lumarel | testcase filename reserved | -| Artwork and Assets (Server and Desktop) | [QA:Testcase Artwork and Assets](Testcase_Template.md) | @lumarel | testcase filename reserved | -| Packages and Module Installation | [QA:Testcase Basic Package installs](Testcase_Template.md)
[QA:Testcase Module Streams](Testcase_Template.md) | @lumarel | | +| Requirement | Test Case | Assignee | Status | +|--------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------|----------|----------------------------------------------------------------------| +| System Services | [QA:Testcase System Services](Testcase_Post_System_Services.md) | @lumarel | manual guide documented or needs new openQA testcase | +| Keyboard Layout | [QA:Testcase Keyboard Layout](Testcase_Post_Keyboard_Layout.md) | @lumarel | implemented in openQA | +| SELinux Errors (Server) | [QA:Testcase SELinux Errors on Server](Testcase_Post_SELinux_Errors_Server.md) | @lumarel | implemented in openQA | +| SELinux and Crash Notifications (Desktop Only) | [QA:Testcase SELinux Errors on Desktop](Testcase_Post_SELinux_Errors_Desktop.md) | @lumarel | partly implemented in openQA | +| Default Application Functionality (Desktop Only) | [QA:Testcase Application Functionality](Testcase_Post_Application_Functionality.md) | @lumarel | manual guide documented | +| Default Panel Functionality (Desktop Only) | [QA:Testcase GNOME UI Functionality](Testcase_Post_GNOME_UI_Functionality.md) | @lumarel | implemented in openQA, additionally documented for manual inspection | +| Dual Monitor Setup (Desktop Only) | [QA:Testcase Multimonitor Setup](Testcase_Post_Multimonitor_Setup.md) | @lumarel | manual guide documented | +| Artwork and Assets (Server and Desktop) | [QA:Testcase Artwork and Assets](Testcase_Post_Artwork_and_Assets.md) | @lumarel | implemented in openQA, additionally documented for manual inspection | +| Packages and Module Installation | [QA:Testcase Basic Package installs](Testcase_Post_Package_installs.md)
[QA:Testcase Module Streams](Testcase_Post_Module_Streams.md) | @lumarel | partly implemented in openQA, manual guide documented | {% include 'content_bottom.md' %} diff --git a/release_criteria/r8/8_release_criteria.md b/release_criteria/r8/8_release_criteria.md index 2bace3f..2e72d76 100644 --- a/release_criteria/r8/8_release_criteria.md +++ b/release_criteria/r8/8_release_criteria.md @@ -1,7 +1,9 @@ --- title: Rocky Linux 8 Release Criteria -author: Trevor Cooper -revision_date: 2022-04-01 +author: + - Trevor Cooper + - Lukas Magauer +revision_date: 2022-05-07 rc: prod: Rocky Linux ver: 8 @@ -164,82 +166,102 @@ Release-blocking cloud disk images must be published to appropriate cloud provid - [QA:Testcase TBD](Testcase_Template.md) ### Post-Installation Requirements + #### System Services + All system services present after installation must start properly, with the exception of services that require hardware which is not present. Examples of such services would be: - - sshd - - firewalld - - auditd - - chronyd + +- sshd +- firewalld +- auditd +- chronyd + ??? tldr "References" - Test cases: - - [QA:Testcase TBD](Testcase_Template.md) + - [QA:Testcase System Services](Testcase_Post_System_Services.md) #### Keyboard Layout + If a particular keyboard layout has been configured for the system, that layout must be used: - - When unlocking storage volumes (encrypted by LUKS) - - When logging in at a TTY console - - When logging in via GDM - - After logging into a GNOME desktop system, if the user does not have their own layout configuration set. + +- When unlocking storage volumes (encrypted by LUKS) +- When logging in at a TTY console +- When logging in via GDM +- After logging into a GNOME desktop system, if the user does not have their own layout configuration set. ??? tldr "References" - Test cases: - - [QA:Testcase TBD](Testcase_Template.md) + - [QA:Testcase Keyboard Layout](Testcase_Post_Keyboard_Layout.md) #### SELinux Errors (Server) + There must be no SELinux denial logs in /var/log/audit/audit.log ??? tldr "References" - Test cases: - - [QA:Testcase TBD](Testcase_Template.md) + - [QA:Testcase SELinux Errors on Server installations](Testcase_Post_SELinux_Errors_Server.md) #### SELinux and Crash Notifications (Desktop Only) + There must be no SELinux denial notifications or crash notifications on boot, during installation, or during first login. ??? tldr "References" - Test cases: - - [QA:Testcase TBD](Testcase_Template.md) + - [QA:Testcase SELinux Errors on Desktop clients](Testcase_Post_SELinux_Errors_Desktop.md) #### Default Application Functionality (Desktop Only) + Applications that can be launched within GNOME or on the command line must start successfully and withstand basic functionality tests. This includes: - - Web browser - - File manager - - Package manager - - Image/Document Viewers - - Text editors (gedit, vim) - - Archive manager - - Terminal Emulator (gnome terminal) - - Problem Reporter - - Help Viewer - - System Settings + +- Web browser +- File manager +- Package manager +- Image/Document Viewers +- Text editors (gedit, vim) +- Archive manager +- Terminal Emulator (GNOME Terminal) +- Problem Reporter +- Help Viewer +- System Settings ??? tldr "References" - Test cases: - - [QA:Testcase TBD](Testcase_Template.md) + - [QA:Testcase Application Functionality](Testcase_Post_Application_Functionality.md) #### Default Panel Functionality (Desktop Only) + All elements of GNOME should function properly in regular use. + ??? tldr "References" - Test cases: - - [QA:Testcase TBD](Testcase_Template.md) + - [QA:Testcase GNOME UI Functionality](Testcase_Post_GNOME_UI_Functionality.md) #### Dual Monitor Setup (Desktop Only) + Computers using two monitors, the graphical output is correctly shown on both monitors. + ??? tldr "References" - Test cases: - - [QA:Testcase TBD](Testcase_Template.md) + - [QA:Testcase Multimonitor Setup](Testcase_Post_Multimonitor_Setup.md) #### Artwork and Assets (Server and Desktop) + Proposed final artwork (such as wallpapers and other assets) must be included. A wallpaper from this package should show up as a default for GDM and GNOME. + ??? tldr "References" - Test cases: - - [QA:Testcase TBD](Testcase_Template.md) + - [QA:Testcase Artwork and Assets](Testcase_Post_Artwork_and_Assets.md) #### Packages and Module Installation + Packages (non-module) should be able to be installed without conflicts or dependent on repositories outside of {{ rc.prod }}. - - Default modules (as listed in dnf module list) should be installed without requiring them to be enabled. - - Module streams should be able to be switched and those packages should be able to be installed without errors or unresolved dependencies. + +- Default modules (as listed in dnf module list) should be installed without requiring them to be enabled. +- Module streams should be able to be switched and those packages should be able to be installed without errors or unresolved dependencies. + ??? tldr "References" - Test cases: - - [QA:Testcase TBD](Testcase_Template.md) + - [QA:Testcase Basic Package installs](Testcase_Post_Package_installs.md) + - [QA:Testcase Module Streams](Testcase_Post_Module_Streams.md) {% include 'testing/rc_content_bottom.md' %} diff --git a/release_criteria/r9/9_release_criteria.md b/release_criteria/r9/9_release_criteria.md index 18522f8..2c7885e 100644 --- a/release_criteria/r9/9_release_criteria.md +++ b/release_criteria/r9/9_release_criteria.md @@ -1,6 +1,8 @@ --- title: Rocky Linux 9 Release Criteria -author: Trevor Cooper +author: + - Trevor Cooper + - Lukas Magauer revision_date: 2022-04-01 rc: prod: Rocky Linux @@ -164,82 +166,102 @@ Release-blocking cloud disk images must be published to appropriate cloud provid - [QA:Testcase TBD](Testcase_Template.md) ### Post-Installation Requirements + #### System Services + All system services present after installation must start properly, with the exception of services that require hardware which is not present. Examples of such services would be: - - sshd - - firewalld - - auditd - - chronyd + +- sshd +- firewalld +- auditd +- chronyd + ??? tldr "References" - Test cases: - - [QA:Testcase TBD](Testcase_Template.md) + - [QA:Testcase System Services](Testcase_Post_System_Services.md) #### Keyboard Layout + If a particular keyboard layout has been configured for the system, that layout must be used: - - When unlocking storage volumes (encrypted by LUKS) - - When logging in at a TTY console - - When logging in via GDM - - After logging into a GNOME desktop system, if the user does not have their own layout configuration set. + +- When unlocking storage volumes (encrypted by LUKS) +- When logging in at a TTY console +- When logging in via GDM +- After logging into a GNOME desktop system, if the user does not have their own layout configuration set. ??? tldr "References" - Test cases: - - [QA:Testcase TBD](Testcase_Template.md) + - [QA:Testcase Keyboard Layout](Testcase_Post_Keyboard_Layout.md) #### SELinux Errors (Server) + There must be no SELinux denial logs in /var/log/audit/audit.log ??? tldr "References" - Test cases: - - [QA:Testcase TBD](Testcase_Template.md) + - [QA:Testcase SELinux Errors on Server installations](Testcase_Post_SELinux_Errors_Server.md) #### SELinux and Crash Notifications (Desktop Only) + There must be no SELinux denial notifications or crash notifications on boot, during installation, or during first login. ??? tldr "References" - Test cases: - - [QA:Testcase TBD](Testcase_Template.md) + - [QA:Testcase SELinux Errors on Desktop clients](Testcase_Post_SELinux_Errors_Desktop.md) #### Default Application Functionality (Desktop Only) + Applications that can be launched within GNOME or on the command line must start successfully and withstand basic functionality tests. This includes: - - Web browser - - File manager - - Package manager - - Image/Document Viewers - - Text editors (gedit, vim) - - Archive manager - - Terminal Emulator (gnome terminal) - - Problem Reporter - - Help Viewer - - System Settings + +- Web browser +- File manager +- Package manager +- Image/Document Viewers +- Text editors (gedit, vim) +- Archive manager +- Terminal Emulator (GNOME Terminal) +- Problem Reporter +- Help Viewer +- System Settings ??? tldr "References" - Test cases: - - [QA:Testcase TBD](Testcase_Template.md) + - [QA:Testcase Application Functionality](Testcase_Post_Application_Functionality.md) #### Default Panel Functionality (Desktop Only) + All elements of GNOME should function properly in regular use. + ??? tldr "References" - Test cases: - - [QA:Testcase TBD](Testcase_Template.md) + - [QA:Testcase GNOME UI Functionality](Testcase_Post_GNOME_UI_Functionality.md) #### Dual Monitor Setup (Desktop Only) + Computers using two monitors, the graphical output is correctly shown on both monitors. + ??? tldr "References" - Test cases: - - [QA:Testcase TBD](Testcase_Template.md) + - [QA:Testcase Multimonitor Setup](Testcase_Post_Multimonitor_Setup.md) #### Artwork and Assets (Server and Desktop) + Proposed final artwork (such as wallpapers and other assets) must be included. A wallpaper from this package should show up as a default for GDM and GNOME. + ??? tldr "References" - Test cases: - - [QA:Testcase TBD](Testcase_Template.md) + - [QA:Testcase Artwork and Assets](Testcase_Post_Artwork_and_Assets.md) #### Packages and Module Installation + Packages (non-module) should be able to be installed without conflicts or dependent on repositories outside of {{ rc.prod }}. - - Default modules (as listed in dnf module list) should be installed without requiring them to be enabled. - - Module streams should be able to be switched and those packages should be able to be installed without errors or unresolved dependencies. + +- Default modules (as listed in dnf module list) should be installed without requiring them to be enabled. +- Module streams should be able to be switched and those packages should be able to be installed without errors or unresolved dependencies. + ??? tldr "References" - Test cases: - - [QA:Testcase TBD](Testcase_Template.md) + - [QA:Testcase Basic Package installs](Testcase_Post_Package_installs.md) + - [QA:Testcase Module Streams](Testcase_Post_Module_Streams.md) {% include 'testing/rc_content_bottom.md' %} From bd35f0cef58169395586b435d6b0b3fa32b58cd9 Mon Sep 17 00:00:00 2001 From: akatch Date: Fri, 8 Jul 2022 19:01:59 -0500 Subject: [PATCH 33/45] Testing: Installer Requirements (#28) feat: Testcase documentation for Installer Requirements Includes the following: - [x] Media Consistency Verification - [x] Packages and Installer Sources - [x] Installation Interfaces - [x] Minimal Installation - [x] Kickstart Installation - [x] Disk Layouts - [x] Bootloader Disk Selection - [x] Storage Volume Resize - [x] Update Image - [x] Installer Help - [x] Installer Translations --- QA/Testcase_Bootloader_Disk_Selection.md | 34 +++++++++++++ QA/Testcase_Disk_Layouts.md | 61 +++++++++++++++++++++++ QA/Testcase_Firmware_RAID.md | 30 +++++++++++ QA/Testcase_Installation_Interfaces.md | 55 ++++++++++++++++++++ QA/Testcase_Installer_Help.md | 46 +++++++++++++++++ QA/Testcase_Installer_Translations.md | 37 ++++++++++++++ QA/Testcase_Kickstart_Installation.md | 41 +++++++++++++++ QA/Testcase_Minimal_Installation.md | 37 ++++++++++++++ QA/Testcase_Network_Attached_Storage.md | 37 ++++++++++++++ QA/Testcase_Packages_Installer_Sources.md | 53 ++++++++++++++++++++ QA/Testcase_Storage_Volume_Resize.md | 54 ++++++++++++++++++++ QA/Testcase_Update_Image.md | 51 +++++++++++++++++++ qa_test_cases.md | 30 +++++------ release_criteria/r8/8_release_criteria.md | 28 ++++++----- release_criteria/r9/9_release_criteria.md | 28 ++++++----- 15 files changed, 581 insertions(+), 41 deletions(-) create mode 100644 QA/Testcase_Bootloader_Disk_Selection.md create mode 100644 QA/Testcase_Disk_Layouts.md create mode 100644 QA/Testcase_Firmware_RAID.md create mode 100644 QA/Testcase_Installation_Interfaces.md create mode 100644 QA/Testcase_Installer_Help.md create mode 100644 QA/Testcase_Installer_Translations.md create mode 100644 QA/Testcase_Kickstart_Installation.md create mode 100644 QA/Testcase_Minimal_Installation.md create mode 100644 QA/Testcase_Network_Attached_Storage.md create mode 100644 QA/Testcase_Packages_Installer_Sources.md create mode 100644 QA/Testcase_Storage_Volume_Resize.md create mode 100644 QA/Testcase_Update_Image.md diff --git a/QA/Testcase_Bootloader_Disk_Selection.md b/QA/Testcase_Bootloader_Disk_Selection.md new file mode 100644 index 0000000..2fcd572 --- /dev/null +++ b/QA/Testcase_Bootloader_Disk_Selection.md @@ -0,0 +1,34 @@ +--- +title: QA:Testcase Bootloader Disk Selection +author: Al Bowles +revision_date: 2022-07-08 +rc: + prod: Rocky Linux + ver: + - 8 + - 9 + level: Final +--- + +!!! info "Associated release criterion" + This test case is associated with the [Release_Criteria#Bootloader Disk Selection](9_release_criteria.md#bootloader-disk-selection) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + +## Description +This test case verifies that the user is able to select an alternative disk on which to install the bootloader. It also verifies that, if the user is so inclined, they may choose not to install a bootloader at all. + +{% include 'testing/qa_data_loss_warning.md' %} + +## Setup +{% include 'testing/qa_setup_boot_to_media.md' %} + +## How to test +1. In the Installation Destination spoke, select the disk(s) to install to, then click the "Full disk summary and bootl loader..." button at the bottom of the screen: ![Full disk summary and bootloader...](/assets/images/bootloader.png){ loading=lazy } +1. Click the checkbox next to the disk on which the bootloader is desired +1. Alternatively, uncheck the boot checkbox next to all disks to skip bootloader installation +1. Proceed with installation on the test system. + +## Expected Results +1. Installation should complete successfully. +1. Note that if no bootloader is installed, the system may not boot after installation is complete. This is expected. + +{% include 'testing/qa_testcase_bottom.md' %} diff --git a/QA/Testcase_Disk_Layouts.md b/QA/Testcase_Disk_Layouts.md new file mode 100644 index 0000000..f8a4133 --- /dev/null +++ b/QA/Testcase_Disk_Layouts.md @@ -0,0 +1,61 @@ +--- +title: QA:Testcase Disk Layouts +author: Al Bowles +revision_date: 2022-07-08 +rc: + prod: Rocky Linux + ver: + - 8 + - 9 + level: Final +--- + +!!! info "Associated release criterion" + This test case is associated with the [Release_Criteria#Disk Layouts](9_release_criteria.md#disk-layouts) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + +## Description +This test case verifies successful installation to any supported partition layout using any file system or format combination. + +{% include 'testing/qa_data_loss_warning.md' %} + +## Setup +{% include 'testing/qa_setup_boot_to_media.md' %} + +## How to test +1. Select the Installation Destination spoke. +1. Select the volumes to which the operating system should be installed. +1. Select the Custom radio button under the Storage Configuration section, then click "Done". +1. For each volume, perform these steps: + 1. Choose the desired partitioning scheme from the dropdown menu. Supported options are Standard Partition, LVM, and LVM Thin Provisioning. + 1. Select the "Encrypt my data" checkbox to create an encrypted filesystem. + 1. Select the plus (+) button in the lower left hand corner to add a partition. + 1. Define the desired mount point and volume capacity, then click "Add mount point". + 1. Set the device type. Supported options are LVM, RAID, Standard Partition, and LVM Thin Provisioning. + 1. If device type was set to RAID, select the RAID level. Supported options are RAID0, RAID1, RAID4, RAID5, RAID6, and RAID10. + 1. Set the filesystem type. Supported options are BIOS Boot, ext2, ext3, ext4, swap, vfat, and xfs. + 1. In supported cases you may choose to disable formatting of existing partitions by unchecking the Reformat checkbox. +1. When all partitions have been created, click the blue Done button in the upper left corner. +1. Review the Summary of Changes dialog, then click Accept Changes. +1. Continue the installation as normal. + +## Expected Results +1. The installation should complete successfully and boot to the appropriate disk. +1. The specified filesystem type and partition scheme should be used. +1. If configured, software RAID should function as expected. + +## Testing with openQA +The following openQA test suites satisfy this release criteria: + +- `install_standard_partition_ext4` +- `install_custom_gui_standard_partition_ext4` +- `install_lvm_ext4` +- `install_custom_gui_lvm_ext4` +- `install_software_raid` +- `install_custom_gui_software_raid` +- `install_xfs` +- `install_custom_gui_xfs` +- `install_lvmthin` +- `install_multi` +- `install_multi_empty` + +{% include 'testing/qa_testcase_bottom.md' %} diff --git a/QA/Testcase_Firmware_RAID.md b/QA/Testcase_Firmware_RAID.md new file mode 100644 index 0000000..2d00bc7 --- /dev/null +++ b/QA/Testcase_Firmware_RAID.md @@ -0,0 +1,30 @@ +--- +title: QA:Testcase Firmware RAID +author: Al Bowles +revision_date: 2022-07-08 +rc: + prod: Rocky Linux + ver: + - 8 + - 9 + level: Final +--- + +!!! info "Associated release criterion" + This test case is associated with the [Release_Criteria#Firmware RAID](9_release_criteria.md#firmware-raid) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + +## Description +The installer must be able to detect and install to firmware RAID devices. Note that system-specific bugs do not count as blockers. It is likely that some hardware support might be broken or not available at all. DUDs (driver update disks) are not considered for this criteria. + +## Setup +1. Add steps for setup for this Testcase. + +## How to test +1. Do this first... +2. Then do this... + +## Expected Results +1. This is what you should see/verify. +2. You should also see/verify this. + +{% include 'testing/qa_testcase_bottom.md' %} diff --git a/QA/Testcase_Installation_Interfaces.md b/QA/Testcase_Installation_Interfaces.md new file mode 100644 index 0000000..8627e98 --- /dev/null +++ b/QA/Testcase_Installation_Interfaces.md @@ -0,0 +1,55 @@ +--- +title: QA:Testcase Installation Interfaces +author: Al Bowles +revision_date: 2022-07-08 +rc: + prod: Rocky Linux + ver: + - 8 + - 9 + level: Final +--- + +!!! info "Associated release criterion" + This test case is associated with the [Release_Criteria#Installation Interfaces](9_release_criteria.md#installation-interfaces) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + +## Description +This test case verifies that the installer can complete an installation using all Anaconda spokes. + +{% include 'testing/qa_data_loss_warning.md' %} + +## Setup +{% include 'testing/qa_setup_boot_to_media.md' %} + +## How to test + +1. Select a keyboard layout in the Keyboard Layout spoke +1. Set language support in the Language spoke +1. Set the system time and date in the Time and Date spoke + +1. Set a root password in the Root Password spoke +1. Create a user in the user creation spoke + +1. Select an installation source from the Installation Source spoke +1. Select a set of packages to install from the Package Selection spoke + +1. Set a disk to which the operating system should install in the Installation Destination spoke +1. Set the kdump state from the Kdump spoke +1. Configure the system's network and hostname from the Network and Hostname spoke +1. Select a security policy from the Security Policy spoke + +## Expected Results +1. The installation should complete and boot successfully. + +## Testing in openQA +The following openQA test suites satisfy this release criteria: + +- `install_arabic_language` OR `install_asian_language` + + +{% include 'testing/qa_testcase_bottom.md' %} diff --git a/QA/Testcase_Installer_Help.md b/QA/Testcase_Installer_Help.md new file mode 100644 index 0000000..0892e0d --- /dev/null +++ b/QA/Testcase_Installer_Help.md @@ -0,0 +1,46 @@ +--- +title: QA:Testcase Installer Help +author: Al Bowles +revision_date: 2022-07-08 +rc: + prod: Rocky Linux + ver: + - 8 + - 9 + level: Final +--- + +!!! info "Associated release criterion" + This test case is associated with the [Release_Criteria#Installer Help](9_release_criteria.md#installer-help) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + +## Description +Any element in the installer which contains a “help” text must display the appropriate help documentation when selected. + +## Setup +{% include 'testing/qa_setup_boot_to_media.md' %} + +## How to test +1. From the Anaconda Hub, click the Help button in the upper right hand corner. +1. Verify that you see the "Customizing your Installation" help page. +1. Verify that the "Configuring language and location settings" link displays a topically appropriate page. +1. Close the Help browser to return to the Anaconda Hub. +1. Verify that the Localization help page displays for the Keyboard, Language Support, and Time & Date spokes: + 1. Select the spoke, then click the Help button. + 1. Verify that you see the "Configuring localization options" page containing a functioning link to the "Configuring keyboard, language, and time and date settings" page. + 1. Close the Help browser (and click Done if necessary) to return to the Anaconda Hub. +1. Verify that the Help button in the Installation Source spoke displays the "Configuring installation source" page. +1. Verify that the Help button in the Software Selection spoke displays the "Configuring software selection" page. +1. Verify that the Help button in the Installation Destination spoke displays the "Configuring storage devices" page. +1. Verify that the Help button in the Network & Host Name spoke displays the "Configuring network and host name options" page. +1. Verify that the Help button in the Root Password spoke displays the "Configuring a root password" page. +1. Verify that the Help button in the User Creation spoke displays the "Creating a user account" page. + +## Expected Results +1. All links should work and display relevant content. + +## Testing in openQA +The following openQA test suites satisfy this release criteria: + +- `anaconda_help` + +{% include 'testing/qa_testcase_bottom.md' %} diff --git a/QA/Testcase_Installer_Translations.md b/QA/Testcase_Installer_Translations.md new file mode 100644 index 0000000..8f70da2 --- /dev/null +++ b/QA/Testcase_Installer_Translations.md @@ -0,0 +1,37 @@ +--- +title: QA:Testcase Installer Translations +author: Al Bowles +revision_date: 2022-07-08 +rc: + prod: Rocky Linux + ver: + - 8 + - 9 + level: Final +--- + +!!! info "Associated release criterion" + This test case is associated with the [Release_Criteria#Installer Translations](9_release_criteria.md#installer-translations) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + +## Description +The installer must correctly display all complete translations that are available for use. + +## Setup +{% include 'testing/qa_setup_boot_to_media.md' %} + +## How to test +1. From the Language Selection spoke, select a language. + +## Expected Results +1. All spokes should display at least some of the content in the selected language. +2. It is expected to still see some content displayed in Latin characters even if a language that does not use Latin characters is selected. + +## Testing in openQA +The following openQA test suites satisfy this release criteria: + +- `install_asian_language` +- `install_arabic_language` +- `install_cyrillic_language` +- `install_european_language` + +{% include 'testing/qa_testcase_bottom.md' %} diff --git a/QA/Testcase_Kickstart_Installation.md b/QA/Testcase_Kickstart_Installation.md new file mode 100644 index 0000000..9792b70 --- /dev/null +++ b/QA/Testcase_Kickstart_Installation.md @@ -0,0 +1,41 @@ +--- +title: QA:Testcase Kickstart Installation +author: Al Bowles +revision_date: 2022-07-08 +rc: + prod: Rocky Linux + ver: + - 8 + - 9 + level: Final +--- + +!!! info "Associated release criterion" + This test case is associated with the [Release_Criteria#Kickstart Installation](9_release_criteria.md#kickstart-installation) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + +## Description +This test case verifies that installations via both local and remote Kickstart configuration files are successful. + +{% include 'testing/qa_data_loss_warning.md' %} + +## Setup +1. Copy a valid Kickstart file to a USB stick +1. Connect the USB stick to the test system +{% include 'testing/qa_setup_boot_to_media.md' %} +1. Hit the Tab key to edit the boot command +1. Provide a local Kickstart file by supplying the GRUB boot option `inst.ks=file:/path/to/local.ks` or a remote Kickstart file by supplying the GRUB boot option `inst.ks=https://dl.rockylinux.org/remote.ks`. + +## How to test +1. Continue booting the installer as normal. + +## Expected Results +1. The installation should complete and boot successfully, automatically populating the options specified in the Kickstart file. + +## Testing in openQA +The following openQA test suites satisfy this release criteria: + +- `install_kickstart_nfs` +- `server_realmd_join_kickstart` + + +{% include 'testing/qa_testcase_bottom.md' %} diff --git a/QA/Testcase_Minimal_Installation.md b/QA/Testcase_Minimal_Installation.md new file mode 100644 index 0000000..7ad1599 --- /dev/null +++ b/QA/Testcase_Minimal_Installation.md @@ -0,0 +1,37 @@ +--- +title: QA:Testcase Minimal Installation +author: Al Bowles +revision_date: 2022-07-08 +rc: + prod: Rocky Linux + ver: + - 8 + - 9 + level: Final +--- + +!!! info "Associated release criterion" + This test case is associated with the [Release_Criteria#Minimal Installation](9_release_criteria.md#minimal-installation) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + +## Description +This test case verifies that a networked minimal installation is able to install the 'Minimal' package set. The installation should not require use of local packages to complete. + +{% include 'testing/qa_data_loss_warning.md' %} + +## Setup +{% include 'testing/qa_setup_boot_to_media.md' %} + +## How to test +1. From the Installation Source spoke, configure a remote repository source from [MirrorManager](https://mirrors.rockylinux.org) appropriate to the architecture under test. +1. From the Software Selection spoke, select the Minimal package set. +1. Complete the installation using desired parameters. + +## Expected Results +1. The installation should complete and boot successfully. + +## Testing in openQA +The following openQA test suites satisfy this release criteria: + +- `install_minimal` + +{% include 'testing/qa_testcase_bottom.md' %} diff --git a/QA/Testcase_Network_Attached_Storage.md b/QA/Testcase_Network_Attached_Storage.md new file mode 100644 index 0000000..a23d1fc --- /dev/null +++ b/QA/Testcase_Network_Attached_Storage.md @@ -0,0 +1,37 @@ +--- +title: QA:Testcase Network Attached Storage +author: Al Bowles +revision_date: 2022-07-08 +rc: + prod: Rocky Linux + ver: + - 8 + - 9 + level: Final +--- + +!!! info "Associated release criterion" + This test case is associated with the [Release_Criteria#Network Attached Storage](9_release_criteria.md#nas-network-attached-storage) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + +## Description +The installer must be able to detect and install to supported NAS devices (if possible and supported by the kernel). + +## Setup +1. Add steps for setup for this Testcase. + +## How to test +### NFS +install nfs-utils +sudo mount -t nfs nfs-server:/nfs/path /mnt +then a created a file echo 1 > /mnt/1 +verified it and permissions ls /mnt; cat /mnt/1 +then deleted it rm /mnt/1 +then unmounted sudo umount /mnt + +### iSCSI + +## Expected Results +1. This is what you should see/verify. +2. You should also see/verify this. + +{% include 'testing/qa_testcase_bottom.md' %} diff --git a/QA/Testcase_Packages_Installer_Sources.md b/QA/Testcase_Packages_Installer_Sources.md new file mode 100644 index 0000000..bc0ac35 --- /dev/null +++ b/QA/Testcase_Packages_Installer_Sources.md @@ -0,0 +1,53 @@ +--- +title: QA:Testcase Packages and Installer Sources +author: Al Bowles +revision_date: 2022-07-08 +rc: + prod: Rocky Linux + ver: 8 + level: Final +--- + +!!! info "Associated release criterion" + This test case is associated with the [Release_Criteria#Packages and Installer Sources](9_release_criteria.md#packages-installer-sources) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + +## Description +This test case verifies that the installer can successfully install any of the supported package sets via any of the supported installer sources. + +The following package sets are supported for installs from local media: + +- server +- minimal + +The following package sets are only available from remote sources and require a network connection: + +- workstation +- graphical-server +- virtualization-host + +{% include 'testing/qa_data_loss_warning.md' %} + +## Setup +{% include 'testing/qa_setup_boot_to_media.md' %} + +## How to test +1. For local package installations it is not necessary to enable networking or specify a mirror. +1. For package installation from remote sources: + 1. From the Network and Hostname spoke, enable networking. + 1. From the Installation Source spoke, configure a remote software source, supplying an appropriate [mirror](https://mirrors.rockylinux.org) for the version and architecture under test. +1. Complete the installer and wait for the machine to reboot. + +## Expected Results +1. The installation should complete and boot successfully. +1. If a graphical package set was specified, the system should boot to a graphical login screen. + +## Testing in openQA +The following openQA test suites satisfy this release criteria, provided they pass the `_do_install_reboot` module at a minimum: + +- `install_packageset_server` +- `install_packageset_minimal` +- `install_packageset_workstation` +- `install_packageset_graphical-server` +- `install_packageset_virtualization-host` + +{% include 'testing/qa_testcase_bottom.md' %} diff --git a/QA/Testcase_Storage_Volume_Resize.md b/QA/Testcase_Storage_Volume_Resize.md new file mode 100644 index 0000000..2ba9478 --- /dev/null +++ b/QA/Testcase_Storage_Volume_Resize.md @@ -0,0 +1,54 @@ +--- +title: QA:Testcase Storage Volume Resize +author: Al Bowles +revision_date: 2022-07-08 +rc: + prod: Rocky Linux + ver: + - 8 + - 9 + level: Final +--- + +!!! info "Associated release criterion" + This test case is associated with the [Release_Criteria#Storage Volume Resize](9_release_criteria.md#storage-volume-resize) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + +## Description +This test case verifies that the installer will successfully resize or delete and overwrite existing partitions on storage volumes. + +{% include 'testing/qa_data_loss_warning.md' %} + +## Setup +{% include 'testing/qa_setup_boot_to_media.md' %} + +## How to test + +### Resize +1. From the Installation Destination spoke, in the Storage Configuration section, select the Custom radio button, then click Done. +1. Expand the list of available partitions by clicking the black arrow to the left of the release version and architecture. +1. Select the partition you wish to resize. Be sure to uncheck the Reformat checkbox if you wish to resize without reformatting the partition. +1. Click the Update Settings button to save your settings. +1. Click the + button to create a new partition off of the existing partition. Provide a mount point and desired capacity, then click Add Mount Point. +1. Repeat as necessary for additional partitions, or click Done to return to the Anaconda main hub. + +### Delete +1. From the Installation Destination spoke, in the Storage Configuration section, select the Automatic radio button, then click Done. +1. You should be presented with an "Installation Options" dialog, indicating the amount of disk space that is available for use and available to reclaim. +1. Click the Reclaim Space button. +1. Select a partition, then click the Delete button to delete the partition and reclaim the space. +1. Alternatively, click the Delete All button to delete all existing partitions. +1. When you have finished, click the Reclaim Space button to reclaim available free space. + +## Expected Results +1. The installation should complete and boot successfully. +1. Resized partitions should correctly reflect the desired size. This may be verified using the `lsblk` command. +1. Deleted partitions should no longer exist. + +## Testing in openQA +The following openQA test suites satisfy this release criteria: + +- `install_delete_partial` +- `install_delete_pata` +- `install_resize_lvm` + +{% include 'testing/qa_testcase_bottom.md' %} diff --git a/QA/Testcase_Update_Image.md b/QA/Testcase_Update_Image.md new file mode 100644 index 0000000..c30812a --- /dev/null +++ b/QA/Testcase_Update_Image.md @@ -0,0 +1,51 @@ +--- +title: QA:Testcase Update Image +author: Al Bowles +revision_date: 2022-07-08 +rc: + prod: Rocky Linux + ver: 8 + level: Final +--- + +!!! info "Associated release criterion" + This test case is associated with the [Release_Criteria#Update Image](9_release_criteria.md#update-image) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + +## Description + +This test case verifies that an [update image]() can be loaded into Anaconda and applied during the install process. + +{% include 'testing/qa_data_loss_warning.md' %} + +## Setup +{% include 'testing/qa_setup_boot_to_media.md' %} +1. Hit the Tab key to edit the boot command + +## How to test + +1. Supply `inst.updates=https://fedorapeople.org/groups/qa/updates/updates-openqa.img` to the GRUB command line +1. Boot into the installer as usual. +1. In Anaconda, open the Installation Destination spoke. + +## Expected Results +1. Within the Installation Destination spoke, the selected install disk should have a pink background +=== "FAIL" + ![No update provided - **FAIL**](/assets/images/no_updates.png){ loading=lazy } + +=== "PASS" + ![Update provided - **PASS**](/assets/images/updates.png){ loading=lazy } + +1. If you cannot verify visually, check for the existence of `/tmp/updates`, which should contain updated source files if the update was successfully applied. Note that if the update image doesn't actually contain any source files, this directory will not be created. + + +## Testing with openQA +The following openQA test suites satisfy this release criteria: + +- `install_scsi_updates_img` + +## Additional References +[Red Hat Debug Boot Options](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/performing_an_advanced_rhel_installation/kickstart-and-advanced-boot-options_installing-rhel-as-an-experienced-user#debug-boot-options_kickstart-and-advanced-boot-options)
+[Fedora QA:Testcase Anaconda updates.img via URL](https://fedoraproject.org/wiki/QA:Testcase_Anaconda_updates.img_via_URL)
+[Fedora QA:Testcase Anaconda updates.img via local media](https://fedoraproject.org/wiki/QA:Testcase_Anaconda_updates.img_via_local_media)
+ +{% include 'testing/qa_testcase_bottom.md' %} diff --git a/qa_test_cases.md b/qa_test_cases.md index 1349b9d..02a5492 100644 --- a/qa_test_cases.md +++ b/qa_test_cases.md @@ -22,21 +22,21 @@ This page lists all test cases in work and who is working on them... ## Installer Requirements -| Requirement | Test Case | Assignee | Status | -| --------------------------------------------------- | ------------------------------------------------------------------------ | ----------------------- | --------------------------------------- | -| Media Consistency Verification | [QA:Testcase TBD](Testcase_Template.md) | @tbd | | -| Packages and Installer Sources | [QA:Testcase TBD](Testcase_Template.md) | @tbd | Implemented in openQA, document | -| NAS (Network Attached Storage) | [QA:Testcase TBD](Testcase_Template.md) | @tbd | | -| Installation Interfaces | [QA:Testcase TBD](Testcase_Template.md) | @tbd | Implemented in openQA, document | -| Minimal Installation | [QA:Testcase TBD](Testcase_Template.md) | @tbd | Implemented in openQA, document | -| Kickstart Installation | [QA:Testcase TBD](Testcase_Template.md) | @tbd | Implemented in openQA, document | -| Disk Layouts | [QA:Testcase TBD](Testcase_Template.md) | @tbd | Implemented in openQA, document | -| Firmware RAID | [QA:Testcase TBD](Testcase_Template.md) | @tbd | | -| Bootloader Disk Selection | [QA:Testcase TBD](Testcase_Template.md) | @tbd | | -| Storage Volume Resize | [QA:Testcase TBD](Testcase_Template.md) | @tbd | Implemented in openQA, document | -| Update Image | [QA:Testcase TBD](Testcase_Template.md) | @tbd | Implemented in openQA, document | -| Installer Help | [QA:Testcase TBD](Testcase_Template.md) | @tbd | Implemented in openQA, document | -| Installer Translations | [QA:Testcase TBD](Testcase_Template.md) | @tbd | Implemented in openQA, document | +| Requirement | Test Case | Assignee | Status | +| --------------------------------------------------- | ----------------------------------------------------------------------------------------- | ----------------------- | --------------------------------------- | +| Media Consistency Verification | [QA:Testcase Media USB dd](Testcase_Media_USB_dd.md)
[QA:Testcase Boot Methods Boot ISO](Testcase_Boot_Methods_Boot_Iso.md)
[QA:Testcase Boot Methods DVD](Testcase_Boot_Methods_Dvd.md) | @raktajino | | +| Packages and Installer Sources | [QA:Testcase Packages and Installer Sources](Testcase_Packages_Installer_Sources.md) | @raktajino | Implemented in openQA, document | +| NAS (Network Attached Storage) | [QA:Testcase Network Attached Storage](Testcase_Network_Attached_Storage.md) | @raktajino | | +| Installation Interfaces | [QA:Testcase Installation Interfaces](Testcase_Installation_Interfaces.md) | @raktajino | Implemented in openQA, document | +| Minimal Installation | [QA:Testcase Minimal Installation](Testcase_Minimal_Installation.md) | @raktajino | Implemented in openQA, document | +| Kickstart Installation | [QA:Testcase Kickstart Installation](Testcase_Kickstart_Installation.md) | @raktajino | Implemented in openQA, document | +| Disk Layouts | [QA:Testcase Disk Layouts](Testcase_Disk_Layouts.md) | @raktajino | Implemented in openQA, document | +| Firmware RAID | [QA:Testcase Firmware RAID](Testcase_Firmware_RAID.md) | @raktajino | | +| Bootloader Disk Selection | [QA:Testcase Bootloader Disk Selection](Testcase_Bootloader_Disk_Selection.md) | @raktajino | | +| Storage Volume Resize | [QA:Testcase Storage Volume Resize](Testcase_Storage_Volume_Resize.md) | @raktajino | Implemented in openQA, document | +| Update Image | [QA:Testcase Update Image](Testcase_Update_Image.md) | @raktajino | Implemented in openQA, document | +| Installer Help | [QA:Testcase Installer Help](Testcase_Installer_Help.md) | @raktajino | Implemented in openQA, document | +| Installer Translations | [QA:Testcase Installer Translations](Testcase_Installer_Translations.md) | @raktajino | Implemented in openQA, document | ## Cloud Image Requirements diff --git a/release_criteria/r8/8_release_criteria.md b/release_criteria/r8/8_release_criteria.md index 2e72d76..8dfde94 100644 --- a/release_criteria/r8/8_release_criteria.md +++ b/release_criteria/r8/8_release_criteria.md @@ -84,79 +84,81 @@ Assets and functionality that are Red Hat specific should not be included. If th This means that the installer’s mechanism for verifying the install medium is intact and must complete successfully, with the assumption that the medium was correctly written. It should return a failure message if this not the case. ??? tldr "References" - Test cases: - - [QA:Testcase TBD](Testcase_Template.md) + - [QA:Testcase Media USB dd](Testcase_Media_USB_dd.md) + - [QA:Testcase Boot Methods Boot ISO](Testcase_Boot_Methods_Boot_Iso.md) + - [QA:Testcase Boot Methods DVD](Testcase_Boot_Methods_Dvd.md) #### Packages and Installer Sources The installer must be able to use all supported local/remote packages and installer sources. ??? tldr "References" - Test cases: - - [QA:Testcase TBD](Testcase_Template.md) + - [QA:Testcase Packages and Installer Sources](Testcase_Packages_Installer_Sources.md) #### NAS (Network Attached Storage)" The installer must be able to detect and install to supported NAS devices (if possible and supported by the kernel). ??? tldr "References" - Test cases: - - [QA:Testcase TBD](Testcase_Template.md) + - [QA:Testcase Network Attached Storage](Testcase_Network_Attached_Storage.md) #### Installation Interfaces The installer must be able to complete an installation using all supported spokes. ??? tldr "References" - Test cases: - - [QA:Testcase TBD](Testcase_Template.md) + - [QA:Testcase Installation Interfaces](Testcase_Installation_Interfaces.md) #### Minimal Installation A minimal installation (via network) must be able to install the minimal package set. ??? tldr "References" - Test cases: - - [QA:Testcase TBD](Testcase_Template.md) + - [QA:Testcase Minimal Installation](Testcase_Minimal_Installation.md) #### Kickstart Installation A kickstart installation should succeed, whether from optical/USB media or via the network. ??? tldr "References" - Test cases: - - [QA:Testcase TBD](Testcase_Template.md) + - [QA:Testcase Kickstart Installation](Testcase_Kickstart_Installation.md) #### Disk Layouts The installer must be able to create and install to any workable partition layout using any file system or format combination offered or supported by the installer. File systems that are not supported by the EL kernel is not tested here (this means btrfs, zfs, both of wish are not supported). ??? tldr "References" - Test cases: - - [QA:Testcase TBD](Testcase_Template.md) + - [QA:Testcase Disk Layouts](Testcase_Disk_Layouts.md) #### Firmware RAID The installer must be able to detect and install to firmware RAID devices. Note that system-specific bugs do not count as blockers. It is likely that some hardware support might be broken or not available at all. DUDs (driver update disks) are not considered for this criteria. ??? tldr "References" - Test cases: - - [QA:Testcase TBD](Testcase_Template.md) + - [QA:Testcase Firmware RAID](Testcase_Firmware_RAID.md) #### Bootloader Disk Selection The installer must allow the user to choose which disk the bootloader will be installed to or, if the user so chooses, not to install a bootloader. ??? tldr "References" - Test cases: - - [QA:Testcase TBD](Testcase_Template.md) + - [QA:Testcase Bootloader Disk Selection](Testcase_Bootloader_Disk_Selection.md) #### Storage Volume Resize Any installer mechanism for resizing storage volumes must correctly attempt the requested operation. This means that if the installer offers a way to resize storage volumes, then it must use the correct resizing tool with the correct parameters. However, it does not require the installer to disallow resizing of unformatted or volumes with an unknown filesystem type. ??? tldr "References" - Test cases: - - [QA:Testcase TBD](Testcase_Template.md) + - [QA:Testcase Storage Volume Resize](Testcase_Storage_Volume_Resize.md) #### Update Image The installer must be able to use an installer update image retrieved from removable media or a remote package source. This includes DUDs (driver update disks). ??? tldr "References" - Test cases: - - [QA:Testcase TBD](Testcase_Template.md) + - [QA:Testcase Update Image](Testcase_Update_Image.md) #### Installer Help Any element in the installer which contains a “help” text must display the appropriate help documentation when selected. ??? tldr "References" - Test cases: - - [QA:Testcase TBD](Testcase_Template.md) + - [QA:Testcase Installer Help](Testcase_Installer_Help.md) #### Installer Translations The installer must correctly display all complete translations that are available for use. ??? tldr "References" - Test cases: - - [QA:Testcase TBD](Testcase_Template.md) + - [QA:Testcase Installer Translations](Testcase_Installer_Translations.md) ### Cloud Image Requirements #### Images Published to Cloud Providers diff --git a/release_criteria/r9/9_release_criteria.md b/release_criteria/r9/9_release_criteria.md index 2c7885e..4fef4c4 100644 --- a/release_criteria/r9/9_release_criteria.md +++ b/release_criteria/r9/9_release_criteria.md @@ -84,79 +84,81 @@ Assets and functionality that are Red Hat specific should not be included. If th This means that the installer’s mechanism for verifying the install medium is intact and must complete successfully, with the assumption that the medium was correctly written. It should return a failure message if this not the case. ??? tldr "References" - Test cases: - - [QA:Testcase TBD](Testcase_Template.md) + - [QA:Testcase Media USB dd](Testcase_Media_USB_dd.md) + - [QA:Testcase Boot Methods Boot ISO](Testcase_Boot_Methods_Boot_Iso.md) + - [QA:Testcase Boot Methods DVD](Testcase_Boot_Methods_Dvd.md) #### Packages and Installer Sources The installer must be able to use all supported local/remote packages and installer sources. ??? tldr "References" - Test cases: - - [QA:Testcase TBD](Testcase_Template.md) + - [QA:Testcase Packages and Installer Sources](Testcase_Packages_Installer_Sources.md) #### NAS (Network Attached Storage)" The installer must be able to detect and install to supported NAS devices (if possible and supported by the kernel). ??? tldr "References" - Test cases: - - [QA:Testcase TBD](Testcase_Template.md) + - [QA:Testcase Network Attached Storage](Testcase_Network_Attached_Storage.md) #### Installation Interfaces The installer must be able to complete an installation using all supported spokes. ??? tldr "References" - Test cases: - - [QA:Testcase TBD](Testcase_Template.md) + - [QA:Testcase Installation Interfaces](Testcase_Installation_Interfaces.md) #### Minimal Installation A minimal installation (via network) must be able to install the minimal package set. ??? tldr "References" - Test cases: - - [QA:Testcase TBD](Testcase_Template.md) + - [QA:Testcase Minimal Installation](Testcase_Minimal_Installation.md) #### Kickstart Installation A kickstart installation should succeed, whether from optical/USB media or via the network. ??? tldr "References" - Test cases: - - [QA:Testcase TBD](Testcase_Template.md) + - [QA:Testcase Kickstart Installation](Testcase_Kickstart_Installation.md) #### Disk Layouts The installer must be able to create and install to any workable partition layout using any file system or format combination offered or supported by the installer. File systems that are not supported by the EL kernel is not tested here (this means btrfs, zfs, both of wish are not supported). ??? tldr "References" - Test cases: - - [QA:Testcase TBD](Testcase_Template.md) + - [QA:Testcase Disk Layouts](Testcase_Disk_Layouts.md) #### Firmware RAID The installer must be able to detect and install to firmware RAID devices. Note that system-specific bugs do not count as blockers. It is likely that some hardware support might be broken or not available at all. DUDs (driver update disks) are not considered for this criteria. ??? tldr "References" - Test cases: - - [QA:Testcase TBD](Testcase_Template.md) + - [QA:Testcase Firmware RAID](Testcase_Firmware_RAID.md) #### Bootloader Disk Selection The installer must allow the user to choose which disk the bootloader will be installed to or, if the user so chooses, not to install a bootloader. ??? tldr "References" - Test cases: - - [QA:Testcase TBD](Testcase_Template.md) + - [QA:Testcase Bootloader Disk Selection](Testcase_Bootloader_Disk_Selection.md) #### Storage Volume Resize Any installer mechanism for resizing storage volumes must correctly attempt the requested operation. This means that if the installer offers a way to resize storage volumes, then it must use the correct resizing tool with the correct parameters. However, it does not require the installer to disallow resizing of unformatted or volumes with an unknown filesystem type. ??? tldr "References" - Test cases: - - [QA:Testcase TBD](Testcase_Template.md) + - [QA:Testcase Storage Volume Resize](Testcase_Storage_Volume_Resize.md) #### Update Image The installer must be able to use an installer update image retrieved from removable media or a remote package source. This includes DUDs (driver update disks). ??? tldr "References" - Test cases: - - [QA:Testcase TBD](Testcase_Template.md) + - [QA:Testcase Update Image](Testcase_Update_Image.md) #### Installer Help Any element in the installer which contains a “help” text must display the appropriate help documentation when selected. ??? tldr "References" - Test cases: - - [QA:Testcase TBD](Testcase_Template.md) + - [QA:Testcase Installer Help](Testcase_Installer_Help.md) #### Installer Translations The installer must correctly display all complete translations that are available for use. ??? tldr "References" - Test cases: - - [QA:Testcase TBD](Testcase_Template.md) + - [QA:Testcase Installer Translations](Testcase_Installer_Translations.md) ### Cloud Image Requirements #### Images Published to Cloud Providers From f2114830c644e433c0e0e9696b884ecb5b8aa53b Mon Sep 17 00:00:00 2001 From: illuminatus Date: Tue, 2 Aug 2022 20:42:05 -0700 Subject: [PATCH 34/45] Update 9.0_qa_testing_go_no_go.md rebased onto development (#38) Signed-off-by: TrevorBenson --- release_criteria/r9/9.0_qa_testing_go_no_go.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/release_criteria/r9/9.0_qa_testing_go_no_go.md b/release_criteria/r9/9.0_qa_testing_go_no_go.md index 1b15c9a..726ea65 100644 --- a/release_criteria/r9/9.0_qa_testing_go_no_go.md +++ b/release_criteria/r9/9.0_qa_testing_go_no_go.md @@ -8,7 +8,7 @@ rc: --- # {{ rc.prod }} {{ rc.ver }} QA and Testing : GO / NO GO Status -This document summarizes the GO / NO GO Status for Release of Rocky Linux 8.6 from perspective of the QA / Testing efforts. It is based largely on the Release Criteria (https://wiki.rockylinux.org/team/testing/release_criteria/) as was started as an import of that document. If there are differences between the official Release Critieria document and this document the official Release Criteria document will override and this document shall be updated. +This document summarizes the GO / NO GO Status for Release of Rocky Linux 9.0 from perspective of the QA / Testing efforts. It is based largely on the Release Criteria (https://wiki.rockylinux.org/team/testing/release_criteria/) as was started as an import of that document. If there are differences between the official Release Critieria document and this document the official Release Criteria document will override and this document shall be updated. As a reminder, the objective of a release (major or minor) is to provide a solid Enterprise Linux release that is suitable to: - Meet the needs of end users From 27dbac6eccf69d828dacba1a0acd7f181b6fa896 Mon Sep 17 00:00:00 2001 From: Stack Korora Date: Thu, 18 Aug 2022 18:01:39 -0500 Subject: [PATCH 35/45] Migration from Etherpad to wiki. --- .../r9/9.0_qa_testing_go_no_go.md | 14 ++-- release_criteria/r9/9.0_qa_testing_summary.md | 81 +++++++++++++++---- 2 files changed, 71 insertions(+), 24 deletions(-) diff --git a/release_criteria/r9/9.0_qa_testing_go_no_go.md b/release_criteria/r9/9.0_qa_testing_go_no_go.md index 726ea65..9a851c6 100644 --- a/release_criteria/r9/9.0_qa_testing_go_no_go.md +++ b/release_criteria/r9/9.0_qa_testing_go_no_go.md @@ -19,8 +19,8 @@ As a reminder, the objective of a release (major or minor) is to provide a solid | Category | Proportion | Remaining Items | | ----------------- | ----------------- | --------------- | -| TO_CONFIRM | 0 / 29 (0%) | | -| PASS | 0 / 29 (0%) | | +| TO_CONFIRM | 1 / 29 (3%) | Firmware RAID | +| PASS | 28/ 29 (97%) | | | FAIL_NON_BLOCKING | 0 / 29 (0%) | | | FAIL_BLOCKING | 0 / 29 (0%) | | @@ -52,15 +52,15 @@ Examples... ## Initialization Requirements -- Release-blocking images must boot - PASS - (@neil, @atomicturtle) -- Optical Media Requirements - PASS - (@atomicturtle) +- Release-blocking images must boot - PASS - (@neil, @atomicturtle, @lumarel) +- Optical Media Requirements - PASS - (@neil) - Basic Graphics Mode behaviors - PASS - (@tcooper, virt only, manual) - No Broken Packages - PASS - (@tcooper, scripted, manual) -- Repositories Must Match Upstream - TO_CONFIRM - (@tcooper, manual) +- Repositories Must Match Upstream - PASS - (@tcooper, manual) - Debranding - PASS - (@tcooper, scripted, manual) - Media Consistency Verification - PASS - (@tcooper, scripted, manual) - Packages and Installer Sources - PASS - (@lumarel, semi-automatic, openQA test) -- NAS (Network Attached Storage) - TO_CONFIRM - (@lumarel?, semi-automatic, openQA dual-host test) +- NAS (Network Attached Storage) - PASS - (@stack, manual) - Installation Interfaces - PASS - (@lumarel, @raktajino, @tcooper, semi-automatic, openQA, @atomicturtle, manual?, SCAP) - Minimal Installation - PASS - (@lumarel, @raktajino, @tcooper, semi-automatic , openQA) - Kickstart Installation - PASS - (@label, @tcooper, manual, createhdds) @@ -74,7 +74,7 @@ Examples... ## Cloud Image Requirements -- Images Published to Cloud Providers - FAIL_NON_BLOCKING - (@neil) +- Images Published to Cloud Providers - PASS - (@neil) ## Post-Installation Requirements diff --git a/release_criteria/r9/9.0_qa_testing_summary.md b/release_criteria/r9/9.0_qa_testing_summary.md index e6deff0..0e22f25 100644 --- a/release_criteria/r9/9.0_qa_testing_summary.md +++ b/release_criteria/r9/9.0_qa_testing_summary.md @@ -28,20 +28,7 @@ This document will record a summary of all QA and Testing results for {{ rc.prod This is a list of items that are being INVESTIGATEd further before being assigned a PASS, FAIL, NOTABUG or UPSTREAM status. PLEASE add your MM handle if you are working on this item to minimize duplication of work. More than one handle is allowed but please communicate. -- QA:Testcase Basic Graphics Mode - [INVESTIGATE] - @tcooper -- QA:Testcase Boot Methods Boot Iso - [INVESTIGATE] - @neil -- QA:Testcase Boot Methods DVD - [INVESTIGATE] - @neil -- QA:Testcase Debranding - [INVESTIGATE] - @tcooper -- QA:Testcase Media Consistency Verification - [INVESTIGATE] - @tcooper -- QA:Testcase Media File Conflicts - [INVESTIGATE] - @tcooper -- QA:Testcase Media File Conflicts - [INVESTIGATE] - @tcooper -- QA:Testcase Media File Conflicts - [INVESTIGATE] - @tcooper -- QA:Testcase Media File Conflicts - [INVESTIGATE] - @tcooper -- QA:Testcase Media Repoclosure - [INVESTIGATE] - @tcooper -- QA:Testcase Storage Volume Resize - [INVESTIGATE] - @raktajino -- QA:Testcase Update Image - [INVESTIGATE] - @raktajino -- QA:Testcase boot/install minimal x86_64 over DVD/Bluray - [INVESTIGATE] - @atomicturtle -- QA:Testcase_Mediacheck - [INVESTIGATE] - @tcooper +- QA:Testcase Firmware RAID - [INVESTIGATE] - @neil ## UPSTREAM This is a list of items that have been verified to be replicated UPSTREAM in RHEL {{ rc.9 }} and/or are described clearly in the RHEL 9 Release Notes. @@ -52,16 +39,76 @@ This is a list of items that have been verified to be replicated UPSTREAM in RHE This is a list of items that have been verified to FAIL the QA:Testcase. In addition to recording who did the test please indicate if the item is BLOCKING release or not. - QA:Testcase_Some_Testcase - [FAIL] - @your_mm_handle - +- RC1 - The blue side banner background at the left side of the Anaconda installer display is not displaying correctly in virt-viewer. [minor BUG] - @alangm ## NOTABUG This is a list of items that have been verified as less than optimal but are expected and NOTABUG. -- QA:Testcase_Some_Testcase - [NOTABUG] - @your_mm_handle - +- QA:Testcase Basic Graphics Mode - [DEPRICATED] - @tcooper ## PASS This is a list of item that have been verified as PASSing the QA:Testcase named (or proposed). -- QA:Testcase_Some_Testcase - [PASS] - @your_mm_handle - +- QA:Testcase_Debranding - [PASS] - @tcooper - preliminary pass based on Peridot yumrepofs - https://rpa.st/PZKA +- RC1 - QA:Testcase System Services - [PASS] - @lumarel - tested with boot-iso and every base environment +- RC1 - Basic Installs - [PASS] - @lumarel - Installs of all base environments on x86_64 without additional optional package groups +- RC1 - QA:Testcase Packages and Installer Sources - [PASS] - @raktajino - tested against RC1-boot-iso in openQA, all package sets pass. +- RC1 - QA:Testcase GNOME UI Functionality - [PASS] - @lumarel - Nothing to mention +- RC1 - QA:Testcase SELinux Errors on Server - [PASS] - @lumarel - no errors +- RC1 - QA:Testcase SELinux Errors on Desktop- [PASS] - @lumarel - no errors +- RC1 - QA:Testcase Artwork and Assets - [PASS] - @lumarel - everything looks correct +- RC1 - QA:Testcase Minimal Installation - [PASS] - @raktajino - nothing to report +- RC1 - QA: Testcase Boot Methods Boot Iso - [PASS] - @alangm - minimal system, graphical server & basic workstation - nothing to report except as above ^^^ +- RC1 - Minimal install on QEMU/KVM passing - [PASS] - @boris - reboot takes slightly longer on the first reboot +- RC1 - Repo Health Check - [PASS] - @lumarel - sqlite database files for repo metadata is missing up to now, which will hinder some applications from syncing the sources - are implemented +- RC1 - Repo Health Check - [PASS] - @lumarel - metadata signing not implemented up to now - looks to be implemented as well now +- RC1 - Repo Health Check - [PASS] - @lumarel - Repos look good now excl. BaseOS of aarch64, ppc64le, s390x, where the boot image is missing, which is referenced in the .treeinfo +- RC1 - Redesign looks to be broken (Post-Install error, effects some of the install groups) - [PASS] - @lumarel +- RC1 - Workstation install with known pesign bug - [PASS] - @alangm - error window "Exit Installer" button froze Anaconda instead of exit so had to initiate shutdown from cockpit - @lumarel - fixed! +- RC1 - Building packages with mock - [PASS] - @hbjy - building works with the mock config from sokels' repo (will be upstreamed for release) +- RC1 - Install apache + mariadb-server - [PASS] - @sspencerwire - mariadb-secure-installation works, nextcloud install on top as well +- RC1 - Install works on macOS M1 - [PASS] - @netwaze, @gardo984, @jordan_c +- RC1 - Install on baremetal went well - [PASS] - @alangm - on a Dell T630 +- RC1 - Install went well - [PASS] - @kars88 - there was some flickering during the install +- RC1.1 dvd - STIG security - [PASS] - @atomicturtle - packages were not complete in RC1 dvd, but are now in RC1.1 +- RC1.1 - Basic installs work - [PASS] - @mkahric - Virtualbox on Windows (UEFI w/o SB), Workstation Player on Windows (UEFI w/ SB), Hyper-V (UEFI w/ SB) +- RC1.1 - docker installation with ansible works - [PASS] - @gardo984 +- RC1 - Repo Health Check - [PASS] - @lumarel - all repos (x86_64, aarch64, ppc64le, s390x) look good now, all metadata is healthy and all package checksums match +- RC1.1 - QA:Testcase Multimonitor Setup - [PASS] - @lumarel - looks good +- RC1.1 - QA:Testcase Artwork and Assets - [PASS] - @lumarel - no issues found +- RC1.1 - QA:Testcase GNOME UI Functionality - [PASS] - @lumarel - everything behaves as it should +- RC1.1 - QA:Testcase SELinux Errors on Desktop - [PASS] - @lumarel - After an hour of usage no errors +- RC1.1 - QA:Testcase Keyboard Layout - [PASS] - @lumarel - OpenQA tests and manual tests show normal behavior +- RC1.1 - QA:Testcase SELinux Errors on Server - [PASS] - @lumarel - after more than an hour of runtime with several installs and application setups (missing, Katello bootstrap, which once brought some SELinux errors) +- RC1.1 - QA:Testcase System Services - [PASS] - @lumarel - no errors +- RC1.1 - QA: Testcase Installation Interfaces - [PASS] - @raktajino - failures on the Welcome Tour screen because we haven't handled it yet. Everything else seems good. +- RC1.1 - QA:Testcase Installer Help - [PASS] - @raktajino - no errors +- RC1.1 - QA: Testcase Disk Layouts - [PASS] - @raktajino - Assorted post-install failures due to either mismatch between console login and GUI login or unhandled Welcome Tour. +- RC1.1 - QA: Testcase Kickstart Installation - [PASS] - @raktajino - Executed manually with virt-install. Nothing to report. +- RC1.1 - QA: Testcase Update Image - [PASS] - @raktajino - failure on _console_wait_login as the GUI login screen is displayed. Update image is applied as expected. +- RC1.1 - QA:Testcase VNC Graphics Mode - [PASS] - @tcooper +- RC1.1 - QA:Testcase Basic Package installs - [PASS] - @lumarel - All package sets and services tested which are currently on the blocking list +- RC1.1 - QA:Testcase Application Functionality - [PASS] - @lumarel - everything looks good +- RC1/RC1.1 - QA:Testcase_Mediacheck - [PASS] - @tcooper +- RC1/RC1.1 - QA:Testcase Media Consistency Verification - [PASS] - @tcooper +- RC1.1 - QA:Testcase Media Repoclosure - [PASS] - @tcooper +- RC1.1 - QA:Testcase Media File Conflicts - [PASS] - @tcooper +- RC1.1 - QA: Testcase Bootloader Disk Selection - [PASS] - @raktajino +- RC1.1 - QA: Testcase Installer Translations - [PASS] - @raktajino +- RC1.1 - QA: Testcase Network Attached Storage - [PASS] - @Stack +- QA: Testcase Boot Methods Boot ISO - [PASS] - all of us <3 +- QA: Testcase Boot Methods DVD - [PASS] - todos nosotros <3 +- QA:Testcase Cloud Images Published - [PASS] - @neil +- QA:Testcase Storage Volume Resize - [PASS] - @raktajino, @tcooper +- QA:Testcase Repocompare - [PASS] - @tcooper +- QA:Testcase Basic install boot final ISO - [PASS] - @lumarel +- QA:Testcase Basic install minimal final ISO - [PASS] - @lumarel +- QA:Testcase Basic installs dvd final ISO - [PASS] - @lumarel - install of every base environment + custom operating system with all options enabled +- OpenQA tests with final ISOs - [PASS] - @lumarel - boot-iso 2/2 | minimal-iso 2/2 | minimal 22/26 (failing are all false-positive) | server 21/26 (failing are all false-positive) | package-set 5/5 | workstation 17/26 (failing are all false-positive) | graphical-server 17/26 (failing are all false-positive) | universal 34/43 (failing are all false-positive) +- QA:Testcase Minimal Installation - [PASS] - @neil ## OTHER NOTABLE ITEMS - +- QA:Testcase Module Streams - @lumarel - 9.0 currently does not have any Stream metadata and only the base Streams +- Repo Health Check - @lumarel - yumrepofs is not sending content lengths, which make syncing with Katello impossible +- Missing package in package group additional-devel - @stack - dnf -y group install additional-devel: No match for group package "hmaccalc" - this is an old EL bug +- Hosting a nfs server in macOS, and using that in Virtualbox for OpenQA shares is a pain - @tcooper From 7944f3e4f72f2b56d7a6ec17e12dc8aaa3df2ecf Mon Sep 17 00:00:00 2001 From: Stack Korora Date: Thu, 18 Aug 2022 18:07:29 -0500 Subject: [PATCH 36/45] remove bad link --- release_criteria/r9/9.0_qa_testing_summary.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/release_criteria/r9/9.0_qa_testing_summary.md b/release_criteria/r9/9.0_qa_testing_summary.md index 0e22f25..6e0e209 100644 --- a/release_criteria/r9/9.0_qa_testing_summary.md +++ b/release_criteria/r9/9.0_qa_testing_summary.md @@ -49,7 +49,7 @@ This is a list of items that have been verified as less than optimal but are exp ## PASS This is a list of item that have been verified as PASSing the QA:Testcase named (or proposed). -- QA:Testcase_Debranding - [PASS] - @tcooper - preliminary pass based on Peridot yumrepofs - https://rpa.st/PZKA +- QA:Testcase_Debranding - [PASS] - @tcooper - preliminary pass based on Peridot yumrepofs - RC1 - QA:Testcase System Services - [PASS] - @lumarel - tested with boot-iso and every base environment - RC1 - Basic Installs - [PASS] - @lumarel - Installs of all base environments on x86_64 without additional optional package groups - RC1 - QA:Testcase Packages and Installer Sources - [PASS] - @raktajino - tested against RC1-boot-iso in openQA, all package sets pass. From 1a6d80cf8e6204955fe4dbc8470385db98b50ec4 Mon Sep 17 00:00:00 2001 From: Lukas Magauer <42647570+lumarel@users.noreply.github.com> Date: Fri, 14 Oct 2022 01:32:51 +0200 Subject: [PATCH 37/45] Add Identity Management testcase (#45) Co-authored-by: lumarel --- QA/Testcase_Post_Identity_Management.md | 68 +++++++++++++++++++++++ qa_test_cases.md | 1 + release_criteria/r8/8_release_criteria.md | 8 +++ release_criteria/r9/9_release_criteria.md | 8 +++ 4 files changed, 85 insertions(+) create mode 100644 QA/Testcase_Post_Identity_Management.md diff --git a/QA/Testcase_Post_Identity_Management.md b/QA/Testcase_Post_Identity_Management.md new file mode 100644 index 0000000..31fe9cd --- /dev/null +++ b/QA/Testcase_Post_Identity_Management.md @@ -0,0 +1,68 @@ +--- +title: QA:Testcase Identity Management +author: Lukas Magauer +revision_date: 2022-10-08 +rc: + prod: Rocky Linux + ver: + - 8 + - 9 + level: Final +--- + +!!! info "Release relevance" + This Testcase applies the following versions of {{ rc.prod }}: {% for version in rc.ver %}{{ version }}{% if not loop.last %}, {% endif %}{% endfor %} + +!!! info "Associated release criterion" + This test case is associated with the [Release_Criteria#packages-and-module-installation](9_release_criteria.md#packages-and-module-installation) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion. + +## Description + +Setting up a IdM system (FreeIPA) and using it's functionality leverages not also a lot of the packages in the official repos, it also tests quite a lot of used functions a corporate environment. This installatation will host it's own dns server for more generic testing without relying on the individual infrastructure of the environment. + +## Requirements + +- A freshly provisioned system (no other functions are allowed on this system except running the IdM services) +- IPv4 network with unmanaged domain name (installer will check for dns servers) and unmanaged reverse dns network (in my case here 10.30.30.0/24 and ipa1.network) +- In the case of this writeup the external dns server has the domain `example.com`, this has to have a entry for `r8-ipa1-dev.example.com` (this could also be replaced by a entry in the `/etc/hosts` file if no external dns server should be involved) + +## Setup + +1. `dnf module enable idm:DL1` +2. `dnf module install idm:DL1/dns` +3. `ipa-server-install` + + - Do you want to configure integrated DNS (BIND)? [no]: yes + - Server host name [r8-ipa1-dev.example.com]: r8-ipa1-dev.example.com + - Please confirm the domain name [ipa1.network]: ipa1.network + - Please provide a realm name [IPA1.NETWORK]: IPA1.NETWORK + - Directory Manager password: `` + Password (confirm): `` + - IPA admin password: `` + Password (confirm): `` + - Please provide the IP address to be used for this host name: 10.30.30.1 + - Enter an additional IP address, or press Enter to skip: `leave empty` + - Do you want to configure DNS forwarders? [yes]: yes + - Do you want to configure these servers as DNS forwarders? [yes]: yes + - Enter an IP address for a DNS forwarder, or press Enter to skip: `leave empty` + - Do you want to search for missing reverse zones? [yes]: yes + - NetBIOS domain name [IPA1]: IPA1 + - Do you want to configure chrony with NTP server or pool address? [no]: yes + - Enter NTP source server addresses separated by comma, or press Enter to skip: `leave empty` + - Enter a NTP source pool address, or press Enter to skip: pool.ntp.org + - Continue to configure the system with these values? [no]: yes + +4. `firewall-cmd --add-service={freeipa-4,dns} --permanent` +5. `firewall-cmd --add-service={freeipa-4,dns}` + +## How to test + +1. Make sure Kerberos works, by running `kinit admin` and `klist` +2. Make sure the webfrontend is reachable and login works +3. Furthermore you can also attach another system (DNS + connecting via SSSD) + +## Expected Results + +After installation all services should be available and work correctly. + +{% include 'testing/qa_testcase_bottom.md' %} diff --git a/qa_test_cases.md b/qa_test_cases.md index 02a5492..dc65d42 100644 --- a/qa_test_cases.md +++ b/qa_test_cases.md @@ -59,6 +59,7 @@ This page lists all test cases in work and who is working on them... | Dual Monitor Setup (Desktop Only) | [QA:Testcase Multimonitor Setup](Testcase_Post_Multimonitor_Setup.md) | @lumarel | manual guide documented | | Artwork and Assets (Server and Desktop) | [QA:Testcase Artwork and Assets](Testcase_Post_Artwork_and_Assets.md) | @lumarel | implemented in openQA, additionally documented for manual inspection | | Packages and Module Installation | [QA:Testcase Basic Package installs](Testcase_Post_Package_installs.md)
[QA:Testcase Module Streams](Testcase_Post_Module_Streams.md) | @lumarel | partly implemented in openQA, manual guide documented | +| Identity Management (FreeIPA) | [QA:Testcase Identity Management](Testcase_Post_Identity_Management.md) | @lumarel | manual guide documented, PR open for openQA implementation | {% include 'content_bottom.md' %} diff --git a/release_criteria/r8/8_release_criteria.md b/release_criteria/r8/8_release_criteria.md index 8dfde94..04aa5a9 100644 --- a/release_criteria/r8/8_release_criteria.md +++ b/release_criteria/r8/8_release_criteria.md @@ -266,4 +266,12 @@ Packages (non-module) should be able to be installed without conflicts or depend - [QA:Testcase Basic Package installs](Testcase_Post_Package_installs.md) - [QA:Testcase Module Streams](Testcase_Post_Module_Streams.md) +#### Identity Management Server Setup + +It should be possible to setup a IdM server (FreeIPA), use it's functionality and connect clients. + +??? tldr "References" + - Test cases + - [QA:Testcase Identity Management](Testcase_Post_Identity_Management.md) + {% include 'testing/rc_content_bottom.md' %} diff --git a/release_criteria/r9/9_release_criteria.md b/release_criteria/r9/9_release_criteria.md index 4fef4c4..f49ff3c 100644 --- a/release_criteria/r9/9_release_criteria.md +++ b/release_criteria/r9/9_release_criteria.md @@ -266,4 +266,12 @@ Packages (non-module) should be able to be installed without conflicts or depend - [QA:Testcase Basic Package installs](Testcase_Post_Package_installs.md) - [QA:Testcase Module Streams](Testcase_Post_Module_Streams.md) +#### Identity Management Server Setup + +It should be possible to setup a IdM server (FreeIPA), use it's functionality and connect clients. + +??? tldr "References" + - Test cases + - [QA:Testcase Identity Management](Testcase_Post_Identity_Management.md) + {% include 'testing/rc_content_bottom.md' %} From 527bfd7646ee5925538cf67aa57dae57df6b006e Mon Sep 17 00:00:00 2001 From: Lukas Magauer <42647570+lumarel@users.noreply.github.com> Date: Fri, 14 Oct 2022 01:35:00 +0200 Subject: [PATCH 38/45] Add toolbox dev box docs (#46) Co-authored-by: lumarel --- dev_guides/wiki_development_boxes.md | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/dev_guides/wiki_development_boxes.md b/dev_guides/wiki_development_boxes.md index eee9564..dacd8fa 100644 --- a/dev_guides/wiki_development_boxes.md +++ b/dev_guides/wiki_development_boxes.md @@ -14,15 +14,25 @@ There are several ways how to setup your development environment, here are the c For now here is the link to [Trevor's vagrant box setup](https://github.com/tcooper/rocky-linux-wikibox), this might be merged here in the future! -## WSL +## Manual setup for WSL and toolbox + +### WSL specific - Make sure the Windows Subsystem for Linux is enabled on your machine - Download the [Cloud image](https://github.com/rocky-linux/sig-cloud-instance-images/actions/workflows/build.yml) - Get the actual disk out of the image (extract rocky-8.5-docker.tar) - Run `wsl --import --version 2` to create the machine - Run `wsl -d ` to initially login to the machine + +### toolbox specific + +- Make sure you have `toolbox` installed on your system +- Create a toolbox `toolbox create rocky-wiki` (on Rocky Linux 8 and 9 machines this will create either a Rocky Linux 8 or 9 toolbox container) + +### Container setup for both + - Run `dnf -y update` to update the system -- Run `dnf -y install git python39-pip` to install Python 3.9 and pip +- Run `dnf -y install git python39-pip` to install Python 3.9 and pip (Python 3.9 is default for Rocky Linux 9, the package is called `python3-pip` there) - Run `python3.9 -m pip install -U pip` to update pip - Clone the repo `git clone ` - And get into the folder of the repo `cd ` @@ -32,6 +42,8 @@ For now here is the link to [Trevor's vagrant box setup](https://github.com/tcoo To develop then, the easiest way is to use VS Code with the [Remote - WSL](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-wsl), where you have to open the repo in the container (click on the Remote symbol on the lower left, 'Open folder in WSL...'). +For toolbox just place the repo inside your user profile and you will be able to access it with VS Code inside and outside of the toolbox container. + And finally run `mkdocs serve 2>&1 | tee ./mkdocs.serve.log` in the terminal of this VS Code session. Then you are ready to start changing stuff! ## Docker From 69b97e461a17edd52a71e062249bf18d0081c96d Mon Sep 17 00:00:00 2001 From: Al Bowles Date: Thu, 3 Nov 2022 12:18:32 -0500 Subject: [PATCH 39/45] Added section on publishing your key to a keyserver --- dev_guides/commit_signing.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/dev_guides/commit_signing.md b/dev_guides/commit_signing.md index 4850280..321afa0 100644 --- a/dev_guides/commit_signing.md +++ b/dev_guides/commit_signing.md @@ -92,6 +92,14 @@ Find the *primary keypair* and import it (preferably into an ephemeral system li # User or workspace setting "git.enableCommitSigning": true +# Upload your public key to a keyserver + + gpg --keyserver pgp.mit.edu --send-keys 0xDEADB33FBAD1D3A + +# Verify your key has been published + + gpg --keyserver pgp.mit.edu --search-key my@email.addr + # References [OpenPGP Best Practices](https://riseup.net/en/security/message-security/openpgp/best-practices#key-configuration)
[Github: Signing Commits](https://docs.github.com/en/enterprise-server@3.5/authentication/managing-commit-signature-verification/signing-commits)
From 7e57d699fd3f2c5025222ed9a3605fe96c377698 Mon Sep 17 00:00:00 2001 From: Trevor Cooper Date: Thu, 13 Apr 2023 21:47:53 -0700 Subject: [PATCH 40/45] update testing team members --- index.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/index.md b/index.md index 722a421..1b0feb2 100644 --- a/index.md +++ b/index.md @@ -1,6 +1,6 @@ --- title: QA Testing Team -revision_date: 2022-04-01 +revision_date: 2023-04-13 rc: prod: Rocky Linux --- @@ -9,9 +9,9 @@ rc: | Role | Name | Email | Mattermost Name | IRC Name | | -------------- | --------------- | ----------------------- | ------------------ | --------- | -| Testing Lead | Trevor Cooper | tcooper@rockylinux.org | @tcooper | | | Testing Lead | Chris Stackpole | stack@rockylinux.org | @stack | | | Testing Team | Al Bowles | | @raktajino | raktajino | +| Testing Team | Trevor Cooper | tcooper@rockylinux.org | @tcooper | | | Testing Team | Lukas Magauer | lukas@magauer.eu | @lumarel | | | Testing Team | Alan Marshall | | @alangm | alangm | From ffca9600278687d4c2b3f44e1b3b9a2ca58d9e10 Mon Sep 17 00:00:00 2001 From: Trevor Cooper Date: Sun, 23 Apr 2023 12:43:48 -0700 Subject: [PATCH 41/45] add openQA developer examples --- dev_guides/.pages | 4 + dev_guides/openqa_access.md | 124 +++++ dev_guides/openqa_cli_post_examples.md | 95 ++++ ...penqa_clone_custom_git_refspec_examples.md | 474 ++++++++++++++++++ dev_guides/openqa_clone_job_examples.md | 249 +++++++++ 5 files changed, 946 insertions(+) create mode 100644 dev_guides/openqa_access.md create mode 100644 dev_guides/openqa_cli_post_examples.md create mode 100644 dev_guides/openqa_clone_custom_git_refspec_examples.md create mode 100644 dev_guides/openqa_clone_job_examples.md diff --git a/dev_guides/.pages b/dev_guides/.pages index 6ecf067..9fb60c1 100644 --- a/dev_guides/.pages +++ b/dev_guides/.pages @@ -2,3 +2,7 @@ nav: - Development Box Setup: wiki_development_boxes.md - Git Commit Signing: commit_signing.md + - openQA - Access: openqa_access.md + - openQA - openqa-cli POST Examples: openqa_cli_post_examples.md + - openQA - openqa-clone-job Examples: openqa_clone_job_examples.md + - openQA - openqa-clone-custom-git-refspect Examples: openqa_clone_custom_git_refspec_examples.md diff --git a/dev_guides/openqa_access.md b/dev_guides/openqa_access.md new file mode 100644 index 0000000..9577335 --- /dev/null +++ b/dev_guides/openqa_access.md @@ -0,0 +1,124 @@ +--- +title: openQA - Access +author: Trevor Cooper +revision_date: 2023-04-22 +rc: + prod: Rocky Linux + level: Final +--- + +## System Requirements + +To complete any of the examples below you will need access to a system providing the openQA client. Typically that will be a Fedora based system/container with the `openqa-client` package and it's (~239) dependencies installed. + + +## Access Requirement + +### API `GET` access + +The {{ rc.prod }} openQA system allows unrestricted public access via it's web interface or using the `openqa-client` for `GET` operations against the API. + +### API `POST` access + +In order to use the openQA client to interact with the {{ rc.prod }} openQA system for `POST` operations the following are required: + +- an account in good standing in the [{{ rc.prod }} Account Services](https://accounts.rockylinux.org) system, +- authorization for API `POST` access from the {{ rc.prod }} Testing Team, and +- an [openQA API key](http://open.qa/docs/#_authentication) generated on the {{ rc.prod }} openQA system. + + +## Configuring your openqa client + +Per the openqa client command help you can configure your client to use your API key in a number of ways. + +The following example shows how to configure your client by the most common method used. It's possible to configure +multiple openqa client API keys in this way. + +``` +$ mkdir -p ~/.config/openqa + +$ vim ~/.config/openqa/client.conf + +$ cat ~/.config/openqa/client.conf +[localhost] +key = your_localhost_api_key +secret = your_localhost_api_secret +[openqa.rockylinux.org] +key = your_api_key +secret = your_api_secret +``` + + +## Testing your openqa client installation + +``` +$ openqa-cli api --host http://openqa.rockylinux.org --pretty jobs/1 +{ + "job" : { + "assets" : { + "iso" : [ + "Rocky-8.6-x86_64-boot.iso" + ] + }, + "assigned_worker_id" : 2, + "blocked_by_id" : null, + "children" : { + "Chained" : [], + "Directly chained" : [], + "Parallel" : [] + }, + "clone_id" : null, + "group" : "Rocky", + "group_id" : 2, + "has_parents" : 0, + "id" : 1, + "name" : "rocky-8.6-boot-iso-x86_64-Build-8.6-boot-iso--20221110.223812.0-install_default@64bit", + "parents" : { + "Chained" : [], + "Directly chained" : [], + "Parallel" : [] + }, + "parents_ok" : 1, + "priority" : 10, + "result" : "failed", + "settings" : { + "ARCH" : "x86_64", + "ARCH_BASE_MACHINE" : "64bit", + "BACKEND" : "qemu", + "BUILD" : "-8.6-boot-iso--20221110.223812.0", + "DESKTOP" : "gnome", + "DISTRI" : "rocky", + "FLAVOR" : "boot-iso", + "GRUB" : "ip=dhcp", + "HDDSIZEGB" : "15", + "ISO" : "Rocky-8.6-x86_64-boot.iso", + "MACHINE" : "64bit", + "NAME" : "00000001-rocky-8.6-boot-iso-x86_64-Build-8.6-boot-iso--20221110.223812.0-install_default@64bit", + "PACKAGE_SET" : "default", + "PART_TABLE_TYPE" : "mbr", + "POSTINSTALL" : "_collect_data", + "QEMUCPU" : "Nehalem", + "QEMUCPUS" : "2", + "QEMURAM" : "3072", + "QEMUVGA" : "virtio", + "QEMU_VIRTIO_RNG" : "1", + "TEST" : "install_default", + "TEST_SUITE_NAME" : "install_default", + "TEST_TARGET" : "ISO", + "VERSION" : "8.6", + "WORKER_CLASS" : "qemu_x86_64" + }, + "state" : "done", + "t_finished" : "2022-11-10T22:44:19", + "t_started" : "2022-11-10T22:38:12", + "test" : "install_default" + } +} +``` + +## References + +[openQA Documentation](http://open.qa/documentation/) + + +{% include 'content_bottom.md' %} diff --git a/dev_guides/openqa_cli_post_examples.md b/dev_guides/openqa_cli_post_examples.md new file mode 100644 index 0000000..c113d5a --- /dev/null +++ b/dev_guides/openqa_cli_post_examples.md @@ -0,0 +1,95 @@ +--- +title: openQA - openqa-cli POST Examples +author: Trevor Cooper +revision_date: 2023-04-23 +rc: + prod: Rocky Linux + level: Final +--- + +# openqa-cli POST Examples + +This page will provide a brief overview of some basic `openqa-cli` `POST` examples. + +## System / Access Requirements + +To complete any of the examples please complete the API `POST` Access steps outlined in the [openQA - Access](openqa_access.md) document. + +## Basic POST + +A basic `POST` can be used for any of the default test suites for the various {{ rc.prod }} media that are made available. The following examples show some of these standard `POST`s that are commonly used by our team and will be used to demonstrate how some minor variations + +### FLAVOR=boot-iso + +This first `POST` is the most basic, simply providing the minimal set of variables required to trigger the standard test suite for the {{ rc.prod }} 9.1 boot ISO on openqa workers for the `x86_64` architecture. All tests of the test suite are predetermined and configure on the openQA server. Since the boot ISO doesn't contain any packages this test suite is effectively a network install from standard {{ rc.prod }} repository servers and/or mirrors. + +```bash +$ openqa-cli api -X POST isos ISO=Rocky-9.1-x86_64-boot.iso ARCH=x86_64 \ + DISTRI=rocky FLAVOR=boot-iso VERSION=9.1 CURRREL=9 BUILD=20230409-Rocky-9.1-x86_64.0 +``` + +### FLAVOR=minimal-iso + +This `POST` demonstrates how a different media type, in this case the minimal ISO, for an alternate {{ rc.prod }} version, in this case {{ rc.prod }} 8.7, can be triggered. As can be seen by this and the previous `POST` the `BUILD` variable will typically be designate the date, version and architecture of the test suite. Since the minimal ISO contains all packages required to conduct a ***minimal*** install of {{ rc.prod }} that is the behavior of this test suite. + +```bash +$ openqa-cli api -X POST isos ISO=Rocky-8.7-x86_64-minimal.iso ARCH=x86_64 \ + DISTRI=rocky FLAVOR=minimal-iso VERSION=8.7 CURRREL=8 BUILD=20230409-Rocky-8.7-x86_64.0 +``` + +### FLAVOR=package-set + +This `POST` demonstrates specification of the final normal media type, the dvd ISO, along with what is called a `FLAVOR`, in this case `package-set` for the `x86_64` architecture and {{ rc.prod }} 9.1. Since the dvd ISO contains all of the packages available at release of a specific version or {{ rc.prod }} the `package-set` test suite will test installation of all primary installation types of {{ rc.prod }} not included in the `minimal-iso` test suite above. + +```bash +$ openqa-cli api -X POST isos ISO=Rocky-9.1-20221214.1-x86_64-dvd.iso ARCH=x86_64 \ + DISTRI=rocky FLAVOR=package-set VERSION=9.1 CURRREL=9 BUILD=20230409-Rocky-9.1-x86_64.0 +``` + +These three test suites provide for the minimal testing of all ISOs produced for a given release of {{ rc.prod }}. + + +## Advanced POST + +In addition to the [Basic POSTs](#basic-post) described above there are additional default test suites that use the dvd ISO media and include substantially more test cases. Those include: + +- installing in graphical, text and serial console +- installation for standard BIOS and UEFI +- validation of the Anaconda help system +- disk layout variations including LVM, RAID, partition shrink and/or grow, iSCSI and LUKS +- PXE installation from various network sources +- installation in various languages + +Standard `POST`s for these test suites is very similar to the basic POSTs above and are shown below... + +### FLAVOR=dvd-iso + +```bash +$ openqa-cli api -X POST isos ISO=Rocky-9.1-20221214.1-x86_64-dvd.iso ARCH=x86_64 \ + DISTRI=rocky FLAVOR=dvd-iso VERSION=9.1 CURRREL=9 BUILD=20230409-Rocky-9.1-x86_64.0 +``` + +### FLAVOR=universal + +```bash +$ openqa-cli api -X POST isos ISO=Rocky-9.1-20221214.1-x86_64-dvd.iso ARCH=x86_64 \ + DISTRI=rocky FLAVOR=universal VERSION=9.1 CURRREL=9 BUILD=20230409-Rocky-9.1-x86_64.0 +``` + + +## Collection of test suites by BUILD + +A feature of openQA is that for a given job group test suites which use the same `BUILD` identifier are collected into a single view in the web UI. + +![openQA Home View...](/assets/images/openqa_home_view.png){ loading=lazy } + +Thus, the examples show above which all use `BUILD=20230409-Rocky-9.1-x86_64.0` are all shown in a single view. Additionally, that view is accessible via a predictable URI, for example [`https://openqa.rockylinux.org/tests/overview?build=20230409-Rocky-9.1-x86_64.0`](https://openqa.rockylinux.org/tests/overview?build=20230409-Rocky-9.1-x86_64.0) as shown below... + +![openQA Build View...](/assets/images/openqa_build_view.png){ loading=lazy } + +## References + +[openQA Documentation](http://open.qa/documentation/) + + +{% include 'content_bottom.md' %} diff --git a/dev_guides/openqa_clone_custom_git_refspec_examples.md b/dev_guides/openqa_clone_custom_git_refspec_examples.md new file mode 100644 index 0000000..09e1f77 --- /dev/null +++ b/dev_guides/openqa_clone_custom_git_refspec_examples.md @@ -0,0 +1,474 @@ +--- +title: openQA - openqa-clone-custom-refspec Examples +author: Trevor Cooper +revision_date: 2023-04-23 +rc: + prod: Rocky Linux + level: Final +--- + +# openqa-clone-custom-git-refspec Examples + +This page will provide a brief overview of basic and advanced job cloning using the `openqa-clone-custom-git-refspec` command. + +At a high level `openqa-clone-custom-git-refspec` can be viewed as a mechanism to test PRs for openQA tests directly in a {{ rc.prod }} openQA instance with making changes to the default configuration. As such, it can support testing of PRs that change test code and needles as long as changes to `templates.fif.json` are not also required. A combination of `openqa-clone-custom-git-refspec` and `openqa-clone-job` (which is actually used by `openqa-clone-custom-git-refspec` under the hood) can be used for some cases where `POST` variables are pre-defined in `templates.fif.json`. + +## System / Access Requirements + +To complete any of the examples please complete the API `POST` Access steps outlined in the [openQA - Access](openqa_access.md) document. + +## Basic `openqa-clone-custom-git-refspec` + +The following example demonstrates the testing of an open Github pull request in the {{ rc.prod }} openQA production system. The PR only changes test code and does not supply updated needles for the test. + +### Github PR information + +***NOTE: The Github CLI tool (`gh`) is used to display PR information statically in this guide.*** + +``` +➜ os-autoinst-distri-rocky git:(develop) gh pr view 168 +Serial console install #168 +Merged • AlanMarshall wants to merge 1 commit into develop from serial_console • about 27 days ago ++5 -2 • No checks +Reviewers: akatch (Approved), tcooper (Approved), lumarel (Requested) +Labels: priority: medium, type: bug, test suite + + + Network is enabled by default at v9 so requires conditional code to handle multiple versions. + Tested for 9.1, 8.7 & 8.8: + + openqa-cli api -X POST isos ISO=Rocky-9.1-20221214.1-x86_64-dvd.iso ARCH=x86_64 DISTRI=rocky FLAVOR=universal + VERSION=9.1 BUILD=-"$(date +%Y%m%d.%H%M%S).0"-9.1-20221214.1-universal TEST=install_serial_console + openqa-cli api -X POST isos ISO=Rocky-8.7-x86_64-dvd1.iso ARCH=x86_64 DISTRI=rocky FLAVOR=universal VERSION=8.7 BUILD=- + "$(date +%Y%m%d.%H%M%S).0"-8.7-20221110-universal TEST=install_serial_console + openqa-cli api -X POST isos ISO=Rocky-8.8-x86_64-dvd1.iso ARCH=x86_64 DISTRI=rocky FLAVOR=universal VERSION=8.8 BUILD=- + "$(date +%Y%m%d.%H%M%S).0"-8.8-lookahead-universal TEST=install_serial_console + + Result: Tests pass. + Also confirm that all main hub check boxes are checked and user test created prior to start of installation. + Fixes Issue #102 + +View this pull request on GitHub: https://github.com/rocky-linux/os-autoinst-distri-rocky/pull/168 +``` + +Above is the information provided in the original PR and it includes tests performed in Alan's openQA development system. We can rerun failing tests in the {{ rc.prod }} openQA system after identifying an appropriate job ID for each Rocky Linux version we are testing. For this example the openQA WebUI was used to find appropriate test IDs to clone. + +### Run `openqa-clone-custom-git-refspec` in `--verbose --dry-run` mode + +In practice it is useful to run `openqa-clone-custom-git-refspec` in `--verbose` and `--dry-run` mode to observe it's behavior even for the Basic cases... + +``` +$ openqa-clone-custom-git-refspec --verbose --dry-run \ + https://github.com/rocky-linux/os-autoinst-distri-rocky/pull/168 \ + https://openqa.rockylinux.org/tests/16080 2>&1 | tee pr-168 +``` + +***NOTE: The full output of `openqa-clone-custom-git-refspece` will not be shown here.*** + +``` ++ shift ++ true ++ case "$1" in ++ dry_run=true ++ shift ++ true ++ case "$1" in ++ shift ++ break ++ job_list=https://openqa.rockylinux.org/tests/16080 ++ [[ -z '' ]] ++ first_arg=https://github.com/rocky-linux/os-autoinst-distri-rocky/pull/168 ++ [[ https://github.com/rocky-linux/os-autoinst-distri-rocky/pull/168 == *\p\u\l\l* ]] ++ pr_url=https://github.com/rocky-linux/os-autoinst-distri-rocky/pull/168 ++ target_repo_part=https://github.com/rocky-linux/os-autoinst-distri-rocky ++ pr=168 ++ pr=168 ++ [[ -z '' ]] ++ pr_url=https://api.github.com/repos/rocky-linux/os-autoinst-distri-rocky/pulls/168 +++ eval 'curl -s https://api.github.com/repos/rocky-linux/os-autoinst-distri-rocky/pulls/168' ++++ curl -s https://api.github.com/repos/rocky-linux/os-autoinst-distri-rocky/pulls/168 + +...... + +++ jq -r '.NEEDLES_DIR | select (.!=null)' ++ old_needledir= ++ local needles_dir= ++ needles_dir=rocky/needles ++ local repo_branch=AlanMarshall/os-autoinst-distri-rocky#serial_console ++ local test_suffix=@AlanMarshall/os-autoinst-distri-rocky#serial_console ++ local build=AlanMarshall/os-autoinst-distri-rocky#168 ++ local casedir=https://github.com/AlanMarshall/os-autoinst-distri-rocky.git#serial_console ++ local GROUP=0 ++ local dry_run=true ++ local scriptdir +++ dirname /usr/bin/openqa-clone-custom-git-refspec ++ scriptdir=/usr/bin ++ local 'cmd=true /usr/bin/openqa-clone-job --skip-chained-deps --parental-inheritance --within-instance "https://openqa.rockylinux.org" "15973" _GROUP="0" TEST+="@AlanMarshall/os-autoinst-distri-rocky#serial_console" BUILD="AlanMarshall/os-autoinst-distri-rocky#168" CASEDIR="https://github.com/AlanMarshall/os-autoinst-distri-rocky.git#serial_console" PRODUCTDIR="os-autoinst-distri-rocky" NEEDLES_DIR="rocky/needles"' ++ [[ 0 -ne 0 ]] ++ [[ -n '' ]] ++ eval 'true /usr/bin/openqa-clone-job --skip-chained-deps --parental-inheritance --within-instance "https://openqa.rockylinux.org" "15973" _GROUP="0" TEST+="@AlanMarshall/os-autoinst-distri-rocky#serial_console" BUILD="AlanMarshall/os-autoinst-distri-rocky#168" CASEDIR="https://github.com/AlanMarshall/os-autoinst-distri-rocky.git#serial_console" PRODUCTDIR="os-autoinst-distri-rocky" NEEDLES_DIR="rocky/needles"' +++ true /usr/bin/openqa-clone-job --skip-chained-deps --parental-inheritance --within-instance https://openqa.rockylinux.org 15973 _GROUP=0 TEST+=@AlanMarshall/os-autoinst-distri-rocky#serial_console BUILD=AlanMarshall/os-autoinst-distri-rocky#168 CASEDIR=https://github.com/AlanMarshall/os-autoinst-distri-rocky.git#serial_console PRODUCTDIR=os-autoinst-distri-rocky NEEDLES_DIR=rocky/needles +``` + +What can be seen from the complete `--dry-run` output for `openqa-clone-custom-git-refspec` is that both the job to be cloned and the PR to be used are inspected and a `openqa-clone-job` command is generated to be submitted to the openQA system the job is being cloned on. + +Without using `--dry-run` the final `openqa-clone-job` command shown above will be run causing the job of interest to be cloned with additional `POST` variables that will cause the repository/branch referenced in the PR to be cloned into the test directory with important files referenced in the cloned job. + + +### Run `openqa-clone-custom-git-refspec` without `--verbose --dry-run` mode... + +``` +$ openqa-clone-custom-git-refspec \ + https://github.com/rocky-linux/os-autoinst-distri-rocky/pull/168 \ + https://openqa.rockylinux.org/tests/16080 +Created job #16119: rocky-9.1-universal-x86_64-Build20230329-Rocky-9.1-x86_64.0-install_serial_console@64bit -> https://openqa.rockylinux.org/t16119 +``` + +### Cloned job information... + +``` +$ openqa-cli api jobs/16119 --pretty +{ + "job" : { + "assets" : { + "iso" : [ + "Rocky-9.1-20221214.1-x86_64-dvd.iso" + ] + }, + "assigned_worker_id" : 5, + "blocked_by_id" : null, + "children" : { + "Chained" : [], + "Directly chained" : [], + "Parallel" : [] + }, + "clone_id" : 16121, + "group_id" : null, + "has_parents" : 0, + "id" : 16119, + "name" : "rocky-9.1-universal-x86_64-BuildAlanMarshall_os-autoinst-distri-rocky_168-install_serial_console@AlanMarshall_os-autoinst-distri-rocky_serial_console@64bit", + "parents" : { + "Chained" : [], + "Directly chained" : [], + "Parallel" : [] + }, + "parents_ok" : 1, + "priority" : 10, + "reason" : "isotovideo abort: isotovideo received signal TERM", + "result" : "user_restarted", + "settings" : { + "ANACONDA_TEXT" : "1", + "ARCH" : "x86_64", + "ARCH_BASE_MACHINE" : "64bit", + "BACKEND" : "qemu", + "BUILD" : "AlanMarshall\/os-autoinst-distri-rocky#168", + "CASEDIR" : "https:\/\/github.com\/AlanMarshall\/os-autoinst-distri-rocky.git#serial_console", + "CLONED_FROM" : "https:\/\/openqa.rockylinux.org\/tests\/15973", + "CURRREL" : "9", + "DISTRI" : "rocky", + "FLAVOR" : "universal", + "HDDSIZEGB" : "15", + "ISO" : "Rocky-9.1-20221214.1-x86_64-dvd.iso", + "LOCATION" : "https:\/\/download.rockylinux.org\/pub\/rocky\/9.1\/BaseOS", + "MACHINE" : "64bit", + "NAME" : "00016119-rocky-9.1-universal-x86_64-BuildAlanMarshall_os-autoinst-distri-rocky_168-install_serial_console@AlanMarshall_os-autoinst-distri-rocky_serial_console@64bit", + "NEEDLES_DIR" : "rocky\/needles", + "NICTYPE_USER_OPTIONS" : "net=172.16.2.0\/24", + "NO_UEFI_POST" : "1", + "PART_TABLE_TYPE" : "mbr", + "PRODUCTDIR" : "os-autoinst-distri-rocky", + "QEMUCPU" : "Nehalem", + "QEMUCPUS" : "2", + "QEMURAM" : "2048", + "QEMU_HOST_IP" : "172.16.2.2", + "QEMU_VIDEO_DEVICE" : "virtio-vga", + "QEMU_VIRTIO_RNG" : "1", + "SERIAL_CONSOLE" : "1", + "TEST" : "install_serial_console@AlanMarshall\/os-autoinst-distri-rocky#serial_console", + "TEST_SUITE_NAME" : "install_serial_console", + "TEST_TARGET" : "ISO", + "VERSION" : "9.1", + "VIRTIO_CONSOLE_NUM" : "2", + "WORKER_CLASS" : "qemu_x86_64", + "XRES" : "1024", + "YRES" : "768" + }, + "state" : "done", + "t_finished" : "2023-03-29T06:19:37", + "t_started" : "2023-03-29T06:12:26", + "test" : "install_serial_console@AlanMarshall\/os-autoinst-distri-rocky#serial_console" + } +} +``` + + +## Advanced `openqa-clone-custom-git-refspec` + +The following example demonstrates the testing of an open Github pull request in the {{ rc.prod }} openQA production system. The PR changes test code and supplies updated needles for the test. + + +### Github PR information + +``` +➜ os-autoinst-distri-rocky git:(nazunalika/develop) gh pr view 162 + +Anaconda text install #162 +Open • AlanMarshall wants to merge 2 commits into develop from anaconda-txt • about 1 day ago ++30 -5 • No checks +Reviewers: akatch (Approved), lumarel (Requested), tcooper (Requested) +Labels: priority: medium, type: bug, test suite + + + Added new needle for text install. + Deleted redundant code. + Tested for 9.1, 8.7 & 8.8: + + openqa-cli api -X POST isos ISO=Rocky-9.1-20221214.1-x86_64-dvd.iso ARCH=x86_64 DISTRI=rocky FLAVOR=universal + VERSION=9.1 BUILD=-"$(date +%Y%m%d.%H%M%S).0"-9.1-20221214.1-universal TEST=install_anaconda_text + openqa-cli api -X POST isos ISO=Rocky-8.7-x86_64-dvd1.iso ARCH=x86_64 DISTRI=rocky FLAVOR=universal VERSION=8.7 BUILD=- + "$(date +%Y%m%d.%H%M%S).0"-8.7-20221110-universal TEST=install_anaconda_text + openqa-cli api -X POST isos ISO=Rocky-8.8-x86_64-dvd1.iso ARCH=x86_64 DISTRI=rocky FLAVOR=universal VERSION=8.8 BUILD=- + "$(date +%Y%m%d.%H%M%S).0"-8.8-lookahead-universal TEST=install_anaconda_text + + Result: Pass + Fixes Issue #145 + + +akatch approved (Member) • 18h • Newest comment + + All indicated tests pass. + + +View this pull request on GitHub: https://github.com/rocky-linux/os-autoinst-distri-rocky/pull/162 +``` + +### Run `openqa-clone-custom-git-refspec` in `--verbose --dry-run` mode + +``` +$ openqa-clone-custom-git-refspec --verbose --dry-run https://github.com/rocky-linux/os-autoinst-d +istri-rocky/pull/162 https://openqa.rockylinux.org/tests/13371 ++ shift ++ true ++ case "$1" in ++ dry_run=true ++ shift ++ true ++ case "$1" in ++ shift ++ break ++ job_list=https://openqa.rockylinux.org/tests/13371 ++ [[ -z '' ]] ++ first_arg=https://github.com/rocky-linux/os-autoinst-distri-rocky/pull/162 ++ [[ https://github.com/rocky-linux/os-autoinst-distri-rocky/pull/162 == *\p\u\l\l* ]] ++ pr_url=https://github.com/rocky-linux/os-autoinst-distri-rocky/pull/162 ++ target_repo_part=https://github.com/rocky-linux/os-autoinst-distri-rocky + + +...... + +++ jq -r '.NEEDLES_DIR | select (.!=null)' ++ old_needledir= ++ local needles_dir= ++ needles_dir=rocky/needles ++ local repo_branch=AlanMarshall/os-autoinst-distri-rocky#anaconda-txt ++ local test_suffix=@AlanMarshall/os-autoinst-distri-rocky#anaconda-txt ++ local build=AlanMarshall/os-autoinst-distri-rocky#162 ++ local casedir=https://github.com/AlanMarshall/os-autoinst-distri-rocky.git#anaconda-txt ++ local GROUP=0 ++ local dry_run=true ++ local scriptdir +++ dirname /usr/bin/openqa-clone-custom-git-refspec ++ scriptdir=/usr/bin ++ local 'cmd=true /usr/bin/openqa-clone-job --skip-chained-deps --parental-inheritance --within-instance "https://openqa.rockylinux.org" "13371" _GROUP="0" TEST+="@AlanMarshall/os-autoinst-distri-rocky#anaconda-txt" BUILD="AlanMarshall/os-autoinst-distri-rocky#162" CASEDIR="https://github.com/AlanMarshall/os-autoinst-distri-rocky.git#anaconda-txt" PRODUCTDIR="os-autoinst-distri-rocky" NEEDLES_DIR="rocky/needles"' ++ [[ 0 -ne 0 ]] ++ [[ -n '' ]] ++ eval 'true /usr/bin/openqa-clone-job --skip-chained-deps --parental-inheritance --within-instance "https://openqa.rockylinux.org" "13371" _GROUP="0" TEST+="@AlanMarshall/os-autoinst-distri-rocky#anaconda-txt" BUILD="AlanMarshall/os-autoinst-distri-rocky#162" CASEDIR="https://github.com/AlanMarshall/os-autoinst-distri-rocky.git#anaconda-txt" PRODUCTDIR="os-autoinst-distri-rocky" NEEDLES_DIR="rocky/needles"' +++ true /usr/bin/openqa-clone-job --skip-chained-deps --parental-inheritance --within-instance https://openqa.rockylinux.org 13371 _GROUP=0 TEST+=@AlanMarshall/os-autoinst-distri-rocky#anaconda-txt BUILD=AlanMarshall/os-autoinst-distri-rocky#162 CASEDIR=https://github.com/AlanMarshall/os-autoinst-distri-rocky.git#anaconda-txt PRODUCTDIR=os-autoinst-distri-rocky NEEDLES_DIR=rocky/needles +``` + +This PR provides updated needles and the default behavior of `openqa-clone-custom-git-refspec` is to **not** provide an alternate location for `NEEDLES`. The `--verbose --dry-run` output needs to be modified to ensure the needles provided in the PR are used in the test. + + +### Modify `--verbose --dry-run` output to point to needles in the PR... + +Use output to modify clone job... + +#### original +``` +$ /usr/bin/openqa-clone-job --skip-chained-deps --parental-inheritance --within-instance https://openqa.rockylinux.org 13371 _GROUP=0 TEST+=@AlanMarshall/os-autoinst-distri-rocky#anaconda-txt BUILD=AlanMarshall/os-autoinst-distri-rocky#162 CASEDIR=https://github.com/AlanMarshall/os-autoinst-distri-rocky.git#anaconda-txt PRODUCTDIR=os-autoinst-distri-rocky +NEEDLES_DIR=rocky/needles +``` + +#### specify NEEDLES_DIR manually pointing at PR branch +``` +$ /usr/bin/openqa-clone-job --skip-chained-deps --parental-inheritance --within-instance https://o +penqa.rockylinux.org 13371 _GROUP=0 TEST+=@AlanMarshall/os-autoinst-distri-rocky#anaconda-txt BUILD=AlanMarshall/os-autoinst-distr +i-rocky#162 CASEDIR=https://github.com/AlanMarshall/os-autoinst-distri-rocky.git#anaconda-txt PRODUCTDIR=os-autoinst-distri-rocky +NEEDLES_DIR=https://github.com/AlanMarshall/os-autoinst-distri-rocky.git/needles#anaconda-txt +``` + +#### {{ rc.prod }} 9.1 + +``` +$ /usr/bin/openqa-clone-job --skip-chained-deps --parental-inheritance --within-instance https://openqa.rockylinux.org 13255 _GROUP=0 TEST+=@AlanMarshall/os-autoinst-distri-rocky#anaconda-txt BUILD=AlanMarshall/os-autoinst-distri-rocky#162 CASEDIR=https://github.com/AlanMarshall/os-autoinst-distri-rocky.git#anaconda-txt PRODUCTDIR=os-autoinst-distri-rocky NEEDLES_DIR=https://github.com/AlanMarshall/os-autoinst-distri-rocky.git#anaconda-txt/needles +Created job #14228: rocky-9.1-universal-x86_64-Build20230319-Rocky-9.1-x86_64.0-install_anaconda_text@64bit -> https://openqa.rockylinux.org/t14228 +``` + +``` +$ openqa-cli api jobs/14228 --pretty +{ + "job" : { + "assets" : { + "iso" : [ + "Rocky-9.1-20221214.1-x86_64-dvd.iso" + ] + }, + "assigned_worker_id" : 9, + "blocked_by_id" : null, + "children" : { + "Chained" : [], + "Directly chained" : [], + "Parallel" : [] + }, + "clone_id" : null, + "group_id" : null, + "has_parents" : 0, + "id" : 14228, + "name" : "rocky-9.1-universal-x86_64-BuildAlanMarshall_os-autoinst-distri-rocky_162-install_anaconda_text@AlanMarshall_os-autoinst-distri-rocky_anaconda-txt@64bit", + "parents" : { + "Chained" : [], + "Directly chained" : [], + "Parallel" : [] + }, + "parents_ok" : 1, + "priority" : 0, + "result" : "passed", + "settings" : { + "ANACONDA_TEXT" : "1", + "ARCH" : "x86_64", + "ARCH_BASE_MACHINE" : "64bit", + "BACKEND" : "qemu", + "BUILD" : "AlanMarshall\/os-autoinst-distri-rocky#162", + "CASEDIR" : "https:\/\/github.com\/AlanMarshall\/os-autoinst-distri-rocky.git#anaconda-txt", + "CLONED_FROM" : "https:\/\/openqa.rockylinux.org\/tests\/13255", + "CURRREL" : "9", + "DISTRI" : "rocky", + "FLAVOR" : "universal", + "HDDSIZEGB" : "15", + "ISO" : "Rocky-9.1-20221214.1-x86_64-dvd.iso", + "LOCATION" : "https:\/\/dl.rockylinux.org\/pub\/rocky\/9.1", + "MACHINE" : "64bit", + "NAME" : "00014228-rocky-9.1-universal-x86_64-BuildAlanMarshall_os-autoinst-distri-rocky_162-install_anaconda_text@AlanMarshall_os-autoinst-distri-rocky_anaconda-txt@64bit", + "NEEDLES_DIR" : "https:\/\/github.com\/AlanMarshall\/os-autoinst-distri-rocky.git#anaconda-txt\/needles", + "NICTYPE_USER_OPTIONS" : "net=172.16.2.0\/24", + "PART_TABLE_TYPE" : "mbr", + "PRODUCTDIR" : "os-autoinst-distri-rocky", + "QEMUCPU" : "Nehalem", + "QEMUCPUS" : "2", + "QEMURAM" : "2048", + "QEMU_HOST_IP" : "172.16.2.2", + "QEMU_VIDEO_DEVICE" : "virtio-vga", + "QEMU_VIRTIO_RNG" : "1", + "TEST" : "install_anaconda_text@AlanMarshall\/os-autoinst-distri-rocky#anaconda-txt", + "TEST_SUITE_NAME" : "install_anaconda_text", + "TEST_TARGET" : "ISO", + "VERSION" : "9.1", + "WORKER_CLASS" : "qemu_x86_64", + "XRES" : "1024", + "YRES" : "768" + }, + "state" : "done", + "t_finished" : "2023-03-22T05:28:28", + "t_started" : "2023-03-22T05:07:09", + "test" : "install_anaconda_text@AlanMarshall\/os-autoinst-distri-rocky#anaconda-txt" + } +} +``` + +![openqa-clone-custome-git-refspec-job-14228 example...](/assets/images/openqa-clone-custom-git-refspec-job-14228.png){ loading=lazy } + +#### {{ rc.prod }} 8.7 + +``` +$ /usr/bin/openqa-clone-job --skip-chained-deps --parental-inheritance --within-instance https://openqa.rockylinux.org 13371 _GROUP=0 TEST+=@AlanMarshall/os-autoinst-distri-rocky#anaconda-txt BUILD=AlanMarshall/os-autoinst-distri-rocky#162 CASEDIR=https://github.com/AlanMarshall/os-autoinst-distri-rocky.git#anaconda-txt PRODUCTDIR=os-autoinst-distri-rocky NEEDLES_DIR=https://github.com/AlanMarshall/os-autoinst-distri-rocky.git#anaconda-txt/needles +Created job #14229: rocky-8.7-universal-x86_64-Build20230319-Rocky-8.7-x86_64.0-install_anaconda_text@64bit -> https://openqa.rockylinux.org/t14229 +``` + +``` +$ openqa-cli api jobs/14229 --pretty +{ + "job" : { + "assets" : { + "iso" : [ + "Rocky-8.7-x86_64-dvd1.iso" + ] + }, + "assigned_worker_id" : 8, + "blocked_by_id" : null, + "children" : { + "Chained" : [], + "Directly chained" : [], + "Parallel" : [] + }, + "clone_id" : null, + "group_id" : null, + "has_parents" : 0, + "id" : 14229, + "name" : "rocky-8.7-universal-x86_64-BuildAlanMarshall_os-autoinst-distri-rocky_162-install_anaconda_text@AlanMarshall_os-autoinst-distri-rocky_anaconda-txt@64bit", + "parents" : { + "Chained" : [], + "Directly chained" : [], + "Parallel" : [] + }, + "parents_ok" : 1, + "priority" : 0, + "result" : "passed", + "settings" : { + "ANACONDA_TEXT" : "1", + "ARCH" : "x86_64", + "ARCH_BASE_MACHINE" : "64bit", + "BACKEND" : "qemu", + "BUILD" : "AlanMarshall\/os-autoinst-distri-rocky#162", + "CASEDIR" : "https:\/\/github.com\/AlanMarshall\/os-autoinst-distri-rocky.git#anaconda-txt", + "CLONED_FROM" : "https:\/\/openqa.rockylinux.org\/tests\/13371", + "CURRREL" : "8", + "DISTRI" : "rocky", + "FLAVOR" : "universal", + "HDDSIZEGB" : "15", + "ISO" : "Rocky-8.7-x86_64-dvd1.iso", + "LOCATION" : "https:\/\/dl.rockylinux.org\/pub\/rocky\/8.7", + "MACHINE" : "64bit", + "NAME" : "00014229-rocky-8.7-universal-x86_64-BuildAlanMarshall_os-autoinst-distri-rocky_162-install_anaconda_text@AlanMarshall_os-autoinst-distri-rocky_anaconda-txt@64bit", + "NEEDLES_DIR" : "https:\/\/github.com\/AlanMarshall\/os-autoinst-distri-rocky.git#anaconda-txt\/needles", + "NICTYPE_USER_OPTIONS" : "net=172.16.2.0\/24", + "PART_TABLE_TYPE" : "mbr", + "PRODUCTDIR" : "os-autoinst-distri-rocky", + "QEMUCPU" : "Nehalem", + "QEMUCPUS" : "2", + "QEMURAM" : "2048", + "QEMU_HOST_IP" : "172.16.2.2", + "QEMU_VIDEO_DEVICE" : "virtio-vga", + "QEMU_VIRTIO_RNG" : "1", + "TEST" : "install_anaconda_text@AlanMarshall\/os-autoinst-distri-rocky#anaconda-txt", + "TEST_SUITE_NAME" : "install_anaconda_text", + "TEST_TARGET" : "ISO", + "VERSION" : "8.7", + "WORKER_CLASS" : "qemu_x86_64", + "XRES" : "1024", + "YRES" : "768" + }, + "state" : "done", + "t_finished" : "2023-03-22T05:31:22", + "t_started" : "2023-03-22T05:10:46", + "test" : "install_anaconda_text@AlanMarshall\/os-autoinst-distri-rocky#anaconda-txt" + } +} +``` + +![openqa-clone-custome-git-refspec-job-14229 example...](/assets/images/openqa-clone-custom-git-refspec-job-14229.png){ loading=lazy } + +## References + +[openQA Documentation](http://open.qa/documentation/) + +{% include 'content_bottom.md' %} diff --git a/dev_guides/openqa_clone_job_examples.md b/dev_guides/openqa_clone_job_examples.md new file mode 100644 index 0000000..aa22bbf --- /dev/null +++ b/dev_guides/openqa_clone_job_examples.md @@ -0,0 +1,249 @@ +--- +title: openQA - openqa-clone-job Examples +author: Trevor Cooper +revision_date: 2023-04-22 +rc: + prod: Rocky Linux + level: Final +--- + +# openqa-clone-job Examples + +This page will provide a brief overview of basic and advanced job cloning using the `openqa-clone-job` command. + +## System / Access Requirements + +To complete any of the examples please complete the API `POST` Access steps outlined in the [openQA - Access](openqa_access.md) document. + +## Basic `openqa-clone-job` + +### Querying openQA for a specific test or job + +First you might want to query the {{ rc.prod }} openQA system for the latest job ID for a specific job or test. The openQA client, hereafter refered to as `openqa-cli` will allow you to quickly do that via the API. Here is an example... + +``` +$ openqa-cli api --host http://openqa.rockylinux.org jobs/overview groupid=0 distri=rocky version=9.1 test=install_default_upload latest=1 | jq '.' +[ + { + "id": 22735, + "name": "rocky-9.1-dvd-iso-x86_64-Build20230423-Rocky-9.1-x86_64.0-install_default_upload@64bit" + } +] +``` + +This basically says "give me the job id and name of the most recent `install_default_upload` test for {{ rc.prod }} 9.1". + + +### Cloning a job "as-is" + +With that job id in hand you can now clone that job directly to your local openQA development system with... + +``` +$ openqa-clone-job --from https://openqa.rockylinux.org --skip-download 22735 +Cloning children of rocky-9.1-dvd-iso-x86_64-Build20230423-Rocky-9.1-x86_64.0-install_default_upload@64bit +Created job #23: rocky-9.1-dvd-iso-x86_64-Build20230423-Rocky-9.1-x86_64.0-install_default_upload@64bit -> http://localhost/t23 +``` + +### Basic job overview + +Now you should have the same job running in your local instance... + +``` +$ openqa-cli api jobs/overview +[{"id":23,"name":"rocky-9.1-dvd-iso-x86_64-Build20230423-Rocky-9.1-x86_64.0-install_default_upload@64bit"}] +``` + +### Basic job details + +``` +$ openqa-cli api jobs/23 | jq '.' +{ + "job": { + "assets": { + "iso": [ + "Rocky-9.1-20221214.1-x86_64-dvd.iso" + ] + }, + "assigned_worker_id": 2, + "blocked_by_id": null, + "children": { + "Chained": [], + "Directly chained": [], + "Parallel": [] + }, + "clone_id": null, + "group": "Rocky", + "group_id": 2, + "has_parents": 0, + "id": 23, + "name": "rocky-9.1-dvd-iso-x86_64-Build20230423-Rocky-9.1-x86_64.0-install_default_upload@64bit", + "parents": { + "Chained": [], + "Directly chained": [], + "Parallel": [] + }, + "parents_ok": 1, + "priority": 50, + "result": "none", + "settings": { + "ARCH": "x86_64", + "ARCH_BASE_MACHINE": "64bit", + "BACKEND": "qemu", + "BUILD": "20230423-Rocky-9.1-x86_64.0", + "CLONED_FROM": "https://openqa.rockylinux.org/tests/22735", + "CURRREL": "9", + "DEPLOY_UPLOAD_TEST": "install_default_upload", + "DESKTOP": "gnome", + "DISTRI": "rocky", + "FLAVOR": "dvd-iso", + "HDDSIZEGB": "15", + "ISO": "Rocky-9.1-20221214.1-x86_64-dvd.iso", + "LOCATION": "https://download.rockylinux.org/pub/rocky/9.1/BaseOS", + "MACHINE": "64bit", + "NAME": "00000023-rocky-9.1-dvd-iso-x86_64-Build20230423-Rocky-9.1-x86_64.0-install_default_upload@64bit", + "NICTYPE_USER_OPTIONS": "net=172.16.2.0/24", + "PACKAGE_SET": "default", + "PART_TABLE_TYPE": "mbr", + "POSTINSTALL": "_collect_data", + "QEMUCPU": "Nehalem", + "QEMUCPUS": "2", + "QEMURAM": "2048", + "QEMU_HOST_IP": "172.16.2.2", + "QEMU_VIDEO_DEVICE": "virtio-vga", + "QEMU_VIRTIO_RNG": "1", + "STORE_HDD_1": "disk_dvd-iso_64bit.qcow2", + "TEST": "install_default_upload", + "TEST_SUITE_NAME": "install_default_upload", + "TEST_TARGET": "ISO", + "VERSION": "9.1", + "WORKER_CLASS": "qemu_x86_64", + "XRES": "1024", + "YRES": "768" + }, + "state": "running", + "t_finished": null, + "t_started": "2023-04-23T03:02:06", + "test": "install_default_upload" + } +} +``` + +***NOTE: In the above job information you can clearly see the job was cloned from `https://openqa.rockylinux.org/tests/22735`. + + +## Advanced `openqa-clone-job` + +You can, of course, perform more elaborate operations while cloneing a job either from your local instance or from the production instance. Typically, this might be done to modify some of the job POST variables in the cloned job while keeping all other variables unchanged. + + +### Changing variable during clone + +Here is an example where the ISO is changed in the cloned job... + +``` +$ openqa-clone-job --from https://openqa.rockylinux.org --skip-download 22735 ISO=Rocky-9.1-x86_64-dvd.iso +Cloning children of rocky-9.1-dvd-iso-x86_64-Build20230423-Rocky-9.1-x86_64.0-install_default_upload@64bit +Created job #24: rocky-9.1-dvd-iso-x86_64-Build20230423-Rocky-9.1-x86_64.0-install_default_upload@64bit -> http://localhost/t24 +``` + + +### Job overview + +``` +$ openqa-cli api jobs/overview +[{"id":24,"name":"rocky-9.1-dvd-iso-x86_64-Build20230423-Rocky-9.1-x86_64.0-install_default_upload@64bit"}] +``` + + +### Job details + +``` +$ openqa-cli api jobs/24 | jq '.' +{ + "job": { + "assets": { + "iso": [ + "Rocky-9.1-x86_64-dvd.iso" + ] + }, + "assigned_worker_id": 1, + "blocked_by_id": null, + "children": { + "Chained": [], + "Directly chained": [], + "Parallel": [] + }, + "clone_id": null, + "group": "Rocky", + "group_id": 2, + "has_parents": 0, + "id": 24, + "name": "rocky-9.1-dvd-iso-x86_64-Build20230423-Rocky-9.1-x86_64.0-install_default_upload@64bit", + "parents": { + "Chained": [], + "Directly chained": [], + "Parallel": [] + }, + "parents_ok": 1, + "priority": 50, + "result": "none", + "settings": { + "ARCH": "x86_64", + "ARCH_BASE_MACHINE": "64bit", + "BACKEND": "qemu", + "BUILD": "20230423-Rocky-9.1-x86_64.0", + "CLONED_FROM": "https://openqa.rockylinux.org/tests/22735", + "CURRREL": "9", + "DEPLOY_UPLOAD_TEST": "install_default_upload", + "DESKTOP": "gnome", + "DISTRI": "rocky", + "FLAVOR": "dvd-iso", + "HDDSIZEGB": "15", + "ISO": "Rocky-9.1-x86_64-dvd.iso", + "LOCATION": "https://download.rockylinux.org/pub/rocky/9.1/BaseOS", + "MACHINE": "64bit", + "NAME": "00000024-rocky-9.1-dvd-iso-x86_64-Build20230423-Rocky-9.1-x86_64.0-install_default_upload@64bit", + "NICTYPE_USER_OPTIONS": "net=172.16.2.0/24", + "PACKAGE_SET": "default", + "PART_TABLE_TYPE": "mbr", + "POSTINSTALL": "_collect_data", + "QEMUCPU": "Nehalem", + "QEMUCPUS": "2", + "QEMURAM": "2048", + "QEMU_HOST_IP": "172.16.2.2", + "QEMU_VIDEO_DEVICE": "virtio-vga", + "QEMU_VIRTIO_RNG": "1", + "STORE_HDD_1": "disk_dvd-iso_64bit.qcow2", + "TEST": "install_default_upload", + "TEST_SUITE_NAME": "install_default_upload", + "TEST_TARGET": "ISO", + "VERSION": "9.1", + "WORKER_CLASS": "qemu_x86_64", + "XRES": "1024", + "YRES": "768" + }, + "state": "running", + "t_finished": null, + "t_started": "2023-04-23T03:08:03", + "test": "install_default_upload" + } +} +``` + +## Difference between Basic and Advanced `openqa-clone-job` + +You should notice that the only substantive difference between the two cloned jobs is the ISO that is used to run the `install_default_upload` test... + +``` +$ openqa-cli api jobs/23 | jq '.job.settings.ISO' +"Rocky-9.1-20221214.1-x86_64-dvd.iso" + +$ openqa-cli api jobs/24 | jq '.job.settings.ISO' +"Rocky-9.1-x86_64-dvd.iso" +``` + +## References + +[openQA Documentation](http://open.qa/documentation/) + +{% include 'content_bottom.md' %} From 73cfc900b3f89278fdcb1ec2c1704db0a46ca7dd Mon Sep 17 00:00:00 2001 From: Trevor Cooper Date: Sun, 23 Apr 2023 13:41:04 -0700 Subject: [PATCH 42/45] fix typo --- dev_guides/.pages | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/dev_guides/.pages b/dev_guides/.pages index 9fb60c1..83d7e4b 100644 --- a/dev_guides/.pages +++ b/dev_guides/.pages @@ -5,4 +5,4 @@ nav: - openQA - Access: openqa_access.md - openQA - openqa-cli POST Examples: openqa_cli_post_examples.md - openQA - openqa-clone-job Examples: openqa_clone_job_examples.md - - openQA - openqa-clone-custom-git-refspect Examples: openqa_clone_custom_git_refspec_examples.md + - openQA - openqa-clone-custom-git-refspec Examples: openqa_clone_custom_git_refspec_examples.md From e16e383ffb026ce3e3ae578ccb9b865b26fcda19 Mon Sep 17 00:00:00 2001 From: lumarel Date: Sat, 29 Apr 2023 20:32:46 +0200 Subject: [PATCH 43/45] Reference to WSL docs instead of having a 2nd opinion --- dev_guides/wiki_development_boxes.md | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/dev_guides/wiki_development_boxes.md b/dev_guides/wiki_development_boxes.md index dacd8fa..a2b4aeb 100644 --- a/dev_guides/wiki_development_boxes.md +++ b/dev_guides/wiki_development_boxes.md @@ -3,7 +3,7 @@ title: Create live devbox for wiki.rockylinux.org author: - Lukas Magauer - Al Bowles -revision_date: 2022-05-07 +revision_date: 2023-04-29 --- # How to create a live system to work on the documentation @@ -18,11 +18,7 @@ For now here is the link to [Trevor's vagrant box setup](https://github.com/tcoo ### WSL specific -- Make sure the Windows Subsystem for Linux is enabled on your machine -- Download the [Cloud image](https://github.com/rocky-linux/sig-cloud-instance-images/actions/workflows/build.yml) -- Get the actual disk out of the image (extract rocky-8.5-docker.tar) -- Run `wsl --import --version 2` to create the machine -- Run `wsl -d ` to initially login to the machine +Create a WSL machine like [described here](https://docs.rockylinux.org/guides/interoperability/import_rocky_to_wsl), make sure to give it a name like `rocky-wiki`. ### toolbox specific From ce3fe043bdd07dbae4ca8185511c5162e7447dfa Mon Sep 17 00:00:00 2001 From: lumarel Date: Mon, 1 May 2023 18:16:15 +0200 Subject: [PATCH 44/45] Fix some formatting and URIs to make the linter happy --- QA/Testcase_Kickstart_Installation.md | 2 +- QA/Testcase_Media_USB_dd.md | 11 ++-- dev_guides/commit_signing.md | 6 +- dev_guides/openqa_access.md | 14 ++--- dev_guides/openqa_cli_post_examples.md | 9 +-- ...penqa_clone_custom_git_refspec_examples.md | 57 +++++++++++-------- dev_guides/openqa_clone_job_examples.md | 21 +++---- 7 files changed, 61 insertions(+), 59 deletions(-) diff --git a/QA/Testcase_Kickstart_Installation.md b/QA/Testcase_Kickstart_Installation.md index 9792b70..1ff0eef 100644 --- a/QA/Testcase_Kickstart_Installation.md +++ b/QA/Testcase_Kickstart_Installation.md @@ -23,7 +23,7 @@ This test case verifies that installations via both local and remote Kickstart c 1. Connect the USB stick to the test system {% include 'testing/qa_setup_boot_to_media.md' %} 1. Hit the Tab key to edit the boot command -1. Provide a local Kickstart file by supplying the GRUB boot option `inst.ks=file:/path/to/local.ks` or a remote Kickstart file by supplying the GRUB boot option `inst.ks=https://dl.rockylinux.org/remote.ks`. +1. Provide a local Kickstart file by supplying the GRUB boot option `inst.ks=file:/path/to/local.ks` or a remote Kickstart file by supplying the GRUB boot option `inst.ks=https://git.resf.org/testing/createhdds/raw/branch/rocky/server.ks`. ## How to test 1. Continue booting the installer as normal. diff --git a/QA/Testcase_Media_USB_dd.md b/QA/Testcase_Media_USB_dd.md index 77d2e32..37df444 100644 --- a/QA/Testcase_Media_USB_dd.md +++ b/QA/Testcase_Media_USB_dd.md @@ -18,18 +18,20 @@ This verifies that Rocky Linux ISO image can be written to USB media using `dd` Any data on the USB stick used for this test is likely to be destroyed. Please do not use a stick whose contents you need to keep. ## Setup + 1. Provide a USB media device that is larger than the ISO image you wish to test and that it can be completely erased. 2. Provide a Linux (or other *nix system) that has the `dd` command available and an unoccupied USB port. 3. Download the Rocky Linux ISO image you wish to test onto the test system. - - Example command:
`curl -LOR http://dl.rockylinux.org/pub/rocky/8/isos/x86_64/Rocky-8.5-x86_64-boot.iso` + - Example command:
`curl -LOR https://dl.rockylinux.org/pub/rocky/8/isos/x86_64/Rocky-x86_64-boot.iso` 4. Download the `CHECKSUM` file that goes with the Rocky Linux ISO image that you wish to test. - - Example command:
`curl -LOR http://dl.rockylinux.org/pub/rocky/8/isos/x86_64/CHECKSUM` + - Example command:
`curl -LOR https://dl.rockylinux.org/pub/rocky/8/isos/x86_64/CHECKSUM` 5. Download the `CHECKSUM.sig` file that does with the `CHECKSUM` file. - - Example command:
`curl -LOR http://dl.rockylinux.org/pub/rocky/8/isos/x86_64/CHECKSUM.sig` + - Example command:
`curl -LOR https://dl.rockylinux.org/pub/rocky/8/isos/x86_64/CHECKSUM.sig` 6. Download the Rocky Release Engineering GPG key. - - Example command:
`curl -LOR http://dl.rockylinux.org/pub/rocky/RPM-GPG-KEY-rockyofficial` + - Example command:
`curl -LOR https://dl.rockylinux.org/pub/rocky/RPM-GPG-KEY-rockyofficial` ## How to test + 1. Import the Rocky Release Engineering GPG key. - Example command:
`gpg --import RPM-GPG-KEY-rockyofficial` 2. Verify the signature of the CHECKSUM file. @@ -43,6 +45,7 @@ This verifies that Rocky Linux ISO image can be written to USB media using `dd` 7. **[OPTIONAL]** Proceed with installation on the test system.
**Depending on installer choices this MAY destroy all the data on the test system.** ## Expected Results + 1. The gpg signature on the `CHECKSUM` file is valid. 2. The `CHECKSUM` of the Rocky Linux ISO is valid. 3. The Rocky Linux ISO is written to the USB stick without errors. diff --git a/dev_guides/commit_signing.md b/dev_guides/commit_signing.md index 321afa0..3400cfe 100644 --- a/dev_guides/commit_signing.md +++ b/dev_guides/commit_signing.md @@ -8,6 +8,7 @@ rc: level: Final --- # Creating your primary keypair + 1. Initiate the keypair generation wizard gpg --full-generate-key --expert @@ -19,6 +20,7 @@ rc: 1. Type a passphrase (twice) # Create a signing keypair + 1. Add a signing subkey gpg --expert --edit-key my@email.addr @@ -43,6 +45,7 @@ Export the *primary keypair* (put these somewhere very safe along with revocatio gpg --export --armor my@email.addr > my_email_addr.public.gpg-key # Remove the *primary keypair* from your keyring + 1. Export all subkeys from the new keypair to a file gpg --export-secret-subkeys my@email.addr > $HOME/.gnupg/subkeys @@ -60,6 +63,7 @@ Export the *primary keypair* (put these somewhere very safe along with revocatio gpg --list-secret-keys $HOME/.gnupg/secring.gpg # Revoking a *signing keypair* + Find the *primary keypair* and import it (preferably into an ephemeral system like a liveUSB) gpg --import /path/to/my_email_addr.public.gpg-key /path/to/my_email_addr.private.gpg-key @@ -68,7 +72,6 @@ Find the *primary keypair* and import it (preferably into an ephemeral system li [ passphrase twice ] gpg> save - # Renew an expired or expiring keypair gpg --edit-key my@email.addr @@ -101,6 +104,7 @@ Find the *primary keypair* and import it (preferably into an ephemeral system li gpg --keyserver pgp.mit.edu --search-key my@email.addr # References + [OpenPGP Best Practices](https://riseup.net/en/security/message-security/openpgp/best-practices#key-configuration)
[Github: Signing Commits](https://docs.github.com/en/enterprise-server@3.5/authentication/managing-commit-signature-verification/signing-commits)
[Braincoke's Log: Create a GPG Key](https://blog.braincoke.fr/security/create-a-gpg-key/)
diff --git a/dev_guides/openqa_access.md b/dev_guides/openqa_access.md index 9577335..5bd2e64 100644 --- a/dev_guides/openqa_access.md +++ b/dev_guides/openqa_access.md @@ -11,7 +11,6 @@ rc: To complete any of the examples below you will need access to a system providing the openQA client. Typically that will be a Fedora based system/container with the `openqa-client` package and it's (~239) dependencies installed. - ## Access Requirement ### API `GET` access @@ -22,10 +21,9 @@ The {{ rc.prod }} openQA system allows unrestricted public access via it's web i In order to use the openQA client to interact with the {{ rc.prod }} openQA system for `POST` operations the following are required: -- an account in good standing in the [{{ rc.prod }} Account Services](https://accounts.rockylinux.org) system, +- an account in good standing in the [{{ rc.prod }} Account Services](https://accounts.rockylinux.org) system, - authorization for API `POST` access from the {{ rc.prod }} Testing Team, and -- an [openQA API key](http://open.qa/docs/#_authentication) generated on the {{ rc.prod }} openQA system. - +- an [openQA API key](https://open.qa/docs/#_authentication) generated on the {{ rc.prod }} openQA system. ## Configuring your openqa client @@ -34,7 +32,7 @@ Per the openqa client command help you can configure your client to use your API The following example shows how to configure your client by the most common method used. It's possible to configure multiple openqa client API keys in this way. -``` +```bash $ mkdir -p ~/.config/openqa $ vim ~/.config/openqa/client.conf @@ -48,10 +46,9 @@ key = your_api_key secret = your_api_secret ``` - ## Testing your openqa client installation -``` +```bash $ openqa-cli api --host http://openqa.rockylinux.org --pretty jobs/1 { "job" : { @@ -118,7 +115,6 @@ $ openqa-cli api --host http://openqa.rockylinux.org --pretty jobs/1 ## References -[openQA Documentation](http://open.qa/documentation/) - +[openQA Documentation](https://open.qa/documentation/) {% include 'content_bottom.md' %} diff --git a/dev_guides/openqa_cli_post_examples.md b/dev_guides/openqa_cli_post_examples.md index c113d5a..541a890 100644 --- a/dev_guides/openqa_cli_post_examples.md +++ b/dev_guides/openqa_cli_post_examples.md @@ -17,7 +17,7 @@ To complete any of the examples please complete the API `POST` Access steps outl ## Basic POST -A basic `POST` can be used for any of the default test suites for the various {{ rc.prod }} media that are made available. The following examples show some of these standard `POST`s that are commonly used by our team and will be used to demonstrate how some minor variations +A basic `POST` can be used for any of the default test suites for the various {{ rc.prod }} media that are made available. The following examples show some of these standard `POST`s that are commonly used by our team and will be used to demonstrate how some minor variations. ### FLAVOR=boot-iso @@ -30,7 +30,7 @@ $ openqa-cli api -X POST isos ISO=Rocky-9.1-x86_64-boot.iso ARCH=x86_64 \ ### FLAVOR=minimal-iso -This `POST` demonstrates how a different media type, in this case the minimal ISO, for an alternate {{ rc.prod }} version, in this case {{ rc.prod }} 8.7, can be triggered. As can be seen by this and the previous `POST` the `BUILD` variable will typically be designate the date, version and architecture of the test suite. Since the minimal ISO contains all packages required to conduct a ***minimal*** install of {{ rc.prod }} that is the behavior of this test suite. +This `POST` demonstrates how a different media type, in this case the minimal ISO, for an alternate {{ rc.prod }} version, in this case {{ rc.prod }} 8.7, can be triggered. As can be seen by this and the previous `POST` the `BUILD` variable will typically be designate the date, version and architecture of the test suite. Since the minimal ISO contains all packages required to conduct a ***minimal*** install of {{ rc.prod }} that is the behavior of this test suite. ```bash $ openqa-cli api -X POST isos ISO=Rocky-8.7-x86_64-minimal.iso ARCH=x86_64 \ @@ -48,7 +48,6 @@ $ openqa-cli api -X POST isos ISO=Rocky-9.1-20221214.1-x86_64-dvd.iso ARCH=x86_6 These three test suites provide for the minimal testing of all ISOs produced for a given release of {{ rc.prod }}. - ## Advanced POST In addition to the [Basic POSTs](#basic-post) described above there are additional default test suites that use the dvd ISO media and include substantially more test cases. Those include: @@ -76,7 +75,6 @@ $ openqa-cli api -X POST isos ISO=Rocky-9.1-20221214.1-x86_64-dvd.iso ARCH=x86_6 DISTRI=rocky FLAVOR=universal VERSION=9.1 CURRREL=9 BUILD=20230409-Rocky-9.1-x86_64.0 ``` - ## Collection of test suites by BUILD A feature of openQA is that for a given job group test suites which use the same `BUILD` identifier are collected into a single view in the web UI. @@ -89,7 +87,6 @@ Thus, the examples show above which all use `BUILD=20230409-Rocky-9.1-x86_64.0` ## References -[openQA Documentation](http://open.qa/documentation/) - +[openQA Documentation](https://open.qa/documentation/) {% include 'content_bottom.md' %} diff --git a/dev_guides/openqa_clone_custom_git_refspec_examples.md b/dev_guides/openqa_clone_custom_git_refspec_examples.md index 09e1f77..c747401 100644 --- a/dev_guides/openqa_clone_custom_git_refspec_examples.md +++ b/dev_guides/openqa_clone_custom_git_refspec_examples.md @@ -25,7 +25,7 @@ The following example demonstrates the testing of an open Github pull request in ***NOTE: The Github CLI tool (`gh`) is used to display PR information statically in this guide.*** -``` +```text ➜ os-autoinst-distri-rocky git:(develop) gh pr view 168 Serial console install #168 Merged • AlanMarshall wants to merge 1 commit into develop from serial_console • about 27 days ago @@ -57,7 +57,7 @@ Above is the information provided in the original PR and it includes tests perfo In practice it is useful to run `openqa-clone-custom-git-refspec` in `--verbose` and `--dry-run` mode to observe it's behavior even for the Basic cases... -``` +```bash $ openqa-clone-custom-git-refspec --verbose --dry-run \ https://github.com/rocky-linux/os-autoinst-distri-rocky/pull/168 \ https://openqa.rockylinux.org/tests/16080 2>&1 | tee pr-168 @@ -65,7 +65,7 @@ $ openqa-clone-custom-git-refspec --verbose --dry-run \ ***NOTE: The full output of `openqa-clone-custom-git-refspece` will not be shown here.*** -``` +```diff + shift + true + case "$1" in @@ -114,10 +114,9 @@ What can be seen from the complete `--dry-run` output for `openqa-clone-custom-g Without using `--dry-run` the final `openqa-clone-job` command shown above will be run causing the job of interest to be cloned with additional `POST` variables that will cause the repository/branch referenced in the PR to be cloned into the test directory with important files referenced in the cloned job. - ### Run `openqa-clone-custom-git-refspec` without `--verbose --dry-run` mode... -``` +```bash $ openqa-clone-custom-git-refspec \ https://github.com/rocky-linux/os-autoinst-distri-rocky/pull/168 \ https://openqa.rockylinux.org/tests/16080 @@ -126,7 +125,7 @@ Created job #16119: rocky-9.1-universal-x86_64-Build20230329-Rocky-9.1-x86_64.0- ### Cloned job information... -``` +```bash $ openqa-cli api jobs/16119 --pretty { "job" : { @@ -206,10 +205,9 @@ $ openqa-cli api jobs/16119 --pretty The following example demonstrates the testing of an open Github pull request in the {{ rc.prod }} openQA production system. The PR changes test code and supplies updated needles for the test. - ### Github PR information -``` +```text ➜ os-autoinst-distri-rocky git:(nazunalika/develop) gh pr view 162 Anaconda text install #162 @@ -244,9 +242,8 @@ View this pull request on GitHub: https://github.com/rocky-linux/os-autoinst-dis ### Run `openqa-clone-custom-git-refspec` in `--verbose --dry-run` mode -``` -$ openqa-clone-custom-git-refspec --verbose --dry-run https://github.com/rocky-linux/os-autoinst-d -istri-rocky/pull/162 https://openqa.rockylinux.org/tests/13371 +```diff +$ openqa-clone-custom-git-refspec --verbose --dry-run https://github.com/rocky-linux/os-autoinst-distri-rocky/pull/162 https://openqa.rockylinux.org/tests/13371 + shift + true + case "$1" in @@ -288,33 +285,40 @@ istri-rocky/pull/162 https://openqa.rockylinux.org/tests/13371 This PR provides updated needles and the default behavior of `openqa-clone-custom-git-refspec` is to **not** provide an alternate location for `NEEDLES`. The `--verbose --dry-run` output needs to be modified to ensure the needles provided in the PR are used in the test. - ### Modify `--verbose --dry-run` output to point to needles in the PR... Use output to modify clone job... #### original -``` -$ /usr/bin/openqa-clone-job --skip-chained-deps --parental-inheritance --within-instance https://openqa.rockylinux.org 13371 _GROUP=0 TEST+=@AlanMarshall/os-autoinst-distri-rocky#anaconda-txt BUILD=AlanMarshall/os-autoinst-distri-rocky#162 CASEDIR=https://github.com/AlanMarshall/os-autoinst-distri-rocky.git#anaconda-txt PRODUCTDIR=os-autoinst-distri-rocky + +```bash +$ /usr/bin/openqa-clone-job --skip-chained-deps --parental-inheritance --within-instance https://openqa.rockylinux.org \ + 13371 _GROUP=0 TEST+=@AlanMarshall/os-autoinst-distri-rocky#anaconda-txt \ + BUILD=AlanMarshall/os-autoinst-distri-rocky#162 CASEDIR=https://github.com/AlanMarshall/os-autoinst-distri-rocky.git#anaconda-txt \ + PRODUCTDIR=os-autoinst-distri-rocky NEEDLES_DIR=rocky/needles ``` #### specify NEEDLES_DIR manually pointing at PR branch -``` -$ /usr/bin/openqa-clone-job --skip-chained-deps --parental-inheritance --within-instance https://o -penqa.rockylinux.org 13371 _GROUP=0 TEST+=@AlanMarshall/os-autoinst-distri-rocky#anaconda-txt BUILD=AlanMarshall/os-autoinst-distr -i-rocky#162 CASEDIR=https://github.com/AlanMarshall/os-autoinst-distri-rocky.git#anaconda-txt PRODUCTDIR=os-autoinst-distri-rocky -NEEDLES_DIR=https://github.com/AlanMarshall/os-autoinst-distri-rocky.git/needles#anaconda-txt + +```bash +$ /usr/bin/openqa-clone-job --skip-chained-deps --parental-inheritance --within-instance https://openqa.rockylinux.org \ + 13371 _GROUP=0 TEST+=@AlanMarshall/os-autoinst-distri-rocky#anaconda-txt \ + BUILD=AlanMarshall/os-autoinst-distri-rocky#162 CASEDIR=https://github.com/AlanMarshall/os-autoinst-distri-rocky.git#anaconda-txt \ + PRODUCTDIR=os-autoinst-distri-rocky NEEDLES_DIR=https://github.com/AlanMarshall/os-autoinst-distri-rocky.git#anaconda-txt/needles ``` #### {{ rc.prod }} 9.1 -``` -$ /usr/bin/openqa-clone-job --skip-chained-deps --parental-inheritance --within-instance https://openqa.rockylinux.org 13255 _GROUP=0 TEST+=@AlanMarshall/os-autoinst-distri-rocky#anaconda-txt BUILD=AlanMarshall/os-autoinst-distri-rocky#162 CASEDIR=https://github.com/AlanMarshall/os-autoinst-distri-rocky.git#anaconda-txt PRODUCTDIR=os-autoinst-distri-rocky NEEDLES_DIR=https://github.com/AlanMarshall/os-autoinst-distri-rocky.git#anaconda-txt/needles +```bash +$ /usr/bin/openqa-clone-job --skip-chained-deps --parental-inheritance --within-instance https://openqa.rockylinux.org \ + 13255 _GROUP=0 TEST+=@AlanMarshall/os-autoinst-distri-rocky#anaconda-txt \ + BUILD=AlanMarshall/os-autoinst-distri-rocky#162 CASEDIR=https://github.com/AlanMarshall/os-autoinst-distri-rocky.git#anaconda-txt \ + PRODUCTDIR=os-autoinst-distri-rocky NEEDLES_DIR=https://github.com/AlanMarshall/os-autoinst-distri-rocky.git#anaconda-txt/needles Created job #14228: rocky-9.1-universal-x86_64-Build20230319-Rocky-9.1-x86_64.0-install_anaconda_text@64bit -> https://openqa.rockylinux.org/t14228 ``` -``` +```bash $ openqa-cli api jobs/14228 --pretty { "job" : { @@ -389,12 +393,15 @@ $ openqa-cli api jobs/14228 --pretty #### {{ rc.prod }} 8.7 -``` -$ /usr/bin/openqa-clone-job --skip-chained-deps --parental-inheritance --within-instance https://openqa.rockylinux.org 13371 _GROUP=0 TEST+=@AlanMarshall/os-autoinst-distri-rocky#anaconda-txt BUILD=AlanMarshall/os-autoinst-distri-rocky#162 CASEDIR=https://github.com/AlanMarshall/os-autoinst-distri-rocky.git#anaconda-txt PRODUCTDIR=os-autoinst-distri-rocky NEEDLES_DIR=https://github.com/AlanMarshall/os-autoinst-distri-rocky.git#anaconda-txt/needles +```bash +$ /usr/bin/openqa-clone-job --skip-chained-deps --parental-inheritance --within-instance https://openqa.rockylinux.org \ + 13371 _GROUP=0 TEST+=@AlanMarshall/os-autoinst-distri-rocky#anaconda-txt \ + BUILD=AlanMarshall/os-autoinst-distri-rocky#162 CASEDIR=https://github.com/AlanMarshall/os-autoinst-distri-rocky.git#anaconda-txt \ + PRODUCTDIR=os-autoinst-distri-rocky NEEDLES_DIR=https://github.com/AlanMarshall/os-autoinst-distri-rocky.git#anaconda-txt/needles Created job #14229: rocky-8.7-universal-x86_64-Build20230319-Rocky-8.7-x86_64.0-install_anaconda_text@64bit -> https://openqa.rockylinux.org/t14229 ``` -``` +```bash $ openqa-cli api jobs/14229 --pretty { "job" : { diff --git a/dev_guides/openqa_clone_job_examples.md b/dev_guides/openqa_clone_job_examples.md index aa22bbf..1c8eb34 100644 --- a/dev_guides/openqa_clone_job_examples.md +++ b/dev_guides/openqa_clone_job_examples.md @@ -21,7 +21,7 @@ To complete any of the examples please complete the API `POST` Access steps outl First you might want to query the {{ rc.prod }} openQA system for the latest job ID for a specific job or test. The openQA client, hereafter refered to as `openqa-cli` will allow you to quickly do that via the API. Here is an example... -``` +```bash $ openqa-cli api --host http://openqa.rockylinux.org jobs/overview groupid=0 distri=rocky version=9.1 test=install_default_upload latest=1 | jq '.' [ { @@ -33,12 +33,11 @@ $ openqa-cli api --host http://openqa.rockylinux.org jobs/overview groupid=0 dis This basically says "give me the job id and name of the most recent `install_default_upload` test for {{ rc.prod }} 9.1". - ### Cloning a job "as-is" With that job id in hand you can now clone that job directly to your local openQA development system with... -``` +```bash $ openqa-clone-job --from https://openqa.rockylinux.org --skip-download 22735 Cloning children of rocky-9.1-dvd-iso-x86_64-Build20230423-Rocky-9.1-x86_64.0-install_default_upload@64bit Created job #23: rocky-9.1-dvd-iso-x86_64-Build20230423-Rocky-9.1-x86_64.0-install_default_upload@64bit -> http://localhost/t23 @@ -48,14 +47,14 @@ Created job #23: rocky-9.1-dvd-iso-x86_64-Build20230423-Rocky-9.1-x86_64.0-insta Now you should have the same job running in your local instance... -``` +```bash $ openqa-cli api jobs/overview [{"id":23,"name":"rocky-9.1-dvd-iso-x86_64-Build20230423-Rocky-9.1-x86_64.0-install_default_upload@64bit"}] ``` ### Basic job details -``` +```bash $ openqa-cli api jobs/23 | jq '.' { "job": { @@ -130,34 +129,30 @@ $ openqa-cli api jobs/23 | jq '.' ***NOTE: In the above job information you can clearly see the job was cloned from `https://openqa.rockylinux.org/tests/22735`. - ## Advanced `openqa-clone-job` You can, of course, perform more elaborate operations while cloneing a job either from your local instance or from the production instance. Typically, this might be done to modify some of the job POST variables in the cloned job while keeping all other variables unchanged. - ### Changing variable during clone Here is an example where the ISO is changed in the cloned job... -``` +```bash $ openqa-clone-job --from https://openqa.rockylinux.org --skip-download 22735 ISO=Rocky-9.1-x86_64-dvd.iso Cloning children of rocky-9.1-dvd-iso-x86_64-Build20230423-Rocky-9.1-x86_64.0-install_default_upload@64bit Created job #24: rocky-9.1-dvd-iso-x86_64-Build20230423-Rocky-9.1-x86_64.0-install_default_upload@64bit -> http://localhost/t24 ``` - ### Job overview -``` +```bash $ openqa-cli api jobs/overview [{"id":24,"name":"rocky-9.1-dvd-iso-x86_64-Build20230423-Rocky-9.1-x86_64.0-install_default_upload@64bit"}] ``` - ### Job details -``` +```bash $ openqa-cli api jobs/24 | jq '.' { "job": { @@ -234,7 +229,7 @@ $ openqa-cli api jobs/24 | jq '.' You should notice that the only substantive difference between the two cloned jobs is the ISO that is used to run the `install_default_upload` test... -``` +```bash $ openqa-cli api jobs/23 | jq '.job.settings.ISO' "Rocky-9.1-20221214.1-x86_64-dvd.iso" From 393384e4dc3b9481bcdfc2773493299882baf4fa Mon Sep 17 00:00:00 2001 From: lumarel Date: Mon, 1 May 2023 19:52:55 +0200 Subject: [PATCH 45/45] Fix bold formatting --- dev_guides/openqa_clone_job_examples.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/dev_guides/openqa_clone_job_examples.md b/dev_guides/openqa_clone_job_examples.md index 1c8eb34..1308652 100644 --- a/dev_guides/openqa_clone_job_examples.md +++ b/dev_guides/openqa_clone_job_examples.md @@ -127,7 +127,7 @@ $ openqa-cli api jobs/23 | jq '.' } ``` -***NOTE: In the above job information you can clearly see the job was cloned from `https://openqa.rockylinux.org/tests/22735`. +**NOTE: In the above job information you can clearly see the job was cloned from `https://openqa.rockylinux.org/tests/22735`.** ## Advanced `openqa-clone-job`