testing/os-autoinst-distri-rocky-migrated
10
0
Fork 0
Code Issues 12 Pull Requests 1 Packages Projects Releases Wiki Activity

Test Suite: install_default on rocky-8.5-[minimal|boot|dvd]-iso-x86_64 #77

Merged
akatch merged 6 commits from package_group_selection into develop 2022-03-31 23:23:01 +00:00
Conversation 5 Commits 6 Files Changed 4 +46 -34
akatch commented 2022-02-10 04:45:00 +00:00 (Migrated from github.com)
Copy Link

Description

Updates standard test suites to complete installations of each default PACKAGE_SET for each ISO, based on data provided here. These changes include updates to needles and templates.fif.json needed to make the tests pass.

Additionally, updates templates.fif.json to test relevant PACKAGE_SETs for each ISO, using the following conventions:

  • Test suites will not be duplicated between the boot and dvd ISOs
  • All default package sets will be tested against dvd-iso
  • Boot ISO will be considered valid if minimal package set installs
    cleanly

This PR fixes #16 when merged.

How Has This Been Tested?

# Reload your FIF templates as there were changes in this PR
./fifloader.py --clean --load templates.fif.json

# Verify default packagesets
openqa-cli api -X POST isos ISO=Rocky-8.5-x86_64-minimal.iso ARCH=x86_64 DISTRI=rocky FLAVOR=minimal-iso VERSION=8.5 BUILD=i16_8.5_minimal-iso_20220209.221910.0 IDENTIFICATION=false TEST=install_minimal
openqa-cli api -X POST isos ISO=Rocky-8.5-x86_64-dvd1.iso ARCH=x86_64 DISTRI=rocky FLAVOR=dvd-iso VERSION=8.5 BUILD=i16_8.5_dvd-iso_20220209.221906.0 IDENTIFICATION=false TEST=install_default
openqa-cli api -X POST isos ISO=Rocky-8.5-x86_64-boot.iso ARCH=x86_64 DISTRI=rocky FLAVOR=boot-iso VERSION=8.5 BUILD=i16_8.5_boot-iso_20220209.221902.0 IDENTIFICATION=false TEST=install_default

# Verify package selection menu looks correct for each valid PACKAGE_SET
# dvd-iso
openqa-cli api -X POST isos ISO=Rocky-8.5-x86_64-dvd1.iso ARCH=x86_64 DISTRI=rocky FLAVOR=dvd-iso VERSION=8.5 BUILD=i16_dvd-iso_workstation_20220306.021009.0 TEST=install_package_set_workstation
openqa-cli api -X POST isos ISO=Rocky-8.5-x86_64-dvd1.iso ARCH=x86_64 DISTRI=rocky FLAVOR=dvd-iso VERSION=8.5 BUILD=i16_dvd-iso_virtualization-host_20220306.021010.0 TEST=install_package_set_virtualization-host
openqa-cli api -X POST isos ISO=Rocky-8.5-x86_64-dvd1.iso ARCH=x86_64 DISTRI=rocky FLAVOR=dvd-iso VERSION=8.5 BUILD=i16_dvd-iso_server_20220306.021011.0 TEST=install_package_set_server
openqa-cli api -X POST isos ISO=Rocky-8.5-x86_64-dvd1.iso ARCH=x86_64 DISTRI=rocky FLAVOR=dvd-iso VERSION=8.5 BUILD=i16_dvd-iso_graphical-server_20220306.021012.0 TEST=install_package_set_graphical-server
openqa-cli api -X POST isos ISO=Rocky-8.5-x86_64-dvd1.iso ARCH=x86_64 DISTRI=rocky FLAVOR=dvd-iso VERSION=8.5 BUILD=i16_dvd-iso_minimal_20220306.021013.0 TEST=install_package_set_minimal

# minimal-iso
openqa-cli api -X POST isos ISO=Rocky-8.5-x86_64-minimal.iso ARCH=x86_64 DISTRI=rocky FLAVOR=minimal-iso VERSION=8.5 BUILD=i16_minimal-iso_server_20220306.021028.0 TEST=install_package_set_server
openqa-cli api -X POST isos ISO=Rocky-8.5-x86_64-minimal.iso ARCH=x86_64 DISTRI=rocky FLAVOR=minimal-iso VERSION=8.5 BUILD=i16_minimal-iso_minimal_20220306.021028.0 TEST=install_package_set_minimal

# boot-iso
openqa-cli api -X POST isos ISO=Rocky-8.5-x86_64-boot.iso ARCH=x86_64 DISTRI=rocky FLAVOR=boot-iso VERSION=8.5 BUILD=i16_boot-iso_minimal_20220306.021045.0 TEST=install_package_set_minimal

Checklist:

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • Any dependent changes have been merged and published in downstream modules

New warnings in this PR

This PR does generate softfailures of the login AVC as we have seen previously in the following suites:

  • dvd-iso with PACKAGE_SET=virtualization-host
  • dvd-iso with PACKAGE_SET=server
# Description Updates standard test suites to complete installations of each default PACKAGE_SET for each ISO, based on data provided [here](https://github.com/tcooper/os-autoinst-distri-rocky/blob/issue_18/issue_18/RESULTS.md). These changes include updates to needles and `templates.fif.json` needed to make the tests pass. Additionally, updates `templates.fif.json` to test relevant PACKAGE_SETs for each ISO, using the following conventions: - Test suites will not be duplicated between the boot and dvd ISOs - All default package sets will be tested against dvd-iso - Boot ISO will be considered valid if `minimal` package set installs cleanly This PR fixes #16 when merged. # How Has This Been Tested? ``` # Reload your FIF templates as there were changes in this PR ./fifloader.py --clean --load templates.fif.json # Verify default packagesets openqa-cli api -X POST isos ISO=Rocky-8.5-x86_64-minimal.iso ARCH=x86_64 DISTRI=rocky FLAVOR=minimal-iso VERSION=8.5 BUILD=i16_8.5_minimal-iso_20220209.221910.0 IDENTIFICATION=false TEST=install_minimal openqa-cli api -X POST isos ISO=Rocky-8.5-x86_64-dvd1.iso ARCH=x86_64 DISTRI=rocky FLAVOR=dvd-iso VERSION=8.5 BUILD=i16_8.5_dvd-iso_20220209.221906.0 IDENTIFICATION=false TEST=install_default openqa-cli api -X POST isos ISO=Rocky-8.5-x86_64-boot.iso ARCH=x86_64 DISTRI=rocky FLAVOR=boot-iso VERSION=8.5 BUILD=i16_8.5_boot-iso_20220209.221902.0 IDENTIFICATION=false TEST=install_default # Verify package selection menu looks correct for each valid PACKAGE_SET # dvd-iso openqa-cli api -X POST isos ISO=Rocky-8.5-x86_64-dvd1.iso ARCH=x86_64 DISTRI=rocky FLAVOR=dvd-iso VERSION=8.5 BUILD=i16_dvd-iso_workstation_20220306.021009.0 TEST=install_package_set_workstation openqa-cli api -X POST isos ISO=Rocky-8.5-x86_64-dvd1.iso ARCH=x86_64 DISTRI=rocky FLAVOR=dvd-iso VERSION=8.5 BUILD=i16_dvd-iso_virtualization-host_20220306.021010.0 TEST=install_package_set_virtualization-host openqa-cli api -X POST isos ISO=Rocky-8.5-x86_64-dvd1.iso ARCH=x86_64 DISTRI=rocky FLAVOR=dvd-iso VERSION=8.5 BUILD=i16_dvd-iso_server_20220306.021011.0 TEST=install_package_set_server openqa-cli api -X POST isos ISO=Rocky-8.5-x86_64-dvd1.iso ARCH=x86_64 DISTRI=rocky FLAVOR=dvd-iso VERSION=8.5 BUILD=i16_dvd-iso_graphical-server_20220306.021012.0 TEST=install_package_set_graphical-server openqa-cli api -X POST isos ISO=Rocky-8.5-x86_64-dvd1.iso ARCH=x86_64 DISTRI=rocky FLAVOR=dvd-iso VERSION=8.5 BUILD=i16_dvd-iso_minimal_20220306.021013.0 TEST=install_package_set_minimal # minimal-iso openqa-cli api -X POST isos ISO=Rocky-8.5-x86_64-minimal.iso ARCH=x86_64 DISTRI=rocky FLAVOR=minimal-iso VERSION=8.5 BUILD=i16_minimal-iso_server_20220306.021028.0 TEST=install_package_set_server openqa-cli api -X POST isos ISO=Rocky-8.5-x86_64-minimal.iso ARCH=x86_64 DISTRI=rocky FLAVOR=minimal-iso VERSION=8.5 BUILD=i16_minimal-iso_minimal_20220306.021028.0 TEST=install_package_set_minimal # boot-iso openqa-cli api -X POST isos ISO=Rocky-8.5-x86_64-boot.iso ARCH=x86_64 DISTRI=rocky FLAVOR=boot-iso VERSION=8.5 BUILD=i16_boot-iso_minimal_20220306.021045.0 TEST=install_package_set_minimal ``` # Checklist: - [x] My code follows the style guidelines of this project - [x] I have performed a self-review of my own code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [x] Any dependent changes have been merged and published in downstream modules ## New warnings in this PR This PR does generate softfailures of the login AVC as we have seen previously in the following suites: - dvd-iso with PACKAGE_SET=virtualization-host - dvd-iso with PACKAGE_SET=server
StackKorora (Migrated from github.com) reviewed 2022-02-10 04:45:00 +00:00
tcooper commented 2022-03-09 04:46:38 +00:00 (Migrated from github.com)
Copy Link

Test of PR in progress... default package_sets all pass, on package selection menu verification now.

Test of PR in progress... default package_sets all pass, on package selection menu verification now.
tcooper (Migrated from github.com) reviewed 2022-03-10 02:12:38 +00:00
tcooper (Migrated from github.com) left a comment
Copy Link

Suggested tests all ran successfully aside from standard soft fails for avc and one hard failure in _console_wait_login / uefi_postinstall which we need to discuss.

There is potential issue with one needle where hostname on login screen is unexpected and is similar to issue seen in screen of my tests.

To discuss in next meeting to get confirmation.

Suggested tests all ran successfully aside from standard soft fails for avc and one hard failure in _console_wait_login / uefi_postinstall which we need to discuss. There is potential issue with one needle where hostname on login screen is unexpected and is similar to issue seen in screen of my tests. To discuss in next meeting to get confirmation.
tcooper commented 2022-03-10 16:28:43 +00:00 (Migrated from github.com)
Copy Link

@lumarel The thing to look out for is the hostname in the openQA guest changing from localhost to whatever the hostname of your openQA dev system is. For @akatch the hostname is in one of the capture needles in this PR. On my test system it is openqa-dev...

_console_login

This happened in tests (2) that were complete passes and also in tests that had soft or full failures in the _console_wait_login or _console_avc_crash.

@lumarel The thing to look out for is the hostname in the openQA guest changing from localhost to whatever the hostname of your openQA dev system is. For @akatch the hostname is in one of the capture needles in this PR. On my test system it is `openqa-dev`... ![_console_login](https://user-images.githubusercontent.com/542846/157708896-fefa6f3d-fcc9-48a6-a89d-42b8308c0884.png) This happened in tests (2) that were complete passes and also in tests that had soft or full failures in the `_console_wait_login` or `_console_avc_crash`.
tcooper commented 2022-03-10 16:49:10 +00:00 (Migrated from github.com)
Copy Link

For one of the tests where a crash log was generated (PACKAGE_SET=virtualization_host) the guests /var/log/messages clearly shows the hostname getting set when the virbr0-nic interface is activated by NetworkManager...

...
Mar  9 05:16:59 localhost NetworkManager[831]: <info>  [1646821019.7827] device (virbr0): Activation: successful, device activated.
Mar  9 05:16:59 localhost NetworkManager[831]: <info>  [1646821019.8072] device (virbr0-nic): state change: ip-check -> secondaries (reason 'none', sys-iface-state: 'external')
Mar  9 05:16:59 localhost NetworkManager[831]: <info>  [1646821019.8082] device (virbr0-nic): state change: secondaries -> activated (reason 'none', sys-iface-state: 'external')
Mar  9 05:16:59 localhost NetworkManager[831]: <info>  [1646821019.8143] device (virbr0-nic): Activation: successful, device activated.
Mar  9 05:16:59 localhost NetworkManager[831]: <info>  [1646821019.9406] policy: set-hostname: set hostname to 'openqa-dev' (from address lookup)
Mar  9 05:16:59 localhost systemd-hostnamed[863]: Changed host name to 'openqa-dev'
...

I'm not sure why the same happened in another test where PACKAGE_SET=graphical_server.

For one of the tests where a crash log was generated (`PACKAGE_SET=virtualization_host`) the guests `/var/log/messages` clearly shows the hostname getting set when the `virbr0-nic` interface is activated by NetworkManager... ``` ... Mar 9 05:16:59 localhost NetworkManager[831]: <info> [1646821019.7827] device (virbr0): Activation: successful, device activated. Mar 9 05:16:59 localhost NetworkManager[831]: <info> [1646821019.8072] device (virbr0-nic): state change: ip-check -> secondaries (reason 'none', sys-iface-state: 'external') Mar 9 05:16:59 localhost NetworkManager[831]: <info> [1646821019.8082] device (virbr0-nic): state change: secondaries -> activated (reason 'none', sys-iface-state: 'external') Mar 9 05:16:59 localhost NetworkManager[831]: <info> [1646821019.8143] device (virbr0-nic): Activation: successful, device activated. Mar 9 05:16:59 localhost NetworkManager[831]: <info> [1646821019.9406] policy: set-hostname: set hostname to 'openqa-dev' (from address lookup) Mar 9 05:16:59 localhost systemd-hostnamed[863]: Changed host name to 'openqa-dev' ... ``` I'm not sure why the same happened in another test where `PACKAGE_SET=graphical_server`.
lumarel commented 2022-03-17 22:17:16 +00:00 (Migrated from github.com)
Copy Link

I have ran all the tests mention up there now and the behavior Trevor is talking about is happening for me for the following tests (they have the hostname of the underlying worker server, so I'm seeing all 3 names of my different nodes):

  • 00003940-rocky-8.5-dvd-iso-x86_64-Buildi16_dvd-iso_workstation_20220306.021009.0-install_package_set_workstation@uefi
  • 00003937-rocky-8.5-dvd-iso-x86_64-Buildi16_8.5_dvd-iso_20220209.221906.0-install_default@uefi
  • 00003942-rocky-8.5-dvd-iso-x86_64-Buildi16_dvd-iso_virtualization-host_20220306.021010.0-install_package_set_virtualization-host@64bit
  • 00003943-rocky-8.5-dvd-iso-x86_64-Buildi16_dvd-iso_virtualization-host_20220306.021010.0-install_package_set_virtualization-host@uefi
  • 00003947-rocky-8.5-dvd-iso-x86_64-Buildi16_dvd-iso_graphical-server_20220306.021012.0-install_package_set_graphical-server@uefi
I have ran all the tests mention up there now and the behavior Trevor is talking about is happening for me for the following tests (they have the hostname of the underlying worker server, so I'm seeing all 3 names of my different nodes): - 00003940-rocky-8.5-dvd-iso-x86_64-Buildi16_dvd-iso_workstation_20220306.021009.0-install_package_set_workstation@uefi - 00003937-rocky-8.5-dvd-iso-x86_64-Buildi16_8.5_dvd-iso_20220209.221906.0-install_default@uefi - 00003942-rocky-8.5-dvd-iso-x86_64-Buildi16_dvd-iso_virtualization-host_20220306.021010.0-install_package_set_virtualization-host@64bit - 00003943-rocky-8.5-dvd-iso-x86_64-Buildi16_dvd-iso_virtualization-host_20220306.021010.0-install_package_set_virtualization-host@uefi - 00003947-rocky-8.5-dvd-iso-x86_64-Buildi16_dvd-iso_graphical-server_20220306.021012.0-install_package_set_graphical-server@uefi
lumarel (Migrated from github.com) approved these changes 2022-03-17 23:19:25 +00:00
lumarel (Migrated from github.com) left a comment
Copy Link

So, for me these changes look good 👌🏻
If the hostname make any problems in the future, we may want to fix that in the future, but didn't see anything up to now.
Thank you al!

So, for me these changes look good 👌🏻 If the hostname make any problems in the future, we may want to fix that in the future, but didn't see anything up to now. Thank you al!
akatch commented 2022-03-22 01:47:30 +00:00 (Migrated from github.com)
Copy Link

I pulled the needle with my openQA host's hostname back out and am working on filing an issue for the hostname oddness now. Let me know if any other changes are needed. Thanks everybody for the reviews!

I pulled the needle with my openQA host's hostname back out and am working on filing an issue for the hostname oddness now. Let me know if any other changes are needed. Thanks everybody for the reviews!
Sign in to join this conversation.
Reviewers
No reviewers
StackKorora
lumarel
Labels
Clear labels   
priority: critical

This issue or pull request is of utmost priority

  
priority: high

This issue or pull request is of high priority

  
priority: low

This issue or pull request is of low priority

  
priority: medium

This issue or pull request is of medium priority

  
status: abandoned

This issue or pull request has been abandoned or is stale

  
status: duplicate

This issue or pull request already exists

  
status: invalid

This doesn't seem right

  
status: peer review needed

Requires at least one review in order to be merged

  
status: wontfix

This will not be worked on

  
tag: content

Additions to (or modifications of) content in the repository

  
tag: good first issue

Good for newcomers

  
tag: help wanted

Extra attention is needed

  
test suite

This issue or pull request is for test suite changes

  
type: bug

Something isn't working

  
type: documentation

Improvements or additions to documentation

  
type: enhancement

New feature or request

  
type: maintenance

To improve or continue maintenance for something

  
type: question

Further information is requested

No Label
priority: critical
priority: high
priority: low
priority: medium
status: abandoned
status: duplicate
status: invalid
status: peer review needed
status: wontfix
tag: content
tag: good first issue
tag: help wanted
test suite
type: bug
type: documentation
type: enhancement
type: maintenance
type: question
Milestone
Clear milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
1 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: testing/os-autoinst-distri-rocky-migrated#77
No description provided.
Delete Branch "package_group_selection"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?