testing/os-autoinst-distri-rocky-migrated
10
0
Fork 0
Code Issues 12 Pull Requests 1 Packages Projects Releases Wiki Activity
533dd806e2
os-autoinst-distri-rocky-mi.../templates

977 lines
44 KiB
Plaintext
Raw Normal View History

OpenQA machines/products template
2015-01-26 10:33:26 +00:00
#!/usr/share/openqa/script/load_templates
#
# openSUSE Machines, Products, TestSuites and JobTemplates
#
# use load_templates to load the file into the database
#
{
JobTemplates => [
{
Test Suites & docs
2015-01-26 13:39:49 +00:00
machine => { name => "64bit" },
OpenQA machines/products template
2015-01-26 10:33:26 +00:00
product => {
arch => "x86_64",
Test Suites & docs
2015-01-26 13:39:49 +00:00
distri => "fedora",
Support for live image testing This requires adding products, flavors and needles and test cases, and tweaking some existing ones to handle the slightly different behaviour of live images in shared tests. To handle the different main hub screens in live and non-live, a less stringent needle is added which is unregistered for non-live tests, so they don't match on it before they've finished updating repository metadata. There are a few small bugfix tweaks in this too, like some delays in user creation to try and avoid intermittent failures there. A new root_logged_in needle is also included, to handle a new console font in Rawhide - that has nothing strictly to do with live testing, it just happened to come up while working on this.
2015-03-18 21:28:03 +00:00
flavor => "workstation_live",
version => "*",
OpenQA machines/products template
2015-01-26 10:33:26 +00:00
},
Support for live image testing This requires adding products, flavors and needles and test cases, and tweaking some existing ones to handle the slightly different behaviour of live images in shared tests. To handle the different main hub screens in live and non-live, a less stringent needle is added which is unregistered for non-live tests, so they don't match on it before they've finished updating repository metadata. There are a few small bugfix tweaks in this too, like some delays in user creation to try and avoid intermittent failures there. A new root_logged_in needle is also included, to handle a new console font in Rawhide - that has nothing strictly to do with live testing, it just happened to come up while working on this.
2015-03-18 21:28:03 +00:00
test_suite => { name => "default_install" },
OpenQA machines/products template
2015-01-26 10:33:26 +00:00
},
Adds uefi support to tests where it makes sense What changed: * There is a new needle for uefi bootloader. * UEFI postinstallation phase * UEFI tests and machine added to template
2015-09-15 09:04:01 +00:00
{
machine => { name => "uefi" },
product => {
arch => "x86_64",
distri => "fedora",
flavor => "workstation_live",
version => "*",
},
test_suite => { name => "default_install" },
},
update templates
2015-02-13 09:58:24 +00:00
{
machine => { name => "64bit" },
product => {
arch => "x86_64",
distri => "fedora",
Support for live image testing This requires adding products, flavors and needles and test cases, and tweaking some existing ones to handle the slightly different behaviour of live images in shared tests. To handle the different main hub screens in live and non-live, a less stringent needle is added which is unregistered for non-live tests, so they don't match on it before they've finished updating repository metadata. There are a few small bugfix tweaks in this too, like some delays in user creation to try and avoid intermittent failures there. A new root_logged_in needle is also included, to handle a new console font in Rawhide - that has nothing strictly to do with live testing, it just happened to come up while working on this.
2015-03-18 21:28:03 +00:00
flavor => "server_boot",
version => "*",
},
test_suite => { name => "default_install" },
},
Adds uefi support to tests where it makes sense What changed: * There is a new needle for uefi bootloader. * UEFI postinstallation phase * UEFI tests and machine added to template
2015-09-15 09:04:01 +00:00
{
machine => { name => "uefi" },
product => {
arch => "x86_64",
distri => "fedora",
flavor => "server_boot",
version => "*",
},
test_suite => { name => "default_install" },
},
Support for live image testing This requires adding products, flavors and needles and test cases, and tweaking some existing ones to handle the slightly different behaviour of live images in shared tests. To handle the different main hub screens in live and non-live, a less stringent needle is added which is unregistered for non-live tests, so they don't match on it before they've finished updating repository metadata. There are a few small bugfix tweaks in this too, like some delays in user creation to try and avoid intermittent failures there. A new root_logged_in needle is also included, to handle a new console font in Rawhide - that has nothing strictly to do with live testing, it just happened to come up while working on this.
2015-03-18 21:28:03 +00:00
{
machine => { name => "64bit" },
product => {
arch => "x86_64",
distri => "fedora",
flavor => "generic_boot",
version => "*",
},
test_suite => { name => "default_install" },
},
Adds uefi support to tests where it makes sense What changed: * There is a new needle for uefi bootloader. * UEFI postinstallation phase * UEFI tests and machine added to template
2015-09-15 09:04:01 +00:00
{
machine => { name => "uefi" },
product => {
arch => "x86_64",
distri => "fedora",
flavor => "generic_boot",
version => "*",
},
test_suite => { name => "default_install" },
},
add KDE live default install test Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D576
2015-09-14 06:52:37 +00:00
{
machine => { name => "64bit" },
product => {
arch => "x86_64",
distri => "fedora",
flavor => "kde_live",
version => "*",
},
test_suite => { name => "default_install" },
},
Support for live image testing This requires adding products, flavors and needles and test cases, and tweaking some existing ones to handle the slightly different behaviour of live images in shared tests. To handle the different main hub screens in live and non-live, a less stringent needle is added which is unregistered for non-live tests, so they don't match on it before they've finished updating repository metadata. There are a few small bugfix tweaks in this too, like some delays in user creation to try and avoid intermittent failures there. A new root_logged_in needle is also included, to handle a new console font in Rawhide - that has nothing strictly to do with live testing, it just happened to come up while working on this.
2015-03-18 21:28:03 +00:00
{
machine => { name => "64bit" },
product => {
arch => "x86_64",
distri => "fedora",
flavor => "universal",
version => "*",
},
test_suite => { name => "package_set_minimal" },
},
{
machine => { name => "64bit" },
product => {
arch => "x86_64",
distri => "fedora",
flavor => "universal",
version => "*",
update templates
2015-02-13 09:58:24 +00:00
},
test_suite => { name => "server_repository_http_variation" },
},
{
machine => { name => "64bit" },
product => {
arch => "x86_64",
distri => "fedora",
Support for live image testing This requires adding products, flavors and needles and test cases, and tweaking some existing ones to handle the slightly different behaviour of live images in shared tests. To handle the different main hub screens in live and non-live, a less stringent needle is added which is unregistered for non-live tests, so they don't match on it before they've finished updating repository metadata. There are a few small bugfix tweaks in this too, like some delays in user creation to try and avoid intermittent failures there. A new root_logged_in needle is also included, to handle a new console font in Rawhide - that has nothing strictly to do with live testing, it just happened to come up while working on this.
2015-03-18 21:28:03 +00:00
flavor => "universal",
version => "*",
update templates
2015-02-13 09:58:24 +00:00
},
test_suite => { name => "server_repository_http_graphical" },
},
{
machine => { name => "64bit" },
product => {
arch => "x86_64",
distri => "fedora",
Support for live image testing This requires adding products, flavors and needles and test cases, and tweaking some existing ones to handle the slightly different behaviour of live images in shared tests. To handle the different main hub screens in live and non-live, a less stringent needle is added which is unregistered for non-live tests, so they don't match on it before they've finished updating repository metadata. There are a few small bugfix tweaks in this too, like some delays in user creation to try and avoid intermittent failures there. A new root_logged_in needle is also included, to handle a new console font in Rawhide - that has nothing strictly to do with live testing, it just happened to come up while working on this.
2015-03-18 21:28:03 +00:00
flavor => "universal",
version => "*",
update templates
2015-02-13 09:58:24 +00:00
},
test_suite => { name => "server_mirrorlist_graphical" },
},
{
machine => { name => "64bit" },
product => {
arch => "x86_64",
distri => "fedora",
Support for live image testing This requires adding products, flavors and needles and test cases, and tweaking some existing ones to handle the slightly different behaviour of live images in shared tests. To handle the different main hub screens in live and non-live, a less stringent needle is added which is unregistered for non-live tests, so they don't match on it before they've finished updating repository metadata. There are a few small bugfix tweaks in this too, like some delays in user creation to try and avoid intermittent failures there. A new root_logged_in needle is also included, to handle a new console font in Rawhide - that has nothing strictly to do with live testing, it just happened to come up while working on this.
2015-03-18 21:28:03 +00:00
flavor => "universal",
version => "*",
update templates
2015-02-13 09:58:24 +00:00
},
test_suite => { name => "server_delete_pata" },
},
Adds uefi support to tests where it makes sense What changed: * There is a new needle for uefi bootloader. * UEFI postinstallation phase * UEFI tests and machine added to template
2015-09-15 09:04:01 +00:00
{
machine => { name => "uefi" },
product => {
arch => "x86_64",
distri => "fedora",
flavor => "universal",
version => "*",
},
test_suite => { name => "server_delete_pata" },
},
update templates
2015-02-13 09:58:24 +00:00
{
machine => { name => "64bit" },
product => {
arch => "x86_64",
distri => "fedora",
Support for live image testing This requires adding products, flavors and needles and test cases, and tweaking some existing ones to handle the slightly different behaviour of live images in shared tests. To handle the different main hub screens in live and non-live, a less stringent needle is added which is unregistered for non-live tests, so they don't match on it before they've finished updating repository metadata. There are a few small bugfix tweaks in this too, like some delays in user creation to try and avoid intermittent failures there. A new root_logged_in needle is also included, to handle a new console font in Rawhide - that has nothing strictly to do with live testing, it just happened to come up while working on this.
2015-03-18 21:28:03 +00:00
flavor => "universal",
version => "*",
update templates
2015-02-13 09:58:24 +00:00
},
test_suite => { name => "server_kickstart_user_creation" },
},
{
machine => { name => "64bit" },
product => {
arch => "x86_64",
distri => "fedora",
Support for live image testing This requires adding products, flavors and needles and test cases, and tweaking some existing ones to handle the slightly different behaviour of live images in shared tests. To handle the different main hub screens in live and non-live, a less stringent needle is added which is unregistered for non-live tests, so they don't match on it before they've finished updating repository metadata. There are a few small bugfix tweaks in this too, like some delays in user creation to try and avoid intermittent failures there. A new root_logged_in needle is also included, to handle a new console font in Rawhide - that has nothing strictly to do with live testing, it just happened to come up while working on this.
2015-03-18 21:28:03 +00:00
flavor => "universal",
version => "*",
update templates
2015-02-13 09:58:24 +00:00
},
test_suite => { name => "server_scsi_updates_img" },
},
{
machine => { name => "64bit" },
product => {
arch => "x86_64",
distri => "fedora",
Support for live image testing This requires adding products, flavors and needles and test cases, and tweaking some existing ones to handle the slightly different behaviour of live images in shared tests. To handle the different main hub screens in live and non-live, a less stringent needle is added which is unregistered for non-live tests, so they don't match on it before they've finished updating repository metadata. There are a few small bugfix tweaks in this too, like some delays in user creation to try and avoid intermittent failures there. A new root_logged_in needle is also included, to handle a new console font in Rawhide - that has nothing strictly to do with live testing, it just happened to come up while working on this.
2015-03-18 21:28:03 +00:00
flavor => "universal",
version => "*",
update templates
2015-02-13 09:58:24 +00:00
},
test_suite => { name => "server_sata_multi" },
},
Adds uefi support to tests where it makes sense What changed: * There is a new needle for uefi bootloader. * UEFI postinstallation phase * UEFI tests and machine added to template
2015-09-15 09:04:01 +00:00
{
machine => { name => "uefi" },
product => {
arch => "x86_64",
distri => "fedora",
flavor => "universal",
version => "*",
},
test_suite => { name => "server_sata_multi" },
},
update templates
2015-02-13 09:58:24 +00:00
{
machine => { name => "64bit" },
product => {
arch => "x86_64",
distri => "fedora",
Support for live image testing This requires adding products, flavors and needles and test cases, and tweaking some existing ones to handle the slightly different behaviour of live images in shared tests. To handle the different main hub screens in live and non-live, a less stringent needle is added which is unregistered for non-live tests, so they don't match on it before they've finished updating repository metadata. There are a few small bugfix tweaks in this too, like some delays in user creation to try and avoid intermittent failures there. A new root_logged_in needle is also included, to handle a new console font in Rawhide - that has nothing strictly to do with live testing, it just happened to come up while working on this.
2015-03-18 21:28:03 +00:00
flavor => "universal",
version => "*",
update templates
2015-02-13 09:58:24 +00:00
},
test_suite => { name => "server_simple_encrypted" },
},
add disk guided free space test
2015-03-05 10:44:02 +00:00
{
machine => { name => "64bit" },
product => {
arch => "x86_64",
distri => "fedora",
correct job templates added after 'live' merge The flavor names changed and we started using the * version wild card in the 'live' merge, so we need to correct the jobs that were added to master in the meantime.
2015-03-19 21:11:17 +00:00
flavor => "universal",
version => "*",
add disk guided free space test
2015-03-05 10:44:02 +00:00
},
test_suite => { name => "server_simple_free_space" },
},
add multiple disks test
2015-03-05 12:57:47 +00:00
{
machine => { name => "64bit" },
product => {
arch => "x86_64",
distri => "fedora",
correct job templates added after 'live' merge The flavor names changed and we started using the * version wild card in the 'live' merge, so we need to correct the jobs that were added to master in the meantime.
2015-03-19 21:11:17 +00:00
flavor => "universal",
version => "*",
add multiple disks test
2015-03-05 12:57:47 +00:00
},
test_suite => { name => "server_multi_empty" },
},
add software RAID test suite
2015-03-06 09:36:25 +00:00
{
machine => { name => "64bit" },
product => {
arch => "x86_64",
distri => "fedora",
correct job templates added after 'live' merge The flavor names changed and we started using the * version wild card in the 'live' merge, so we need to correct the jobs that were added to master in the meantime.
2015-03-19 21:11:17 +00:00
flavor => "universal",
version => "*",
add software RAID test suite
2015-03-06 09:36:25 +00:00
},
test_suite => { name => "server_software_raid" },
},
add kickstart hdd testsuite
2015-03-12 09:58:04 +00:00
{
machine => { name => "64bit" },
product => {
arch => "x86_64",
distri => "fedora",
correct job templates added after 'live' merge The flavor names changed and we started using the * version wild card in the 'live' merge, so we need to correct the jobs that were added to master in the meantime.
2015-03-19 21:11:17 +00:00
flavor => "universal",
version => "*",
add kickstart hdd testsuite
2015-03-12 09:58:04 +00:00
},
test_suite => { name => "server_delete_partial" },
},
add ext3, btrfs and LVM thin tests, tweaks to custom methods Summary: This adds three new custom storage tests and some needles to support them, and tweaks the custom storage methods a bit to address some things that cropped up in writing the tests. A new method is added for changing the filesystem, as that's a distinct operation from changing the device type. This also restores the previous behaviour of select_disks() where it handled selecting custom partitioning when needed. Turns out it's pretty common to use regex'es in perl! Who'd'a thought. A corresponding commit to add the tests to openqa_fedora_tools is coming. There's no post-install step for the tests yet; I'll try and write those up and add them soon. Test Plan: Do a full run, including the new tests, on Alpha RC2 and check all are scheduled correctly and run correctly. The LVM thinp test is expected to fail as it catches a genuine bug. Reviewers: jskladan, garretraziel Reviewed By: garretraziel Subscribers: tflink Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D490
2015-08-10 18:01:12 +00:00
{
machine => { name => "64bit" },
product => {
arch => "x86_64",
distri => "fedora",
flavor => "universal",
version => "*",
},
test_suite => { name => "server_btrfs" },
},
{
machine => { name => "64bit" },
product => {
arch => "x86_64",
distri => "fedora",
flavor => "universal",
version => "*",
},
test_suite => { name => "server_ext3" },
},
{
machine => { name => "64bit" },
product => {
arch => "x86_64",
distri => "fedora",
flavor => "universal",
version => "*",
},
test_suite => { name => "server_lvmthin" },
},
add a 'no swap partition' test Summary: since we did this live at Flock today, I figured I'd tidy it up and submit it. This is an 'optional' test, but some people do run this way so it'd be nice to have it. This adds another little helper method in anacondatest.pm, for deleting partitions, which works much like the others added in previous commits. Test Plan: Schedule a test run, see if the test runs and works. Reviewers: jskladan, garretraziel Reviewed By: jskladan, garretraziel Subscribers: tflink Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D503
2015-08-19 21:41:41 +00:00
{
machine => { name => "64bit" },
product => {
arch => "x86_64",
distri => "fedora",
flavor => "universal",
version => "*",
},
test_suite => { name => "server_no_swap" },
},
Adds uefi support to tests where it makes sense What changed: * There is a new needle for uefi bootloader. * UEFI postinstallation phase * UEFI tests and machine added to template
2015-09-15 09:04:01 +00:00
{
machine => { name => "uefi" },
product => {
arch => "x86_64",
distri => "fedora",
flavor => "universal",
version => "*",
},
test_suite => { name => "server_simple_encrypted" },
},
{
machine => { name => "uefi" },
product => {
arch => "x86_64",
distri => "fedora",
flavor => "universal",
version => "*",
},
test_suite => { name => "server_simple_free_space" },
},
{
machine => { name => "uefi" },
product => {
arch => "x86_64",
distri => "fedora",
flavor => "universal",
version => "*",
},
test_suite => { name => "server_multi_empty" },
},
{
machine => { name => "uefi" },
product => {
arch => "x86_64",
distri => "fedora",
flavor => "universal",
version => "*",
},
test_suite => { name => "server_software_raid" },
},
{
machine => { name => "uefi" },
product => {
arch => "x86_64",
distri => "fedora",
flavor => "universal",
version => "*",
},
test_suite => { name => "server_delete_partial" },
},
{
machine => { name => "uefi" },
product => {
arch => "x86_64",
distri => "fedora",
flavor => "universal",
version => "*",
},
test_suite => { name => "server_btrfs" },
},
{
machine => { name => "uefi" },
product => {
arch => "x86_64",
distri => "fedora",
flavor => "universal",
version => "*",
},
test_suite => { name => "server_ext3" },
},
{
machine => { name => "uefi" },
product => {
arch => "x86_64",
distri => "fedora",
flavor => "universal",
version => "*",
},
test_suite => { name => "server_lvmthin" },
},
{
machine => { name => "uefi" },
product => {
arch => "x86_64",
distri => "fedora",
flavor => "universal",
version => "*",
},
test_suite => { name => "server_no_swap" },
},
add kickstart hdd testsuite
2015-03-12 09:58:04 +00:00
{
machine => { name => "64bit" },
product => {
arch => "x86_64",
distri => "fedora",
correct job templates added after 'live' merge The flavor names changed and we started using the * version wild card in the 'live' merge, so we need to correct the jobs that were added to master in the meantime.
2015-03-19 21:11:17 +00:00
flavor => "universal",
version => "*",
add kickstart hdd testsuite
2015-03-12 09:58:04 +00:00
},
test_suite => { name => "server_kickstart_hdd" },
},
Add fedup_minimal test Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D358
2015-05-13 11:00:46 +00:00
{
machine => { name => "64bit" },
product => {
arch => "x86_64",
distri => "fedora",
flavor => "universal",
version => "*",
},
convert upgrade tests to dnf-plugin-system-upgrade Summary: This is a first cut which more or less works for now. Issues: 1) We're not really testing the BUILD, here. All the test does is try and upgrade to the specified VERSION - so it'll be using the latest 'stable' for the given VERSION at the time the test runs. This isn't really that terrible, but especially for TC/RC validation, we might want to make things a bit more elaborate and set up the repo for the actual BUILD (and disable the main repos). 2) We'd actually need --nogpgcheck for non-Rawhide, at one specific point in the release cycle - after Branching but before Bodhi activation (which is when we can be sure all packages are signed). This won't matter until 24 branches, and maybe releng will have it fixed by then...if not, I'll tweak it. 3) We don't really test that the upgrade actually *happened* for desktop, at the moment - the only thing in the old test that really checked that was where we checked for the fedup boot menu entry, but that has no analog in dnf. What we should probably do is check that GUI login works, then switch to a console and check /etc/fedora-release just as the minimal test does. Test Plan: Run the tests. Note that creating the desktop disk image doesn't work ATM, so I can't verify the desktop test works, but the minimal one seems to (with D565). There'll be a matching diff for openqa_fedora_tools to update the test case names there. Reviewers: jskladan, garretraziel Reviewed By: jskladan, garretraziel Subscribers: tflink Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D567
2015-09-10 21:49:13 +00:00
test_suite => { name => "upgrade_minimal_64bit" },
Add fedup_minimal test Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D358
2015-05-13 11:00:46 +00:00
},
add fedup desktop test Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D451
2015-07-17 10:52:56 +00:00
{
machine => { name => "64bit" },
product => {
arch => "x86_64",
distri => "fedora",
flavor => "universal",
version => "*",
},
convert upgrade tests to dnf-plugin-system-upgrade Summary: This is a first cut which more or less works for now. Issues: 1) We're not really testing the BUILD, here. All the test does is try and upgrade to the specified VERSION - so it'll be using the latest 'stable' for the given VERSION at the time the test runs. This isn't really that terrible, but especially for TC/RC validation, we might want to make things a bit more elaborate and set up the repo for the actual BUILD (and disable the main repos). 2) We'd actually need --nogpgcheck for non-Rawhide, at one specific point in the release cycle - after Branching but before Bodhi activation (which is when we can be sure all packages are signed). This won't matter until 24 branches, and maybe releng will have it fixed by then...if not, I'll tweak it. 3) We don't really test that the upgrade actually *happened* for desktop, at the moment - the only thing in the old test that really checked that was where we checked for the fedup boot menu entry, but that has no analog in dnf. What we should probably do is check that GUI login works, then switch to a console and check /etc/fedora-release just as the minimal test does. Test Plan: Run the tests. Note that creating the desktop disk image doesn't work ATM, so I can't verify the desktop test works, but the minimal one seems to (with D565). There'll be a matching diff for openqa_fedora_tools to update the test case names there. Reviewers: jskladan, garretraziel Reviewed By: jskladan, garretraziel Subscribers: tflink Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D567
2015-09-10 21:49:13 +00:00
test_suite => { name => "upgrade_desktop_64bit" },
add fedup desktop test Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D451
2015-07-17 10:52:56 +00:00
},
add updates.img via local media test Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D496
2015-08-17 15:36:40 +00:00
{
machine => { name => "64bit" },
product => {
arch => "x86_64",
distri => "fedora",
flavor => "universal",
version => "*",
},
test_suite => { name => "server_updates_img_local" },
},
add guided shrink test Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D561
2015-09-08 13:54:22 +00:00
{
machine => { name => "64bit" },
product => {
arch => "x86_64",
distri => "fedora",
flavor => "universal",
version => "*",
},
test_suite => { name => "server_shrink_ext4" },
},
{
machine => { name => "64bit" },
product => {
arch => "x86_64",
distri => "fedora",
flavor => "universal",
version => "*",
},
test_suite => { name => "server_shrink_ntfs" },
},
add a french (encrypted) test Summary: this handles Non-English European Language Install. Basically it's a bunch of new screenshots for existing tag names, plus a bit of configurability in _boot_to_anaconda and tweaking some existing needles to do non-text matches. The weird 'half-the- icon' needles are for cases where there may or may not be a warning triangle but we want to click it either way (saves duplicating the needle). This also sets up a convention for tagging what languages a needle is appropriate for. If it's specifically appropriate for one or more languages, a tag ENV-LANGUAGE-(LANGUAGE) should be applied for each language, where (LANGUAGE) is the install language in upper-case ('LANGUAGE' variable, which should also be the string that will be typed into the language selection screen). If the needle ought to be used for *all* languages - i.e. it's not a text match, or any text in the match is known not to be translated - the tag ENV-INSTLANG-ALL should be applied. To back this, main.pm now unregisters all needles that are not tagged with either ENV-LANGUAGE-ALL or the tag for the language actually being used (if the LANGUAGE var is not set, we assume english). The point of this is to check the install is actually translated; if we allow all needles to match, the test would pass even if no translations appeared at all. Test Plan: Run all tests and make sure you get the expected results. You can schedule a run against 23 Beta TC1 to see the French test fails 'correctly' when translations are missing. Reviewers: jskladan, garretraziel Reviewed By: garretraziel Subscribers: tflink Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D577
2015-09-15 01:08:58 +00:00
{
machine => { name => "64bit" },
product => {
arch => "x86_64",
distri => "fedora",
flavor => "universal",
version => "*",
},
test_suite => { name => "european_language_install" },
},
add 32-bit tests to templates Summary: This duplicates basically the entire test suite for 32-bit. We could choose to run only a few tests for 32-bit if we wanted, but I figure we may as well do as much testing as we can. Only a few of the results will actually wind up 'counting' separately in the wiki, but we can always see the results for all the tests in openQA itself. Next we can duplicate the whole set again for UEFI! Test Plan: Schedule a full test run and make sure all the tests run for both arches. Also check result submission works correctly. Requires the corresponding change to tools (or you can just use one of the trigger commands which lets you specify arch with a parameter). Reviewers: jskladan, garretraziel Reviewed By: garretraziel Subscribers: tflink Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D500
2015-08-16 01:10:14 +00:00
{
machine => { name => "32bit" },
product => {
arch => "i386",
distri => "fedora",
flavor => "workstation_live",
version => "*",
},
test_suite => { name => "default_install" },
},
{
machine => { name => "32bit" },
product => {
arch => "i386",
distri => "fedora",
flavor => "server_boot",
version => "*",
},
test_suite => { name => "default_install" },
},
{
machine => { name => "32bit" },
product => {
arch => "i386",
distri => "fedora",
flavor => "generic_boot",
version => "*",
},
test_suite => { name => "default_install" },
},
add KDE live default install test Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D576
2015-09-14 06:52:37 +00:00
{
machine => { name => "32bit" },
product => {
arch => "i386",
distri => "fedora",
correct 32-bit kde live test entry in templates
2015-09-15 01:22:05 +00:00
flavor => "kde_live",
add KDE live default install test Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D576
2015-09-14 06:52:37 +00:00
version => "*",
},
correct 32-bit kde live test entry in templates
2015-09-15 01:22:05 +00:00
test_suite => { name => "default_install" },
add KDE live default install test Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D576
2015-09-14 06:52:37 +00:00
},
add 32-bit tests to templates Summary: This duplicates basically the entire test suite for 32-bit. We could choose to run only a few tests for 32-bit if we wanted, but I figure we may as well do as much testing as we can. Only a few of the results will actually wind up 'counting' separately in the wiki, but we can always see the results for all the tests in openQA itself. Next we can duplicate the whole set again for UEFI! Test Plan: Schedule a full test run and make sure all the tests run for both arches. Also check result submission works correctly. Requires the corresponding change to tools (or you can just use one of the trigger commands which lets you specify arch with a parameter). Reviewers: jskladan, garretraziel Reviewed By: garretraziel Subscribers: tflink Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D500
2015-08-16 01:10:14 +00:00
{
machine => { name => "32bit" },
product => {
arch => "i386",
distri => "fedora",
flavor => "universal",
version => "*",
},
test_suite => { name => "package_set_minimal" },
},
{
machine => { name => "32bit" },
product => {
arch => "i386",
distri => "fedora",
flavor => "universal",
version => "*",
},
test_suite => { name => "server_repository_http_graphical" },
},
{
machine => { name => "32bit" },
product => {
arch => "i386",
distri => "fedora",
flavor => "universal",
version => "*",
},
test_suite => { name => "server_scsi_updates_img" },
},
{
machine => { name => "32bit" },
product => {
arch => "i386",
distri => "fedora",
flavor => "universal",
version => "*",
},
test_suite => { name => "server_simple_encrypted" },
},
{
machine => { name => "32bit" },
product => {
arch => "i386",
distri => "fedora",
flavor => "universal",
version => "*",
},
test_suite => { name => "server_software_raid" },
},
{
machine => { name => "32bit" },
product => {
arch => "i386",
distri => "fedora",
flavor => "universal",
version => "*",
},
test_suite => { name => "server_btrfs" },
},
{
machine => { name => "32bit" },
product => {
arch => "i386",
distri => "fedora",
flavor => "universal",
version => "*",
},
test_suite => { name => "server_ext3" },
},
{
machine => { name => "32bit" },
product => {
arch => "i386",
distri => "fedora",
flavor => "universal",
version => "*",
},
test_suite => { name => "server_lvmthin" },
},
{
machine => { name => "32bit" },
product => {
arch => "i386",
distri => "fedora",
flavor => "universal",
version => "*",
},
convert upgrade tests to dnf-plugin-system-upgrade Summary: This is a first cut which more or less works for now. Issues: 1) We're not really testing the BUILD, here. All the test does is try and upgrade to the specified VERSION - so it'll be using the latest 'stable' for the given VERSION at the time the test runs. This isn't really that terrible, but especially for TC/RC validation, we might want to make things a bit more elaborate and set up the repo for the actual BUILD (and disable the main repos). 2) We'd actually need --nogpgcheck for non-Rawhide, at one specific point in the release cycle - after Branching but before Bodhi activation (which is when we can be sure all packages are signed). This won't matter until 24 branches, and maybe releng will have it fixed by then...if not, I'll tweak it. 3) We don't really test that the upgrade actually *happened* for desktop, at the moment - the only thing in the old test that really checked that was where we checked for the fedup boot menu entry, but that has no analog in dnf. What we should probably do is check that GUI login works, then switch to a console and check /etc/fedora-release just as the minimal test does. Test Plan: Run the tests. Note that creating the desktop disk image doesn't work ATM, so I can't verify the desktop test works, but the minimal one seems to (with D565). There'll be a matching diff for openqa_fedora_tools to update the test case names there. Reviewers: jskladan, garretraziel Reviewed By: jskladan, garretraziel Subscribers: tflink Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D567
2015-09-10 21:49:13 +00:00
test_suite => { name => "upgrade_desktop_32bit" },
add updates.img via local media test Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D496
2015-08-17 15:36:40 +00:00
}
OpenQA machines/products template
2015-01-26 10:33:26 +00:00
],
Machines => [
{
backend => "qemu",
name => "64bit",
changes in user creation, http mirrors
2015-02-19 13:15:29 +00:00
settings => [
{ key => "QEMUCPU", value => "qemu64" },
Support for live image testing This requires adding products, flavors and needles and test cases, and tweaking some existing ones to handle the slightly different behaviour of live images in shared tests. To handle the different main hub screens in live and non-live, a less stringent needle is added which is unregistered for non-live tests, so they don't match on it before they've finished updating repository metadata. There are a few small bugfix tweaks in this too, like some delays in user creation to try and avoid intermittent failures there. A new root_logged_in needle is also included, to handle a new console font in Rawhide - that has nothing strictly to do with live testing, it just happened to come up while working on this.
2015-03-18 21:28:03 +00:00
{ key => "QEMUCPUS", value => "2"},
{ key => "QEMUVGA", value => "std"},
{ key => "QEMURAM", value => "2048"}
changes in user creation, http mirrors
2015-02-19 13:15:29 +00:00
],
OpenQA machines/products template
2015-01-26 10:33:26 +00:00
variables => "",
},
add 32-bit tests to templates Summary: This duplicates basically the entire test suite for 32-bit. We could choose to run only a few tests for 32-bit if we wanted, but I figure we may as well do as much testing as we can. Only a few of the results will actually wind up 'counting' separately in the wiki, but we can always see the results for all the tests in openQA itself. Next we can duplicate the whole set again for UEFI! Test Plan: Schedule a full test run and make sure all the tests run for both arches. Also check result submission works correctly. Requires the corresponding change to tools (or you can just use one of the trigger commands which lets you specify arch with a parameter). Reviewers: jskladan, garretraziel Reviewed By: garretraziel Subscribers: tflink Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D500
2015-08-16 01:10:14 +00:00
{
backend => "qemu",
name => "32bit",
settings => [
{ key => "QEMUCPU", value => "qemu32" },
{ key => "QEMUCPUS", value => "2"},
{ key => "QEMUVGA", value => "std"},
{ key => "QEMURAM", value => "2048"}
],
variables => "",
},
Adds uefi support to tests where it makes sense What changed: * There is a new needle for uefi bootloader. * UEFI postinstallation phase * UEFI tests and machine added to template
2015-09-15 09:04:01 +00:00
{
backend => "qemu",
name => "uefi",
settings => [
{ key => "QEMUCPU", value => "qemu64" },
{ key => "QEMUCPUS", value => "2"},
{ key => "QEMUVGA", value => "std"},
{ key => "QEMURAM", value => "2048"},
{ key => "UEFI", value => "1"}
],
variables => "",
},
OpenQA machines/products template
2015-01-26 10:33:26 +00:00
],
Products => [
{
arch => "x86_64",
distri => "fedora",
Support for live image testing This requires adding products, flavors and needles and test cases, and tweaking some existing ones to handle the slightly different behaviour of live images in shared tests. To handle the different main hub screens in live and non-live, a less stringent needle is added which is unregistered for non-live tests, so they don't match on it before they've finished updating repository metadata. There are a few small bugfix tweaks in this too, like some delays in user creation to try and avoid intermittent failures there. A new root_logged_in needle is also included, to handle a new console font in Rawhide - that has nothing strictly to do with live testing, it just happened to come up while working on this.
2015-03-18 21:28:03 +00:00
flavor => "universal",
name => "",
settings => [
],
variables => "",
version => "*",
},
{
arch => "x86_64",
distri => "fedora",
flavor => "generic_boot",
name => "",
settings => [
],
variables => "",
version => "*",
},
{
arch => "x86_64",
distri => "fedora",
flavor => "server_boot",
OpenQA machines/products template
2015-01-26 10:33:26 +00:00
name => "",
settings => [
],
variables => "",
Support for live image testing This requires adding products, flavors and needles and test cases, and tweaking some existing ones to handle the slightly different behaviour of live images in shared tests. To handle the different main hub screens in live and non-live, a less stringent needle is added which is unregistered for non-live tests, so they don't match on it before they've finished updating repository metadata. There are a few small bugfix tweaks in this too, like some delays in user creation to try and avoid intermittent failures there. A new root_logged_in needle is also included, to handle a new console font in Rawhide - that has nothing strictly to do with live testing, it just happened to come up while working on this.
2015-03-18 21:28:03 +00:00
version => "*",
OpenQA machines/products template
2015-01-26 10:33:26 +00:00
},
Support for live image testing This requires adding products, flavors and needles and test cases, and tweaking some existing ones to handle the slightly different behaviour of live images in shared tests. To handle the different main hub screens in live and non-live, a less stringent needle is added which is unregistered for non-live tests, so they don't match on it before they've finished updating repository metadata. There are a few small bugfix tweaks in this too, like some delays in user creation to try and avoid intermittent failures there. A new root_logged_in needle is also included, to handle a new console font in Rawhide - that has nothing strictly to do with live testing, it just happened to come up while working on this.
2015-03-18 21:28:03 +00:00
{
arch => "x86_64",
distri => "fedora",
flavor => "workstation_live",
name => "",
settings => [
add 32-bit tests to templates Summary: This duplicates basically the entire test suite for 32-bit. We could choose to run only a few tests for 32-bit if we wanted, but I figure we may as well do as much testing as we can. Only a few of the results will actually wind up 'counting' separately in the wiki, but we can always see the results for all the tests in openQA itself. Next we can duplicate the whole set again for UEFI! Test Plan: Schedule a full test run and make sure all the tests run for both arches. Also check result submission works correctly. Requires the corresponding change to tools (or you can just use one of the trigger commands which lets you specify arch with a parameter). Reviewers: jskladan, garretraziel Reviewed By: garretraziel Subscribers: tflink Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D500
2015-08-16 01:10:14 +00:00
{ key => "LIVE", value => "1" },
{ key => "DESKTOP", value => "gnome" }
],
variables => "",
version => "*",
},
add KDE live default install test Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D576
2015-09-14 06:52:37 +00:00
{
arch => "x86_64",
distri => "fedora",
flavor => "kde_live",
name => "",
settings => [
{ key => "LIVE", value => "1" },
{ key => "DESKTOP", value => "kde" }
],
variables => "",
version => "*",
},
add 32-bit tests to templates Summary: This duplicates basically the entire test suite for 32-bit. We could choose to run only a few tests for 32-bit if we wanted, but I figure we may as well do as much testing as we can. Only a few of the results will actually wind up 'counting' separately in the wiki, but we can always see the results for all the tests in openQA itself. Next we can duplicate the whole set again for UEFI! Test Plan: Schedule a full test run and make sure all the tests run for both arches. Also check result submission works correctly. Requires the corresponding change to tools (or you can just use one of the trigger commands which lets you specify arch with a parameter). Reviewers: jskladan, garretraziel Reviewed By: garretraziel Subscribers: tflink Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D500
2015-08-16 01:10:14 +00:00
{
arch => "i386",
distri => "fedora",
flavor => "universal",
name => "",
settings => [
],
variables => "",
version => "*",
},
{
arch => "i386",
distri => "fedora",
flavor => "generic_boot",
name => "",
settings => [
],
variables => "",
version => "*",
},
{
arch => "i386",
distri => "fedora",
flavor => "server_boot",
name => "",
settings => [
],
variables => "",
version => "*",
},
{
arch => "i386",
distri => "fedora",
flavor => "workstation_live",
name => "",
settings => [
Support for live image testing This requires adding products, flavors and needles and test cases, and tweaking some existing ones to handle the slightly different behaviour of live images in shared tests. To handle the different main hub screens in live and non-live, a less stringent needle is added which is unregistered for non-live tests, so they don't match on it before they've finished updating repository metadata. There are a few small bugfix tweaks in this too, like some delays in user creation to try and avoid intermittent failures there. A new root_logged_in needle is also included, to handle a new console font in Rawhide - that has nothing strictly to do with live testing, it just happened to come up while working on this.
2015-03-18 21:28:03 +00:00
{ key => "LIVE", value => "1" },
{ key => "DESKTOP", value => "gnome" }
],
variables => "",
version => "*",
add KDE live default install test Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D576
2015-09-14 06:52:37 +00:00
},
{
arch => "i386",
distri => "fedora",
flavor => "kde_live",
name => "",
settings => [
{ key => "LIVE", value => "1" },
{ key => "DESKTOP", value => "kde" }
],
variables => "",
version => "*",
}
OpenQA machines/products template
2015-01-26 10:33:26 +00:00
],
TestSuites => [
{
Support for live image testing This requires adding products, flavors and needles and test cases, and tweaking some existing ones to handle the slightly different behaviour of live images in shared tests. To handle the different main hub screens in live and non-live, a less stringent needle is added which is unregistered for non-live tests, so they don't match on it before they've finished updating repository metadata. There are a few small bugfix tweaks in this too, like some delays in user creation to try and avoid intermittent failures there. A new root_logged_in needle is also included, to handle a new console font in Rawhide - that has nothing strictly to do with live testing, it just happened to come up while working on this.
2015-03-18 21:28:03 +00:00
name => "default_install",
prio => 1,
settings => [
{ key => "PACKAGE_SET", value => "default" },
],
variables => "",
},
{
name => "package_set_minimal",
Added 'server_simple' testsuite
2015-01-26 14:58:07 +00:00
prio => 1,
Support for live image testing This requires adding products, flavors and needles and test cases, and tweaking some existing ones to handle the slightly different behaviour of live images in shared tests. To handle the different main hub screens in live and non-live, a less stringent needle is added which is unregistered for non-live tests, so they don't match on it before they've finished updating repository metadata. There are a few small bugfix tweaks in this too, like some delays in user creation to try and avoid intermittent failures there. A new root_logged_in needle is also included, to handle a new console font in Rawhide - that has nothing strictly to do with live testing, it just happened to come up while working on this.
2015-03-18 21:28:03 +00:00
settings => [
{ key => "PACKAGE_SET", value => "minimal" },
],
OpenQA machines/products template
2015-01-26 10:33:26 +00:00
variables => "",
},
Added 'server_sata_multi' testsuite The testsuite covers guided installation on multiple devices. TODO: change from VirtIO to SATA driver
2015-01-26 16:21:32 +00:00
{
name => "server_sata_multi",
prio => 2,
settings => [
merge DISK_GUIDED and DISK_CUSTOM into PARTITIONING Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D486
2015-08-06 09:02:18 +00:00
{ key => "PARTITIONING", value => "guided_multi" },
Added few more tests
2015-01-27 12:35:27 +00:00
{ key => "HDDMODEL", value => "ide-hd,bus=ahci0.0" },
Updated templates
2015-01-27 15:35:10 +00:00
{ key => "NUMDISKS", value => "2" },
fixed typo
2015-02-13 09:01:38 +00:00
{ key => "HDD_2", value => "disk_full.img" },
{ key => "ROOT_PASSWORD", value => "weakpassword" },
Added few more tests
2015-01-27 12:35:27 +00:00
],
variables => "",
},
{
name => "server_scsi_updates_img",
prio => 3,
settings => [
{ key => "BOOT_UPDATES_IMG_URL", value => "1" },
Simplified the templates
2015-02-04 13:05:20 +00:00
{ key => "GRUB", value => "inst.updates=https://fedorapeople.org/groups/qa/updates/updates-unipony.img" },
Added few more tests
2015-01-27 12:35:27 +00:00
{ key => "HDDMODEL", value => "virtio-scsi-pci" },
],
variables => "",
},
{
name => "server_kickstart_user_creation",
prio => 4,
settings => [
{ key => "KICKSTART", value => "1" },
Simplified the templates
2015-02-04 13:05:20 +00:00
{ key => "GRUB", value => "inst.ks=http://jskladan.fedorapeople.org/kickstarts/root-user-crypted-net.ks" },
Added few more tests
2015-01-27 12:35:27 +00:00
{ key => "USER_LOGIN", value => "test" },
{ key => "USER_PASSWORD", value => "test" },
{ key => "ROOT_PASSWORD", value => "111111" },
Added 'server_sata_multi' testsuite The testsuite covers guided installation on multiple devices. TODO: change from VirtIO to SATA driver
2015-01-26 16:21:32 +00:00
],
variables => "",
},
Added 'server_delete_pata' testsuite
2015-01-27 13:22:35 +00:00
{
name => "server_delete_pata",
prio => 5,
settings => [
merge DISK_GUIDED and DISK_CUSTOM into PARTITIONING Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D486
2015-08-06 09:02:18 +00:00
{ key => "PARTITIONING", value => "guided_delete_all" },
Added 'server_delete_pata' testsuite
2015-01-27 13:22:35 +00:00
{ key => "HDDMODEL", value => "ide-hd" },
{ key => "HDD_1", value => "disk_full.img" },
],
variables => "",
},
Simplified the templates
2015-02-04 13:05:20 +00:00
{
name => "server_mirrorlist_graphical",
prio => 6,
settings => [
{ key => "MIRRORLIST_GRAPHICAL", value => "1" },
],
variables => "",
},
{
name => "server_repository_http_graphical",
prio => 7,
settings => [
remove hardcoded repo urls
2015-03-04 11:26:02 +00:00
{ key => "REPOSITORY_GRAPHICAL", value => "http://dl.fedoraproject.org/pub/fedora/linux/development" },
Simplified the templates
2015-02-04 13:05:20 +00:00
],
variables => "",
},
Inst-repo HTTP variation
2015-02-04 13:45:37 +00:00
{
name => "server_repository_http_variation",
prio => 8,
settings => [
remove hardcoded repo urls
2015-03-04 11:26:02 +00:00
{ key => "REPOSITORY_VARIATION", value => "http://dl.fedoraproject.org/pub/fedora/linux/development" },
Inst-repo HTTP variation
2015-02-04 13:45:37 +00:00
],
variables => "",
},
New tests
2015-02-13 08:51:24 +00:00
{
name => "server_delete_partial",
prio => 9,
settings => [
merge DISK_GUIDED and DISK_CUSTOM into PARTITIONING Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D486
2015-08-06 09:02:18 +00:00
{ key => "PARTITIONING", value => "guided_delete_partial" },
New tests
2015-02-13 08:51:24 +00:00
{ key => "HDD_1", value => "disk_full.img" },
{ key => "ROOT_PASSWORD", value => "weakpassword" },
],
variables => "",
},
update templates
2015-02-13 09:58:24 +00:00
{
name => "server_simple_encrypted",
add disk guided free space test
2015-03-05 10:44:02 +00:00
prio => 10,
update templates
2015-02-13 09:58:24 +00:00
settings => [
{ key => "ENCRYPT_PASSWORD", value => "weakpassword" },
],
variables => "",
},
add disk guided free space test
2015-03-05 10:44:02 +00:00
{
name => "server_simple_free_space",
prio => 11,
settings => [
merge DISK_GUIDED and DISK_CUSTOM into PARTITIONING Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D486
2015-08-06 09:02:18 +00:00
{ key => "PARTITIONING", value => "guided_free_space" },
add disk guided free space test
2015-03-05 10:44:02 +00:00
{ key => "HDD_1", value => "disk_freespace.img" },
{ key => "ROOT_PASSWORD", value => "weakpassword" },
],
variables => "",
},
add multiple disks test
2015-03-05 12:57:47 +00:00
{
name => "server_multi_empty",
prio => 12,
settings => [
merge DISK_GUIDED and DISK_CUSTOM into PARTITIONING Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D486
2015-08-06 09:02:18 +00:00
{ key => "PARTITIONING", value => "guided_multi_empty_all" },
add multiple disks test
2015-03-05 12:57:47 +00:00
{ key => "NUMDISKS", value => "2" },
{ key => "ROOT_PASSWORD", value => "weakpassword" },
],
variables => "",
},
add software RAID test suite
2015-03-06 09:36:25 +00:00
{
name => "server_software_raid",
prio => 13,
settings => [
merge DISK_GUIDED and DISK_CUSTOM into PARTITIONING Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D486
2015-08-06 09:02:18 +00:00
{ key => "PARTITIONING", value => "custom_software_raid" },
add software RAID test suite
2015-03-06 09:36:25 +00:00
{ key => "NUMDISKS", value => "2" },
{ key => "ROOT_PASSWORD", value => "weakpassword" },
],
variables => "",
},
add kickstart hdd testsuite
2015-03-12 09:58:04 +00:00
{
add ext3, btrfs and LVM thin tests, tweaks to custom methods Summary: This adds three new custom storage tests and some needles to support them, and tweaks the custom storage methods a bit to address some things that cropped up in writing the tests. A new method is added for changing the filesystem, as that's a distinct operation from changing the device type. This also restores the previous behaviour of select_disks() where it handled selecting custom partitioning when needed. Turns out it's pretty common to use regex'es in perl! Who'd'a thought. A corresponding commit to add the tests to openqa_fedora_tools is coming. There's no post-install step for the tests yet; I'll try and write those up and add them soon. Test Plan: Do a full run, including the new tests, on Alpha RC2 and check all are scheduled correctly and run correctly. The LVM thinp test is expected to fail as it catches a genuine bug. Reviewers: jskladan, garretraziel Reviewed By: garretraziel Subscribers: tflink Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D490
2015-08-10 18:01:12 +00:00
name => "server_btrfs",
Add fedup_minimal test Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D358
2015-05-13 11:00:46 +00:00
prio => 14,
add ext3, btrfs and LVM thin tests, tweaks to custom methods Summary: This adds three new custom storage tests and some needles to support them, and tweaks the custom storage methods a bit to address some things that cropped up in writing the tests. A new method is added for changing the filesystem, as that's a distinct operation from changing the device type. This also restores the previous behaviour of select_disks() where it handled selecting custom partitioning when needed. Turns out it's pretty common to use regex'es in perl! Who'd'a thought. A corresponding commit to add the tests to openqa_fedora_tools is coming. There's no post-install step for the tests yet; I'll try and write those up and add them soon. Test Plan: Do a full run, including the new tests, on Alpha RC2 and check all are scheduled correctly and run correctly. The LVM thinp test is expected to fail as it catches a genuine bug. Reviewers: jskladan, garretraziel Reviewed By: garretraziel Subscribers: tflink Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D490
2015-08-10 18:01:12 +00:00
settings => [
{ key => "PARTITIONING", value => "custom_btrfs" },
{ key => "ROOT_PASSWORD", value => "weakpassword" },
],
variables => "",
},
{
name => "server_ext3",
prio => 15,
settings => [
{ key => "PARTITIONING", value => "custom_ext3" },
{ key => "ROOT_PASSWORD", value => "weakpassword" },
],
variables => "",
},
{
name => "server_lvmthin",
prio => 16,
settings => [
{ key => "PARTITIONING", value => "custom_lvmthin" },
{ key => "ROOT_PASSWORD", value => "weakpassword" },
],
variables => "",
},
{
add a 'no swap partition' test Summary: since we did this live at Flock today, I figured I'd tidy it up and submit it. This is an 'optional' test, but some people do run this way so it'd be nice to have it. This adds another little helper method in anacondatest.pm, for deleting partitions, which works much like the others added in previous commits. Test Plan: Schedule a test run, see if the test runs and works. Reviewers: jskladan, garretraziel Reviewed By: jskladan, garretraziel Subscribers: tflink Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D503
2015-08-19 21:41:41 +00:00
name => "server_no_swap",
add ext3, btrfs and LVM thin tests, tweaks to custom methods Summary: This adds three new custom storage tests and some needles to support them, and tweaks the custom storage methods a bit to address some things that cropped up in writing the tests. A new method is added for changing the filesystem, as that's a distinct operation from changing the device type. This also restores the previous behaviour of select_disks() where it handled selecting custom partitioning when needed. Turns out it's pretty common to use regex'es in perl! Who'd'a thought. A corresponding commit to add the tests to openqa_fedora_tools is coming. There's no post-install step for the tests yet; I'll try and write those up and add them soon. Test Plan: Do a full run, including the new tests, on Alpha RC2 and check all are scheduled correctly and run correctly. The LVM thinp test is expected to fail as it catches a genuine bug. Reviewers: jskladan, garretraziel Reviewed By: garretraziel Subscribers: tflink Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D490
2015-08-10 18:01:12 +00:00
prio => 17,
add a 'no swap partition' test Summary: since we did this live at Flock today, I figured I'd tidy it up and submit it. This is an 'optional' test, but some people do run this way so it'd be nice to have it. This adds another little helper method in anacondatest.pm, for deleting partitions, which works much like the others added in previous commits. Test Plan: Schedule a test run, see if the test runs and works. Reviewers: jskladan, garretraziel Reviewed By: jskladan, garretraziel Subscribers: tflink Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D503
2015-08-19 21:41:41 +00:00
settings => [
{ key => "PARTITIONING", value => "custom_no_swap" },
{ key => "ROOT_PASSWORD", value => "weakpassword" },
],
variables => "",
},
{
name => "server_kickstart_hdd",
prio => 18,
add kickstart hdd testsuite
2015-03-12 09:58:04 +00:00
settings => [
{ key => "KICKSTART", value => "1" },
{ key => "GRUB", value => "inst.ks=hd:vdb1:/root-user-crypted-net.ks" },
{ key => "NUMDISKS", value => "2" },
fix typo
2015-03-12 10:11:47 +00:00
{ key => "HDD_2", value => "disk_ks.img" },
add kickstart hdd testsuite
2015-03-12 09:58:04 +00:00
{ key => "ROOT_PASSWORD", value => "111111" },
{ key => "USER_LOGIN", value => "test" },
{ key => "USER_PASSWORD", value => "test" },
],
variables => "",
},
Add fedup_minimal test Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D358
2015-05-13 11:00:46 +00:00
{
convert upgrade tests to dnf-plugin-system-upgrade Summary: This is a first cut which more or less works for now. Issues: 1) We're not really testing the BUILD, here. All the test does is try and upgrade to the specified VERSION - so it'll be using the latest 'stable' for the given VERSION at the time the test runs. This isn't really that terrible, but especially for TC/RC validation, we might want to make things a bit more elaborate and set up the repo for the actual BUILD (and disable the main repos). 2) We'd actually need --nogpgcheck for non-Rawhide, at one specific point in the release cycle - after Branching but before Bodhi activation (which is when we can be sure all packages are signed). This won't matter until 24 branches, and maybe releng will have it fixed by then...if not, I'll tweak it. 3) We don't really test that the upgrade actually *happened* for desktop, at the moment - the only thing in the old test that really checked that was where we checked for the fedup boot menu entry, but that has no analog in dnf. What we should probably do is check that GUI login works, then switch to a console and check /etc/fedora-release just as the minimal test does. Test Plan: Run the tests. Note that creating the desktop disk image doesn't work ATM, so I can't verify the desktop test works, but the minimal one seems to (with D565). There'll be a matching diff for openqa_fedora_tools to update the test case names there. Reviewers: jskladan, garretraziel Reviewed By: jskladan, garretraziel Subscribers: tflink Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D567
2015-09-10 21:49:13 +00:00
name => "upgrade_minimal_64bit",
add a 'no swap partition' test Summary: since we did this live at Flock today, I figured I'd tidy it up and submit it. This is an 'optional' test, but some people do run this way so it'd be nice to have it. This adds another little helper method in anacondatest.pm, for deleting partitions, which works much like the others added in previous commits. Test Plan: Schedule a test run, see if the test runs and works. Reviewers: jskladan, garretraziel Reviewed By: jskladan, garretraziel Subscribers: tflink Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D503
2015-08-19 21:41:41 +00:00
prio => 19,
Add fedup_minimal test Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D358
2015-05-13 11:00:46 +00:00
settings => [
{ key => "ROOT_PASSWORD", value => "weakpassword" },
{ key => "BOOTFROM", value => "c" },
separate 32bit and 64bit upgrade tests Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D535
2015-08-27 11:49:09 +00:00
{ key => "HDD_1", value => "disk_f22_minimal_x86_64.img" },
Add fedup_minimal test Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D358
2015-05-13 11:00:46 +00:00
{ key => "UPGRADE", value => "minimal" },
],
variables => "",
},
add fedup desktop test Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D451
2015-07-17 10:52:56 +00:00
{
convert upgrade tests to dnf-plugin-system-upgrade Summary: This is a first cut which more or less works for now. Issues: 1) We're not really testing the BUILD, here. All the test does is try and upgrade to the specified VERSION - so it'll be using the latest 'stable' for the given VERSION at the time the test runs. This isn't really that terrible, but especially for TC/RC validation, we might want to make things a bit more elaborate and set up the repo for the actual BUILD (and disable the main repos). 2) We'd actually need --nogpgcheck for non-Rawhide, at one specific point in the release cycle - after Branching but before Bodhi activation (which is when we can be sure all packages are signed). This won't matter until 24 branches, and maybe releng will have it fixed by then...if not, I'll tweak it. 3) We don't really test that the upgrade actually *happened* for desktop, at the moment - the only thing in the old test that really checked that was where we checked for the fedup boot menu entry, but that has no analog in dnf. What we should probably do is check that GUI login works, then switch to a console and check /etc/fedora-release just as the minimal test does. Test Plan: Run the tests. Note that creating the desktop disk image doesn't work ATM, so I can't verify the desktop test works, but the minimal one seems to (with D565). There'll be a matching diff for openqa_fedora_tools to update the test case names there. Reviewers: jskladan, garretraziel Reviewed By: jskladan, garretraziel Subscribers: tflink Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D567
2015-09-10 21:49:13 +00:00
name => "upgrade_desktop_64bit",
add a 'no swap partition' test Summary: since we did this live at Flock today, I figured I'd tidy it up and submit it. This is an 'optional' test, but some people do run this way so it'd be nice to have it. This adds another little helper method in anacondatest.pm, for deleting partitions, which works much like the others added in previous commits. Test Plan: Schedule a test run, see if the test runs and works. Reviewers: jskladan, garretraziel Reviewed By: jskladan, garretraziel Subscribers: tflink Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D503
2015-08-19 21:41:41 +00:00
prio => 20,
add fedup desktop test Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D451
2015-07-17 10:52:56 +00:00
settings => [
{ key => "ROOT_PASSWORD", value => "weakpassword" },
{ key => "BOOTFROM", value => "c" },
separate 32bit and 64bit upgrade tests Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D535
2015-08-27 11:49:09 +00:00
{ key => "HDD_1", value => "disk_f22_desktop_x86_64.img" },
{ key => "UPGRADE", value => "desktop" },
],
variables => "",
},
{
convert upgrade tests to dnf-plugin-system-upgrade Summary: This is a first cut which more or less works for now. Issues: 1) We're not really testing the BUILD, here. All the test does is try and upgrade to the specified VERSION - so it'll be using the latest 'stable' for the given VERSION at the time the test runs. This isn't really that terrible, but especially for TC/RC validation, we might want to make things a bit more elaborate and set up the repo for the actual BUILD (and disable the main repos). 2) We'd actually need --nogpgcheck for non-Rawhide, at one specific point in the release cycle - after Branching but before Bodhi activation (which is when we can be sure all packages are signed). This won't matter until 24 branches, and maybe releng will have it fixed by then...if not, I'll tweak it. 3) We don't really test that the upgrade actually *happened* for desktop, at the moment - the only thing in the old test that really checked that was where we checked for the fedup boot menu entry, but that has no analog in dnf. What we should probably do is check that GUI login works, then switch to a console and check /etc/fedora-release just as the minimal test does. Test Plan: Run the tests. Note that creating the desktop disk image doesn't work ATM, so I can't verify the desktop test works, but the minimal one seems to (with D565). There'll be a matching diff for openqa_fedora_tools to update the test case names there. Reviewers: jskladan, garretraziel Reviewed By: jskladan, garretraziel Subscribers: tflink Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D567
2015-09-10 21:49:13 +00:00
name => "upgrade_minimal_32bit",
separate 32bit and 64bit upgrade tests Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D535
2015-08-27 11:49:09 +00:00
prio => 19,
settings => [
{ key => "ROOT_PASSWORD", value => "weakpassword" },
{ key => "BOOTFROM", value => "c" },
{ key => "HDD_1", value => "disk_f22_minimal_i686.img" },
{ key => "UPGRADE", value => "minimal" },
],
variables => "",
},
{
convert upgrade tests to dnf-plugin-system-upgrade Summary: This is a first cut which more or less works for now. Issues: 1) We're not really testing the BUILD, here. All the test does is try and upgrade to the specified VERSION - so it'll be using the latest 'stable' for the given VERSION at the time the test runs. This isn't really that terrible, but especially for TC/RC validation, we might want to make things a bit more elaborate and set up the repo for the actual BUILD (and disable the main repos). 2) We'd actually need --nogpgcheck for non-Rawhide, at one specific point in the release cycle - after Branching but before Bodhi activation (which is when we can be sure all packages are signed). This won't matter until 24 branches, and maybe releng will have it fixed by then...if not, I'll tweak it. 3) We don't really test that the upgrade actually *happened* for desktop, at the moment - the only thing in the old test that really checked that was where we checked for the fedup boot menu entry, but that has no analog in dnf. What we should probably do is check that GUI login works, then switch to a console and check /etc/fedora-release just as the minimal test does. Test Plan: Run the tests. Note that creating the desktop disk image doesn't work ATM, so I can't verify the desktop test works, but the minimal one seems to (with D565). There'll be a matching diff for openqa_fedora_tools to update the test case names there. Reviewers: jskladan, garretraziel Reviewed By: jskladan, garretraziel Subscribers: tflink Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D567
2015-09-10 21:49:13 +00:00
name => "upgrade_desktop_32bit",
separate 32bit and 64bit upgrade tests Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D535
2015-08-27 11:49:09 +00:00
prio => 20,
settings => [
{ key => "ROOT_PASSWORD", value => "weakpassword" },
{ key => "BOOTFROM", value => "c" },
{ key => "HDD_1", value => "disk_f22_desktop_i686.img" },
add fedup desktop test Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D451
2015-07-17 10:52:56 +00:00
{ key => "UPGRADE", value => "desktop" },
],
variables => "",
},
add updates.img via local media test Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D496
2015-08-17 15:36:40 +00:00
{
name => "server_updates_img_local",
add guided shrink test Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D561
2015-09-08 13:54:22 +00:00
prio => 21,
add updates.img via local media test Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D496
2015-08-17 15:36:40 +00:00
settings => [
{ key => "NUMDISKS", value => "2" },
{ key => "HDD_2", value => "disk_updates_img.img" },
{ key => "BOOT_UPDATES_IMG_URL", value => "1" },
{ key => "GRUB", value => "inst.updates=hd:LABEL=UPDATES_IMG:/updates.img" },
],
variables => "",
},
add guided shrink test Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D561
2015-09-08 13:54:22 +00:00
{
name => "server_shrink_ext4",
prio => 22,
settings => [
{ key => "ROOT_PASSWORD", value => "weakpassword" },
{ key => "HDD_1", value => "disk_shrink_ext4.img" },
{ key => "PARTITIONING", value => "guided_shrink" },
],
variables => "",
},
{
name => "server_shrink_ntfs",
prio => 23,
settings => [
{ key => "ROOT_PASSWORD", value => "weakpassword" },
{ key => "HDD_1", value => "disk_shrink_ntfs.img" },
{ key => "PARTITIONING", value => "guided_shrink" },
],
variables => "",
},
add a french (encrypted) test Summary: this handles Non-English European Language Install. Basically it's a bunch of new screenshots for existing tag names, plus a bit of configurability in _boot_to_anaconda and tweaking some existing needles to do non-text matches. The weird 'half-the- icon' needles are for cases where there may or may not be a warning triangle but we want to click it either way (saves duplicating the needle). This also sets up a convention for tagging what languages a needle is appropriate for. If it's specifically appropriate for one or more languages, a tag ENV-LANGUAGE-(LANGUAGE) should be applied for each language, where (LANGUAGE) is the install language in upper-case ('LANGUAGE' variable, which should also be the string that will be typed into the language selection screen). If the needle ought to be used for *all* languages - i.e. it's not a text match, or any text in the match is known not to be translated - the tag ENV-INSTLANG-ALL should be applied. To back this, main.pm now unregisters all needles that are not tagged with either ENV-LANGUAGE-ALL or the tag for the language actually being used (if the LANGUAGE var is not set, we assume english). The point of this is to check the install is actually translated; if we allow all needles to match, the test would pass even if no translations appeared at all. Test Plan: Run all tests and make sure you get the expected results. You can schedule a run against 23 Beta TC1 to see the French test fails 'correctly' when translations are missing. Reviewers: jskladan, garretraziel Reviewed By: garretraziel Subscribers: tflink Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D577
2015-09-15 01:08:58 +00:00
{
name => "european_language_install",
prio => 24,
settings => [
{ key => "LANGUAGE", value => "french" },
{ key => "USER_LOGIN", value => "qwerty" },
{ key => "ROOT_PASSWORD", value => "weakpassword" },
{ key => "ENCRYPT_PASSWORD", value => "weakpassword" },
],
variables => "",
},
OpenQA machines/products template
2015-01-26 10:33:26 +00:00
],
}
Reference in New Issue Copy Permalink