Improve test logic, to be sure #174

2023-05-03T12:42:34Z

AlanMarshall commented

2023-05-03 12:42:34 +00:00

(Migrated from github.com)

When job 00025799 runs on the prod system, it fails but when it is cloned locally:

openqa-clone-job --from https://openqa.rockylinux.org --skip-download 00025799
created job #7117: rocky-8.7-universal-x86_64-Build20230503-Rocky-8.7-x86_64.0-install_serial_console@64bit -> http://localhost/t7117

it passes.
This PR makes a minor improvement in the test logic to get it definitely correct and thereby put my mind at rest.

Tested locally:

openqa-cli api -X POST isos ISO=Rocky-8.8-x86_64-dvd1.iso ARCH=x86_64 DISTRI=rocky FLAVOR=universal VERSION=8.8 DNF_CONTENTDIR=stg DNF_RELEASEVER=8.8-Beta BUILD=-"$(date +%Y%m%d.%H%M%S).0"-8.8-Beta-universal TEST=install_serial_console
openqa-cli api -X POST isos ISO=Rocky-9.1-20221214.1-x86_64-dvd.iso ARCH=x86_64 DISTRI=rocky FLAVOR=universal VERSION=9.1 BUILD=-"$(date +%Y%m%d.%H%M%S).0"-9.1-20221214.1-universal TEST=install_serial_console
openqa-cli api -X POST isos ISO=Rocky-8.7-x86_64-dvd1.iso ARCH=x86_64 DISTRI=rocky FLAVOR=universal VERSION=8.7 BUILD=-"$(date +%Y%m%d.%H%M%S).0"-8.7-20221110-universal TEST=install_serial_console

Result: All pass

When job 00025799 runs on the prod system, it fails but when it is cloned locally: ``` openqa-clone-job --from https://openqa.rockylinux.org --skip-download 00025799 created job #7117: rocky-8.7-universal-x86_64-Build20230503-Rocky-8.7-x86_64.0-install_serial_console@64bit -> http://localhost/t7117 ``` it passes. This PR makes a minor improvement in the test logic to get it definitely correct and thereby put my mind at rest. Tested locally: ``` openqa-cli api -X POST isos ISO=Rocky-8.8-x86_64-dvd1.iso ARCH=x86_64 DISTRI=rocky FLAVOR=universal VERSION=8.8 DNF_CONTENTDIR=stg DNF_RELEASEVER=8.8-Beta BUILD=-"$(date +%Y%m%d.%H%M%S).0"-8.8-Beta-universal TEST=install_serial_console openqa-cli api -X POST isos ISO=Rocky-9.1-20221214.1-x86_64-dvd.iso ARCH=x86_64 DISTRI=rocky FLAVOR=universal VERSION=9.1 BUILD=-"$(date +%Y%m%d.%H%M%S).0"-9.1-20221214.1-universal TEST=install_serial_console openqa-cli api -X POST isos ISO=Rocky-8.7-x86_64-dvd1.iso ARCH=x86_64 DISTRI=rocky FLAVOR=universal VERSION=8.7 BUILD=-"$(date +%Y%m%d.%H%M%S).0"-8.7-20221110-universal TEST=install_serial_console ``` Result: All pass

lumarel (Migrated from github.com) reviewed 2023-05-03 12:42:34 +00:00

akatch (Migrated from github.com) approved these changes 2023-05-03 13:12:03 +00:00

tcooper commented

2023-05-03 15:47:33 +00:00

(Migrated from github.com)

@AlanMarshall It looks to me like you are trying to run a test on production openQA system for code that is not yet merged to the main branch and sync'd to the host.

I think you should follow the Basic example provided in the wiki for openqa-clone-custom-git-refspec to run your PR on the production system.

That would suggest you run...

$ openqa-clone-custom-git-refspec --verbose --dry-run \
    https://github.com/rocky-linux/os-autoinst-distri-rocky/pull/174 \
    https://openqa.rockylinux.org/tests/25799

...and would ultimately produce an openqa-clone-job command that looks like this...

/usr/bin/openqa-clone-job \
  --skip-chained-deps \
  --parental-inheritance \
  --within-instance https://openqa.rockylinux.org 25799 \
  _GROUP=0 \
  TEST+=@AlanMarshall/os-autoinst-distri-rocky#serial_console_install \
  BUILD=AlanMarshall/os-autoinst-distri-rocky#174 \
  CASEDIR=https://github.com/AlanMarshall/os-autoinst-distri-rocky.git#serial_console_install \
  PRODUCTDIR=os-autoinst-distri-rocky \
  NEEDLES_DIR=rocky/needles

I think you'll benefit from doing it yourself but am happy to help or do it for you. Let me know if you need a hand...

@AlanMarshall It looks to me like you are trying to run a test on production openQA system for code that is not yet merged to the `main` branch and sync'd to the host. I think you should follow the Basic example provided in the wiki for [`openqa-clone-custom-git-refspec`](https://wiki.rockylinux.org/team/testing/dev_guides/openqa_clone_custom_git_refspec_examples/#basic-openqa-clone-custom-git-refspec) to run your PR on the production system. That would suggest you run... ```bash $ openqa-clone-custom-git-refspec --verbose --dry-run \ https://github.com/rocky-linux/os-autoinst-distri-rocky/pull/174 \ https://openqa.rockylinux.org/tests/25799 ``` ...and would ultimately produce an `openqa-clone-job` command that looks like this... ```bash /usr/bin/openqa-clone-job \ --skip-chained-deps \ --parental-inheritance \ --within-instance https://openqa.rockylinux.org 25799 \ _GROUP=0 \ TEST+=@AlanMarshall/os-autoinst-distri-rocky#serial_console_install \ BUILD=AlanMarshall/os-autoinst-distri-rocky#174 \ CASEDIR=https://github.com/AlanMarshall/os-autoinst-distri-rocky.git#serial_console_install \ PRODUCTDIR=os-autoinst-distri-rocky \ NEEDLES_DIR=rocky/needles ``` I think you'll benefit from doing it yourself but am happy to help or do it for you. Let me know if you need a hand...

tcooper (Migrated from github.com) approved these changes 2023-05-03 15:48:50 +00:00

tcooper (Migrated from github.com) left a comment

That looks better.

AlanMarshall commented

2023-05-03 16:20:51 +00:00

(Migrated from github.com)

Thanks @tcooper I'll have a go at it and see how I get on.

AlanMarshall commented

2023-05-03 18:19:06 +00:00

(Migrated from github.com)

The test

install_serial_console@AlanMarshall/os-autoinst-distri-rocky#serial_console_install@64bit](https://openqa.rockylinux.org/tests/25837

run in prod system passed so that gives a complete set of successful tests.
I must admit to being puzzled why job #25837 passed and job #25799 failed given that both ran in prod.

The test ``` install_serial_console@AlanMarshall/os-autoinst-distri-rocky#serial_console_install@64bit](https://openqa.rockylinux.org/tests/25837 ``` run in prod system passed so that gives a complete set of successful tests. I must admit to being puzzled why job #25837 passed and job #25799 failed given that both ran in prod.

Sign in to join this conversation.

No reviewers

lumarel

akatch

tcooper