use compose repository (not master repo) for most tests

Summary:
we have a long-standing problem with all the tests that hit
the repositories. The tests are triggered as soon as a compose
completes. At this point in time, the compose is not synced to
the mirrors, where the default 'fedora' repo definition looks;
the sync happens after the compose completes, and there is also
a metadata sync step that must happen after *that* before any
operation that uses the 'fedora' repository definition will
actually use the packages from the new compose. Thus all net
install tests and tests that installed packages have been
effectively testing the previous compose, not the current one.

We have some thoughts about how to fix this 'properly' (such
that the openQA tests wouldn't have to do anything special,
but their 'fedora' repository would somehow reflect the compose
under test), but none of them is in place right now or likely
to happen in the short term, so in the mean time this should
deal with most of the issues. With this change, everything but
the default_install tests for the netinst images should use
the compose-under-test's Everything tree instead of the 'fedora'
repository, and thus should install and test the correct
packages.

This relies on a corresponding change to openqa_fedora_tools
to set the LOCATION openQA setting (which is simply the base
location of the compose under test).

Test Plan:
Do a full test run, check (as far as you can) tests run sensibly
and use appropriate repositories.

Reviewers: jskladan, garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D989
This commit is contained in:
Adam Williamson 2016-09-01 08:22:59 -07:00
parent ea2f71b2cb
commit ef689e75a9
11 changed files with 54 additions and 35 deletions

View File

@ -198,7 +198,7 @@ sub get_full_repo {
# repo URL with flavor and arch, leave NFS ones alone (as for # repo URL with flavor and arch, leave NFS ones alone (as for
# NFS tests we just use a mounted ISO and the URL is complete) # NFS tests we just use a mounted ISO and the URL is complete)
if ($repourl !~ m/^nfs/) { if ($repourl !~ m/^nfs/) {
$repourl .= "/".lc(get_var("VERSION"))."/Everything/".get_var("ARCH")."/os"; $repourl .= "/Everything/".get_var("ARCH")."/os";
} }
return $repourl; return $repourl;
} }

View File

@ -87,6 +87,19 @@ sub start_cockpit {
} }
} }
sub repo_setup {
# disable updates-testing and use the compose location rather than
# mirrorlist, so we're testing the right packages
my $location = get_var("LOCATION");
assert_script_run 'dnf config-manager --set-disabled updates-testing';
# we use script_run here as the rawhide repo file won't always exist
# and we don't want to bother testing or predicting its existence;
# assert_script_run doesn't buy you much with sed anyway as it'll
# return 0 even if it replaced nothing
script_run "sed -i -e 's,^metalink,#metalink,g' -e 's,^#baseurl.*basearch,baseurl=${location}/Everything/\$basearch,g' /etc/yum.repos.d/{fedora,fedora-rawhide}.repo";
script_run "cat /etc/yum.repos.d/{fedora,fedora-rawhide}.repo";
}
1; 1;
# vim: set sw=4 et: # vim: set sw=4 et:

View File

@ -1,17 +1,17 @@
{ {
"area": [
{
"height": 19,
"type": "match",
"width": 105,
"xpos": 56,
"ypos": 303
}
],
"properties": [], "properties": [],
"tags": [ "tags": [
"anaconda_install_source_selected", "anaconda_install_source_selected",
"ENV-DISTRI-fedora", "ENV-DISTRI-fedora",
"ENV-FLAVOR-server" "ENV-FLAVOR-server"
],
"area": [
{
"width": 115,
"xpos": 56,
"height": 21,
"type": "match",
"ypos": 163
}
] ]
} }

Binary file not shown.

Before

Width:  |  Height:  |  Size: 79 KiB

After

Width:  |  Height:  |  Size: 88 KiB

View File

@ -1481,7 +1481,7 @@
{ {
name => "install_repository_http_graphical", name => "install_repository_http_graphical",
settings => [ settings => [
{ key => "REPOSITORY_GRAPHICAL", value => "http://dl.fedoraproject.org/pub/fedora/linux/development" }, { key => "REPOSITORY_GRAPHICAL", value => "%LOCATION%" },
], ],
}, },
{ {
@ -1497,7 +1497,7 @@
{ {
name => "install_repository_http_variation", name => "install_repository_http_variation",
settings => [ settings => [
{ key => "REPOSITORY_VARIATION", value => "http://dl.fedoraproject.org/pub/fedora/linux/development" }, { key => "REPOSITORY_VARIATION", value => "%LOCATION%" },
], ],
}, },
{ {
@ -1797,7 +1797,7 @@
settings => [ settings => [
{ key => "DESKTOP", value => "kde" }, { key => "DESKTOP", value => "kde" },
{ key => "PACKAGE_SET", value => "kde" }, { key => "PACKAGE_SET", value => "kde" },
{ key => "REPOSITORY_VARIATION", value => "http://dl.fedoraproject.org/pub/fedora/linux/development" }, { key => "REPOSITORY_VARIATION", value => "%LOCATION%" },
{ key => "USER_LOGIN", value => "false" }, { key => "USER_LOGIN", value => "false" },
], ],
}, },

View File

@ -41,7 +41,7 @@ sub run {
assert_script_run 'printf "loremipsum" | kinit test1'; assert_script_run 'printf "loremipsum" | kinit test1';
# change password via CLI (back to batterystaple, as that's what # change password via CLI (back to batterystaple, as that's what
# freeipa_client test expects) # freeipa_client test expects)
assert_script_run 'dnf -y --nogpgcheck install freeipa-admintools'; assert_script_run 'dnf -y install freeipa-admintools';
assert_script_run 'printf "batterystaple\nbatterystaple" | ipa user-mod test1 --password'; assert_script_run 'printf "batterystaple\nbatterystaple" | ipa user-mod test1 --password';
# check we can kinit again # check we can kinit again
assert_script_run 'printf "batterystaple" | kinit test1'; assert_script_run 'printf "batterystaple" | kinit test1';

View File

@ -13,9 +13,9 @@ sub run {
# select appropriate protocol on the network # select appropriate protocol on the network
assert_and_click "anaconda_install_source_on_the_network"; assert_and_click "anaconda_install_source_on_the_network";
send_key "tab"; send_key "tab";
# if we have an NFS repo select NFS (one 'up'), otherwise HTTP (four 'ups') # if we have an NFS repo select NFS (one 'up'), otherwise HTTPS (three 'ups')
my $num; my $num;
$num = get_var("REPOSITORY_GRAPHICAL") =~ m/^nfs:/ ? 1 : 4; $num = get_var("REPOSITORY_GRAPHICAL") =~ m/^nfs:/ ? 1 : 3;
for (my $i=0; $i<$num; $i++){ for (my $i=0; $i<$num; $i++){
send_key "up"; send_key "up";
} }

View File

@ -17,8 +17,8 @@ sub run {
# resolution is working before we proceed) # resolution is working before we proceed)
mutex_lock "freeipa_ready"; mutex_lock "freeipa_ready";
mutex_unlock "freeipa_ready"; mutex_unlock "freeipa_ready";
# disable updates-testing # use compose repo, disable u-t, etc.
assert_script_run 'dnf config-manager --set-disabled updates-testing'; $self->repo_setup();
# do the enrolment # do the enrolment
assert_script_run "echo 'monkeys123' | realm join --user=admin ipa001.domain.local", 300; assert_script_run "echo 'monkeys123' | realm join --user=admin ipa001.domain.local", 300;
} }

View File

@ -20,11 +20,11 @@ sub run {
$self->setup_tap_static("10.0.2.100", "ipa001.domain.local"); $self->setup_tap_static("10.0.2.100", "ipa001.domain.local");
# clone host's resolv.conf to get name resolution # clone host's resolv.conf to get name resolution
$self->clone_host_file("/etc/resolv.conf"); $self->clone_host_file("/etc/resolv.conf");
# we don't want updates-testing for validation purposes # use compose repo, disable u-t, etc.
assert_script_run 'dnf config-manager --set-disabled updates-testing'; $self->repo_setup();
# we need a lot of entropy for this, and we don't care how good # we need a lot of entropy for this, and we don't care how good
# it is, so let's use haveged # it is, so let's use haveged
assert_script_run 'dnf -y --nogpgcheck install haveged', 300; assert_script_run 'dnf -y install haveged', 300;
assert_script_run 'systemctl start haveged.service'; assert_script_run 'systemctl start haveged.service';
# read DNS server IPs from host's /etc/resolv.conf for passing to # read DNS server IPs from host's /etc/resolv.conf for passing to
# rolectl # rolectl

View File

@ -8,11 +8,11 @@ sub run {
assert_script_run 'systemctl is-enabled cockpit.socket'; assert_script_run 'systemctl is-enabled cockpit.socket';
assert_script_run 'systemctl is-active cockpit.socket'; assert_script_run 'systemctl is-active cockpit.socket';
assert_script_run 'firewall-cmd --query-service cockpit'; assert_script_run 'firewall-cmd --query-service cockpit';
# we don't want updates-testing for validation purposes # use compose repo, disable u-t, etc.
assert_script_run 'dnf config-manager --set-disabled updates-testing'; $self->repo_setup();
# install a desktop and firefox so we can actually try it # install a desktop and firefox so we can actually try it
assert_script_run 'dnf -y --nogpgcheck groupinstall "base-x"', 300; assert_script_run 'dnf -y groupinstall "base-x"', 300;
assert_script_run 'dnf -y --nogpgcheck install firefox', 120; assert_script_run 'dnf -y install firefox', 120;
$self->start_cockpit(0); $self->start_cockpit(0);
# quit firefox (return to console) # quit firefox (return to console)
send_key "ctrl-q"; send_key "ctrl-q";

View File

@ -8,7 +8,13 @@ sub run {
# disable screen blanking (download can take a long time) # disable screen blanking (download can take a long time)
script_run "setterm -blank 0"; script_run "setterm -blank 0";
assert_script_run "dnf -y --nogpgcheck --releasever=${release} system-upgrade download", 6000; # use compose repo
$self->repo_setup();
my $params = "-y --releasever=${release}";
if ($release eq "rawhide") {
$params .= " --nogpgcheck";
}
assert_script_run "dnf ${params} system-upgrade download", 6000;
upload_logs "/var/log/dnf.log"; upload_logs "/var/log/dnf.log";
upload_logs "/var/log/dnf.rpm.log"; upload_logs "/var/log/dnf.rpm.log";