Commit Graph

25 Commits

Author SHA1 Message Date
Adam Williamson
a63437527c upgrade_preinstall: use UP1REL not CURRREL
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-05-26 19:04:57 -07:00
Adam Williamson
acc46464f8 Revise release variable handling, prerelease checks, os-release
I started out trying to fix os-release for the recent change to
add "Prerelease" tags to the VERSION and PRETTY_NAME fields, then
things spiralled. It got me thinking about the awkward DEVELOPMENT
variable we use, so I decided to get rid of it and refactor the
few things that use it. I refactored the anaconda prerelease tag
check, and wrote a new giant comment that gives details about
exactly how anaconda decides whether to show those tags, to give
context to our choices about when to expect them. This check now
uses a new LABEL variable the scheduler now sets. I also wound up
creating new UP1REL and UP2REL vars to define the 'source' release
for upgrade tests, separate from CURRREL and PREVREL, which are
now never lies - they really are the current stable and previous
stable release, even for update upgrade tests.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-05-07 15:42:08 -07:00
Adam Williamson
55a2229e4c Fix upgrade workaround stuff for upgrade_2
Initial implementation wasn't correct, I forgot CURRREL is not
'the pre-upgrade release version' but just 'the current stable
release'. This is a dumb way to figure out the correct release
number for this context but off-hand I can't think of a better
one.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-03-12 09:30:51 -07:00
Adam Williamson
e7c6501737 Extend update test workaround mechanism to upgrade tests
To cover cases like  where we need to apply a workaround
to the pre-upgrade environment.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-03-06 19:21:44 -08:00
Adam Williamson
8754611eef Extend some boot timeouts in upgrade tests
Sometimes rebooting during upgrade tests seems to take longer
than these timeouts allow, so let's bump them a bit.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-05-11 09:02:21 -07:00
Michel Normand
47b2855d55 Add --refresh to force dnf synchronisation
because started qcow2 may be obsolete for update repo.

Note: despite deprecated "update" alias,
continue to use it rather than "upgrade" command.

Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2017-09-06 08:43:04 +02:00
Adam Williamson
df2c3cd906 Test upgrade of FreeIPA server and client deployment
Summary:
This adds an upgrade variant of the FreeIPA tests, with only
the simplest client enrolment (sssd) for now. The server test
starts from the N-1 release and deploys the domain controller
role. The client test similarly starts from the N-1 release
and, when the server is deployed, enrols as a domain client.
Then the server upgrades itself, while the client waits (as the
server is its name server). Then the client upgrades itself,
while the server does some self-checks. The server then waits
for the client to do its checks before decommissioning itself,
as usual. So, summary: *deployment* of both server and client
occurs on N-1, then both are upgraded, then the actual *checks*
occur on N.

In my testing, this all more or less works, except the role
decommission step fails. This failure seems to be a genuine one
so far as I can tell; I intend to file a bug for it soon.

Test Plan:
Run the new tests, check they work. Run the existing
FreeIPA tests (both the compose and the update variants), check
they both behave the same.

Reviewers: jsedlak, jskladan

Reviewed By: jsedlak

Subscribers: tflink

Differential Revision: https://phab.qa.fedoraproject.org/D1204
2017-06-02 12:17:07 -07:00
Adam Williamson
e68e113f76 Remove test_flags comments, add ignore_failure flag
It's not really a good idea to have the comments that explain
the test_flags in *every* test, because they can go stale and
then we either have to live with them being old or update them
all. Like, now. So let's just take 'em all out. There's always
a reference in the openQA and os-autoinst docs, and those get
updated faster.

More importantly, add the new `ignore_failure` flag to relevant
tests - all the tests that don't have the 'important' or
'fatal' flag at present. Upstream killed the 'important' flag
(making all tests 'important' by default), I got it replaced
with the 'ignore_failure' flag, we now need to explicitly mark
all modules we want the 'ignore_failure' behaviour for.
2017-04-10 15:00:10 -07:00
Adam Williamson
602a1ee101 Call boot_decrypt properly (function not method)
Noticed in e.g. https://openqa.fedoraproject.org/tests/58798
we're doing this wrong, `boot_decrypt` was moved into utils as
a function, but we were still calling it as a method...
2017-03-02 14:32:17 -08:00
Adam Williamson
b67f604894 Move all remaining utility functions into exporter modules
Summary:
This adds a couple of new exporter modules, renames main_common
to utils (this is a better name: openSUSE's main_common is
functions used in main.pm, utils is what they call their module
full of miscellaneous commonly-used functions), and moves a
bunch of utility functions that were previously needlessly
implemented as instance methods in base classes into the
exporter modules. That means we can get rid of all the annoying
$self-> syntax for calling them.

We get rid of `fedorabase` entirely, as it's no longer useful
for anything. Other base classes keep the 'standard' methods
(like `post_fail_hook`) and methods which actually need to be
methods (like `root_console`, whose behaviour is different in
anacondatest and installedtest).

Test Plan:
Do a full test suite run and check everything lines
up. There should be no functional differences from before at all,
this is just a re-org.

Reviewers: jskladan, garretraziel_but_actually_jsedlak_who_uses_stupid_nicknames

Reviewed By: garretraziel_but_actually_jsedlak_who_uses_stupid_nicknames

Subscribers: tflink

Differential Revision: https://phab.qa.fedoraproject.org/D1080
2017-01-17 23:15:44 -08:00
Adam Williamson
dcb68d93c8 drop our implementation of script_run in favour of os-autoinst
Summary:
os-autoinst implements `script_run` itself now, we aren't
required to implement it ourselves any more. os-autoinst's
implementation is better than ours, as it allows for verifying
the script actually ran (via the redirect-output-to-serial-
console trick).

So this drops our implementation so we'll just use the upstream
one. Where I judged we don't want to bother with the 'check
the command actually ran' feature I've adjusted our direct
`script_run` calls to pass a wait time of 0, which skips the
'wait for command to run' stuff entirely and just does a simple
'type the string and hit enter'.

Because of how the inheritance works, our `assert_script_run`
calls already used the os-autoinst `script_run`, rather than
the one from our distribution.

This should prevent `prepare_test_packages` sometimes going
wrong right after removing the python3-kickstart package, as
we'll properly wait for that removal to complete now (before
we weren't, we'd just start typing the next command while it
was still running, which could result in lost keypresses).

Test Plan:
Check all tests still run OK (I've tried this on
staging and it seems fine).

Reviewers: jskladan, garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D1034
2016-10-20 09:24:48 -07:00
Adam Williamson
e9ce14a891 consolidate login waits, use postinstall not entrypoint for base
Summary:
I started out wanting to fix an issue I noticed today where
graphical upgrade tests were failing because they didn't wait
for the graphical login screen properly; the test was sitting
at the 'full Fedora logo' state of plymouth for a long time,
so the current boot_to_login_screen's wait_still_screen was
triggered by it and the function wound up failing on the
assert_screen, because it was still some time before the real
login screen appeared.

So I tweaked the boot_to_login_screen implementation to work
slightly differently (look for a login screen match, *then* -
if we're dealing with a graphical login - wait_still_screen
to defeat the 'old GPU buffer showing login screen' problem
and assert the login screen again). But while working on it,
I figured we really should consolidate all the various places
that handle the bootloader -> login, we were doing it quite
differently in all sorts of different places. And as part of
that, I converted the base tests to use POSTINSTALL (and thus
go through the shared _wait_login tests) instead of handling
boot themselves. As part of *that*, I tweaked main.pm to not
require all POSTINSTALL tests have the _postinstall suffix on
their names, as it really doesn't make sense, and renamed the
tests.

Test Plan: Run all tests, see if they work.

Reviewers: jskladan, garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D1015
2016-09-27 11:48:15 -07:00
Adam Williamson
683b819bf9 extend the wait_still_screen time in upgrade_pre too
I have a better fix for this coming, but it's a big change that
requires review, so for now, do this. The tests are actually
failing because the 30 second wait_still_screen is too short(!)
- it's triggering while the test is sitting at the 'full Fedora
logo' bootsplash screen then doing a 30 second assert_screen
graphical_login, which is failing because it actually takes
more than 60 seconds to get from the 'full F' screen to the
login screen. So let's just make the wait_still_screen 45
seconds for now.
2016-09-24 16:39:52 -07:00
Adam Williamson
032d8f76a3 give upgrade_preinstall a bit longer for desktop startup
90 seems too tight. See:
https://openqa.fedoraproject.org/tests/36079
2016-09-24 14:28:20 -07:00
Adam Williamson
aacd01ea8b add encrypted workstation upgrade tests (current and previous)
Summary:
This requires us to handle decryption each time we reboot in
the upgrade process, so factor that little block out into the
base class so we don't have to keep pasting it. It's also a
bit tricky to integrate into the 'catch a boot loop' code we
have to deal with , but I think this should work. There
is a matching openqa_fedora_tools diff to generate the disk
image.

Test Plan:
Run the tests, check that they work, run the other
upgrade and encrypted install tests and check they still work
properly too.

Reviewers: garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D922
2016-07-08 08:56:57 -07:00
Adam Williamson
92ba718de3 dnf-plugin-system-upgrade: don't use updates-testing, 10 mins
the updates-testing here was never meant to be permanent, it
was only added when the plugin was very new and we knew the
version in stable was busted. The test cases do not say to use
u-t, so let's not.

We've seen this test fail several times recently because of very
slow metadata downloads at this point, so let's give it longer
to run.
2016-05-06 13:37:19 -07:00
Adam Williamson
880bc477a0 fix mistake: graphical_login still time should stay 30 secs
I didn't mean to drop it to 15, only meant to extend the timeout
to 90.
2016-05-05 16:41:08 -07:00
Adam Williamson
2df55efb49 add desktop_terminal test, refactor test loading a bit
Summary:
I really just want to add the desktop_terminal test, but I think
this refactor is in order now. It splits up loading of the
various test phases (much as SUSE do it) and allows us to run
the post-install tests without the install tests, for e.g. I
tweaked things to allow the upgrade tests to use the existing
_wait_login tests for final login and combine the two upgrade
postinstall tests into one simple one.

This comes with a bit of a behaviour change to make graphical
wait login behave the same as console wait login: it will log
in unless USER_LOGIN is set to 'false'. Previously it only
logged in if both USER_LOGIN and USER_PASSWORD were set, which
I don't think ever happened in a graphical test, so we never
actually did a graphical login. The intent here is we should do
a login on the default_install tests. That's going a bit beyond
the test case, but it seems like a reasonable thing to test. We
can set USER_LOGIN to false if we don't want to do it.

Test Plan:
Do a full test run, make sure the new tests work and
no old tests break.

Reviewers: jskladan, garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D839
2016-05-05 16:39:47 -07:00
Adam Williamson
aa20bab713 use assert_script_run when possible
Summary:
D637 / ec6b3ff4 switched from using needle matches to using
validate_script_output when we want to run a console command
and check the result. validate_script_output is kinda over-
powered when all you want to do is check the command succeeded
(returned 0), though. testapi provides assert_script_run for
doing exactly that - it runs a script and fails if the script
fails (returns anything but 0). This gives us cleaner code and
is slightly more robust; validate_script_output uses the mini
web server on the worker, which I've occasionally seen crap
out, so it seems good to avoid using it when possible. assert_
script_run doesn't need it.

Test Plan: Check all (affected) tests still work properly.

Reviewers: jskladan, garretraziel

Reviewed By: jskladan, garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D714
2016-01-12 09:27:14 -08:00
Jan Sedlák
ec6b3ff4a3 use validate_script_output instead of typing and needles matching
Use validate_script_output and regex matching instead
of type_string and needles.

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D637
2015-11-04 14:38:36 +01:00
Adam Williamson
52ceed6f39 upgrade: disable screen blanking before long-running commands
Summary:
Updating the stable release prior to doing the update can take
a long time if the image hasn't been updated for a while, and
the upgrade download process itself can take a long time. If
the screen blanks out in either case, either the following
needle match may fail (if we're waiting for a needle) or 'still
screen' may be detected early (if we're waiting for a still
screen), so let's disable screen blanking to avoid it.

Test Plan: Run the upgrade tests and see if they work.

Reviewers: garretraziel, jskladan

Reviewed By: jskladan

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D628
2015-10-26 18:02:22 -07:00
Adam Williamson
68acecb6d4 convert upgrade tests to dnf-plugin-system-upgrade
Summary:
This is a first cut which more or less works for now. Issues:

1) We're not really testing the BUILD, here. All the test does
is try and upgrade to the specified VERSION - so it'll be using
the latest 'stable' for the given VERSION at the time the test
runs. This isn't really that terrible, but especially for TC/RC
validation, we might want to make things a bit more elaborate
and set up the repo for the actual BUILD (and disable the main
repos).

2) We'd actually need --nogpgcheck for non-Rawhide, at one
specific point in the release cycle - after Branching but
before Bodhi activation (which is when we can be sure all
packages are signed). This won't matter until 24 branches, and
maybe releng will have it fixed by then...if not, I'll tweak
it.

3) We don't really test that the upgrade actually *happened*
for desktop, at the moment - the only thing in the old test
that really checked that was where we checked for the fedup
boot menu entry, but that has no analog in dnf. What we should
probably do is check that GUI login works, then switch to a
console and check /etc/fedora-release just as the minimal test
does.

Test Plan:
Run the tests. Note that creating the desktop disk
image doesn't work ATM, so I can't verify the desktop test
works, but the minimal one seems to (with D565). There'll be
a matching diff for openqa_fedora_tools to update the test
case names there.

Reviewers: jskladan, garretraziel

Reviewed By: jskladan, garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D567
2015-09-10 14:49:13 -07:00
Jan Sedlák
e000209967 add comments and documentation
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D481
2015-08-05 08:23:59 +02:00
Adam Williamson
4b8e411479 create fedora base class, factor out console login
Summary:
Root console in anaconda got broken by RHBZ  - no
shell on tty2. Decided to clean up console use in general as
part of fixing it.

This creates a class 'fedorabase' and has 'anacondalog' and
'fedoralog' both inherit from it. boot_to_login_screen is
moved there (as it seems appropriate) and it has a new
method, console_login, which basically handles 'get me a
shell on a console': if we're already at one it returns,
if not it'll type the user name and the password *if
necessary* (sometimes it's not) and return once it sees a
prompt. It takes a hash of named parameters for user,
password and 'check', which is whether it should die if it
fails to reach a console or not (some users don't want it
to).

anacondalog and fedoralog both get 'root_console' methods
which do something appropriate and then call
console_login; both have a hash of named parameters,
anacondalog's version only bothers with 'check', while
fedoralog's also accepts 'tty' to pick the tty to use.

This also adjusts all things which try to get to a console
prompt to use either root_console or console_login as
appropriate.

It also tweaks the needle tags a bit, drops some unneeded
needles, and adds a new 'user console prompt' needle; we
really just need two versions of the root prompt needle
and two of the user prompt needle (one for <F23, one for
F23+ - the console font changed in F23, and the @ character
at least doesn't match between the two). I think we still
need the <F23 case for upgrade tests, for now.

Test Plan:
Do a full test run and see that more tests
succeed. I've done a run on happyassassin with a hack to
workaround the SELinux issue for interactive installs,
and the results look good. I also fiddled about a bit to
test some different cases, like forcing a failure in a
live test to test post_fail_hook (and hence root_console)
in that scenario, and forcing failures after some console
commands had been run to check that it DTRT when we've
already reached a console, etc.

Reviewers: jskladan, garretraziel

Reviewed By: jskladan, garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D462
2015-07-22 11:24:40 -07:00
Garret Raziel
c8f02969f2 add fedup desktop test
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D451
2015-07-17 12:52:56 +02:00