Commit Graph

263 Commits

Author SHA1 Message Date
Guy Menanteau
9c8a4949fa Modify "install_multi" test for PowerPC
PowerPC arches have the empty disk automatically
mounted on the second position in anaconda (vdb).
Thus, trig installation on second disk.
Change disk checking to point on correct disk.

Warning: this is a workaround specific correction
addressing a specific case.
This will have to be improved/changed with a more
generic code as suggested by Adam Williamson in
https://pagure.io/fedora-qa/os-autoinst-distri-fedora/pull-request/1#comment-31858
proposal for a next commit :)

Signed-off-by: Guy Menanteau <menantea@linux.vnet.ibm.com>
Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2017-09-06 08:43:04 +02:00
Guy Menanteau
06839422e5 First PowerPC specific changes (not templates)
* New OFW variable to identify Open Firmware (used by PowerPC)
* Few needles changes for PowerPC support
* as requested do not change the timers value below for PowerPC
  tests/install_source_graphical.pm (300 to 600)
  tests/_boot_to_anaconda.pm (300 to 1200)
  This will be handled by TIMEOUT_SCALE in templates

Signed-off-by: Guy Menanteau <menantea@linux.vnet.ibm.com>
2017-09-06 08:43:04 +02:00
Adam Williamson
004121a38d Upload all logs when AVCs and/or crashes detected
Previously we just got a screenshot of some AVCs or coredumps,
which told us something was wrong but didn't really help debug
it. So, let's upload the output of the commands and then also
use the post-fail hook to upload the system logs, which should
give us much more info to work with.
2017-09-05 09:35:59 -07:00
Adam Williamson
60913a20e0 Always upload logs at end of domain_controller test
We often want to see the logs from the FreeIPA deployment test
even if that test passes - to look for some detail that doesn't
cause a test to fail, for instance, or if one of the *client*
tests failed for a reason that involves the server. So, let's
do that.
2017-09-05 09:32:33 -07:00
Michel Normand
53f0acea50 Add allowerasing for upgrade if failed without it
Note:
The dnf.log and dnf.rpm.log are already uploaded
to help for investigation.

Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2017-09-05 13:38:14 +02:00
Adam Williamson
a7a6219276 Re-confirm passphrase when working around #1444225
When we hit #1444225, as well as hitting Done a second time, we
have to confirm the passphrase again to work around it.
2017-07-26 15:05:55 -07:00
Adam Williamson
81ebcd882c Install libglvnd-egl when installing base-x, X seems to need it
On Rawhide, X fails to start unless this is installed. Should
probably be in the base-x group, I'll file a bug or add it.
2017-07-24 18:58:31 -07:00
Adam Williamson
6443d364b9 Fix a typo in the RHBZ#1444225 workaround 2017-07-10 15:15:41 -07:00
Adam Williamson
9a7d51f841 Tweak _graphical_input to work with transparent top bar
At least, I hope this will work.
2017-07-10 14:41:57 -07:00
Adam Williamson
e61cfa699c Try a different workaround for RHBZ #1444225
That other one didn't help, so let's try this - try and spot if
the spoke is in the unexpected state (the needle should only
match if the spoke is done processing and still in warning
state, it shouldn't match while the needle is still thinking)
and click through it again if so.
2017-07-10 12:24:41 -07:00
Adam Williamson
aca7de2861 Change up 'clean desktop' check again (use a util function)
Well, that OCR needle isn't working out so great, as it seems
to match when it shouldn't:

https://openqa.fedoraproject.org/tests/119217#step/_graphical_wait_login/5

So let's try another approach. Ditch the OCR needle and have a
function for checking we're at a clean desktop. It does the
normal needle match, but if we're on GNOME, it also tries
hitting alt+f1 and seeing if we're at the overview; if so, it
hits alt+f1 again (to go back to the desktop) and returns.
2017-07-10 11:47:07 -07:00
Adam Williamson
ddb6f8e416 Adjust cockpit log needles for Rawhide change
The Logs screen changed a bit. Also, it seems wrong to have
'notices' in all these needle names, so let's fix that.
2017-07-10 10:35:27 -07:00
Adam Williamson
d119e1e53e Check for NFS repo message in anaconda.log too
It seems to have moved there in F27.
2017-07-10 10:21:57 -07:00
Adam Williamson
ee2a3daefd Try to work around #1444225 with a sleep in disk_guided_empty
IIRC disk_guided_empty is the only storage test that clicks
through INSTALLATION DESTINATION *really* fast, so let's try
adding a 2-second sleep to it to see if it works around the
'sometimes spoke shows as incomplete' bug that cropped up in
F26 and hasn't been fixed yet and tends to cause failures.
2017-07-06 13:15:20 -07:00
Adam Williamson
09c264fe16 Add Workstation dvd-ostree flavor and tests
Summary:
As we're getting the Workstation dvd-ostree (OStree installer
image) built for Rawhide now, let's try testing it.

Test Plan:
Run the tests on a Rawhide compose that works and
has the image (e.g. 20170615.n.0). Check that new tests work
as expected and old tests are not adversely affected. A
corresponding diff for fedora_openqa will be coming to take
care of scheduling. Note that the tests will often soft fail
for now; this is intentional due to RHBZ#1193590, the bash
prompt for root is incorrect on ostree installs, so I have
added a needle that matches the incorrect prompt but which is
flagged as a workaround needle (so causing the test result to
be a soft fail).

Reviewers: jsedlak, jskladan

Reviewed By: jsedlak

Subscribers: tflink

Differential Revision: https://phab.qa.fedoraproject.org/D1211
2017-06-26 18:48:27 -07:00
Adam Williamson
a29875a2ff Upgrade tests: Run FreeIPA webUI and password change modules
Summary:
This just adds the FreeIPA web UI and password change
test modules to the FreeIPA upgrade test (client end). It's
useful to check out these features too. We don't need to
separate these into separate jobs, as we're not trying to
fill out different matrix checkboxes here, we just want to
know whether everything works.

Test Plan:
Run the test, see that the modules work properly.
I was actually expecting this to fail given the issues with
the upgrade on the server end, but it seems to pass.

Reviewers: jsedlak, jskladan

Reviewed By: jsedlak

Subscribers: tflink

Differential Revision: https://phab.qa.fedoraproject.org/D1207
2017-06-13 10:06:15 -07:00
Adam Williamson
25b910135b Workaround cursor showing up on GDM in desktop_notifications 2017-06-06 18:16:52 -07:00
Adam Williamson
df2c3cd906 Test upgrade of FreeIPA server and client deployment
Summary:
This adds an upgrade variant of the FreeIPA tests, with only
the simplest client enrolment (sssd) for now. The server test
starts from the N-1 release and deploys the domain controller
role. The client test similarly starts from the N-1 release
and, when the server is deployed, enrols as a domain client.
Then the server upgrades itself, while the client waits (as the
server is its name server). Then the client upgrades itself,
while the server does some self-checks. The server then waits
for the client to do its checks before decommissioning itself,
as usual. So, summary: *deployment* of both server and client
occurs on N-1, then both are upgraded, then the actual *checks*
occur on N.

In my testing, this all more or less works, except the role
decommission step fails. This failure seems to be a genuine one
so far as I can tell; I intend to file a bug for it soon.

Test Plan:
Run the new tests, check they work. Run the existing
FreeIPA tests (both the compose and the update variants), check
they both behave the same.

Reviewers: jsedlak, jskladan

Reviewed By: jsedlak

Subscribers: tflink

Differential Revision: https://phab.qa.fedoraproject.org/D1204
2017-06-02 12:17:07 -07:00
Jan Sedlák
48b99a2291 add base system logging test
Differential Revision: https://phab.qa.fedoraproject.org/D1202
2017-06-01 11:06:04 +02:00
Jan Sedlák
4114406668 add UEFI for blivet tests
Differential Revision: https://phab.qa.fedoraproject.org/D1201
2017-05-22 09:26:58 +02:00
Jan Sedlák
0b5f865c8f add custom btrfs partitioning test for blivet-gui
Differential Revision: https://phab.qa.fedoraproject.org/D1194
2017-05-19 13:58:16 +02:00
Jan Sedlák
ea5296b306 add postinstalls to custom partitioning tests
Differential Revision: https://phab.qa.fedoraproject.org/D1192
2017-05-09 09:11:34 +02:00
Adam Williamson
c2b1ea4e8c Use test3 not test1 for FreeIPA password change test module
Previously this module changed test1's password, so that it
would still be able to work even if the webui test module
failed (so test3/test4 didn't get created). But this means
that, for about 30 seconds, test1's password is 'loremipsum'
not 'batterystaple', and if one of the *other* client test
jobs happens to hit a point where it has to auth as test1
during the 30 seconds test1's password is different, it will
fail. This looks to be what happened to the join_kickstart
test the last few days - it failed because it tried to login
as test1 during the password change window.

By using the test3 user instead (which is only used by the
join_cockpit test, currently) we avoid this problem, at the
cost that the password_change module will always fail if the
webui module fails.
2017-05-04 17:29:35 -07:00
Adam Williamson
7d07147b7e Work around F26 base image bug: g-i-s failing to run
With the latest F26 base images, it seems like g-i-s fails to
run at first login. This is clearly some kind of bug somewhere
and I'll investigate it, but it shouldn't be causing the update
tests to fail - we can still validly run the tests with g-i-s
not running. So for now, adjust the _graphical_wait_login test
to tolerate this behaviour when running update tests.
2017-04-24 18:49:25 -07:00
Jan Sedlák
140c5f0a42 Add custom partitioning tests for blivet
Differential Revision: https://phab.qa.fedoraproject.org/D1188
2017-04-24 14:23:35 +02:00
Adam Williamson
e6076e14b8 Workaround issues with loading same module more than once
Summary:
Loading the same module more than once *kinda* works, but it
shows up all kinds of funky in the openQA web interface. There's
a POO for this:
https://progress.opensuse.org/issues/10514
But it doesn't seem like it's going to be resolved immediately,
so in the mean time maybe we should avoid doing it so we don't
have to deal with the weirdness it produces in the web UI. So
here's a kinda icky hack that uses symlinks and stuff to load
multiple instances of 'the same' test module.

Test Plan:
Run an update test, look at how it looks in the web
UI and confirm it's a lot clearer and less buggy. Check there
aren't any bugs in the loading approach. This is deployed on stg
so you can look at it there.

Reviewers: jsedlak, jskladan

Reviewed By: jsedlak

Subscribers: tflink

Differential Revision: https://phab.qa.fedoraproject.org/D1186
2017-04-18 10:38:16 -07:00
Adam Williamson
9da72de4ed Hide mouse on login screen (BGO #781294)
GDM 3.24.1 dumps a cursor in the middle of the login screen even
if you didn't move the mouse yet, so let's workaround that.
2017-04-13 15:28:37 -07:00
Adam Williamson
ee283b1411 Tweak _console_avc_crash so we get soft failures
We do want soft failures (but not hard) when _console_avc_crash
fails. So let's tweak it to ensure that's what happens.
2017-04-10 15:44:13 -07:00
Adam Williamson
e68e113f76 Remove test_flags comments, add ignore_failure flag
It's not really a good idea to have the comments that explain
the test_flags in *every* test, because they can go stale and
then we either have to live with them being old or update them
all. Like, now. So let's just take 'em all out. There's always
a reference in the openQA and os-autoinst docs, and those get
updated faster.

More importantly, add the new `ignore_failure` flag to relevant
tests - all the tests that don't have the 'important' or
'fatal' flag at present. Upstream killed the 'important' flag
(making all tests 'important' by default), I got it replaced
with the 'ignore_failure' flag, we now need to explicitly mark
all modules we want the 'ignore_failure' behaviour for.
2017-04-10 15:00:10 -07:00
Adam Williamson
5b4ce80487 Tweak needle matches at start of _do_install_and_reboot
The way this was set up before, if `anaconda_main_hub` matched
immediately but some spoke was still in a 'processing' state,
it only had 30 seconds (default `assert_and_click` timeout) to
complete and allow the 'Begin Installation' button to appear.
It seems unnecessary to match on *both* needles, really, so
let's just give 300 seconds for the `begin_installation` needle
to appear. It's not going to appear on any other screen.

This problem caused a couple of spurious failures today -
https://openqa.fedoraproject.org/tests/77839 and
https://openqa.fedoraproject.org/tests/77858 - because they
took a bit too long for the INSTALLATION DESTINATION spoke to
clear.
2017-04-07 11:31:34 -07:00
Adam Williamson
1444c5030b Add tests with no user created during install
Summary:
This adds a new test suite, run for Workstation and KDE live
images, which does not create a user during install. It then
expects initial-setup (KDE) or gnome-initial-setup (Workstation)
to appear after install, creates a user, and proceeds with
normal boot.

Note the ARM image test already covers the initial-setup text
mode, and the ARM minimal image is the only case where that
actually matters (it's not included in Server).

Test Plan:
Run the new tests, check they work. Run all old
tests, check the changes didn't break them.

Reviewers: jsedlak, jskladan

Reviewed By: jsedlak

Subscribers: tflink

Differential Revision: https://phab.qa.fedoraproject.org/D1185
2017-04-05 09:43:26 -07:00
Adam Williamson
4ea5129312 Goddamnit, perl, and your goddamn brackets 2017-03-29 15:09:25 -07:00
Adam Williamson
4f014d4f11 Use tty2 not tty3 for freeipa_client login tests
There's an issue where the follow-on _advisory_post test tries
to log in before the 'login failed' error has cleared. We can
easily avoid this by using tty2 for the login tests, then
_advisory_post will switch to tty3 for its stuff.
2017-03-29 14:33:25 -07:00
Adam Williamson
b6d4fd7d4c Don't create user when USER_LOGIN is false, but for KDE install
Summary:
For some reason, we have `USER_LOGIN` set to 'false' for the KDE
package set install test. I really don't know / remember why
that would be; I'd think we should create a user and log in as
that user to make sure it works properly when installing KDE
from the traditional installer. It's not strictly part of the
package set test, true, but still, seems worth doing.

Also, when `USER_LOGIN` is set to 'false' and the installer runs,
we create a user called 'false'. This doesn't seem like what we
wanted, so let's not do that. I dunno if there are any other
cases besides the KDE one that this commit changes, but still.

Test Plan:
Run the full test suite and look for weirdness, check
KDE package set test works as intended (now creates a user called
'test' and logs in as that user).

Reviewers: jsedlak, jskladan

Reviewed By: jsedlak

Subscribers: tflink

Differential Revision: https://phab.qa.fedoraproject.org/D1182
2017-03-29 09:30:16 -07:00
Adam Williamson
cc2d53b4ed Wait a bit after input switch before doing Japanese typing 2017-03-20 11:32:50 -07:00
Adam Williamson
ba3a5152c1 Improve FreeIPA debug logging a bit
Committing without review as this is pretty trivial and I've
had it on staging for the last few days without issue. Just gets
us somewhat better info for debugging FreeIPA issues.
2017-03-16 12:36:33 -07:00
Adam Williamson
1b26f7c04c Add missing 'use utils' to disk_guided_encrypted_postinstall 2017-03-09 13:57:06 -08:00
Adam Williamson
a581d20f8e Desktop update: Wait before clicking update apply button
We used to do this only for KDE, but I've seen the new update
tests sometimes fail at this point for no apparent reason, and
I'm thinking a wait may help (in case they're clicking the
button before it's really 'ready').
2017-03-02 15:33:23 -08:00
Adam Williamson
8d55935b41 Bump up the ARM boot timeout
Let's see if it can get a bit further, at least...
2017-03-02 15:16:02 -08:00
Adam Williamson
e9fda18016 Explicit timeout for the 'top' in _collect_data
This keeps failing because the default `assert_script_run`
timeout changed from 90 to 30 in the last os-autoinst update
(an unintended consequence of a change I made). This has been
fixed upstream, but in the mean time, let's just set an
explicit timeout on the call.
2017-03-02 14:47:44 -08:00
Adam Williamson
602a1ee101 Call boot_decrypt properly (function not method)
Noticed in e.g. https://openqa.fedoraproject.org/tests/58798
we're doing this wrong, `boot_decrypt` was moved into utils as
a function, but we were still calling it as a method...
2017-03-02 14:32:17 -08:00
Adam Williamson
9aed11a77b Work around RHBZ #1400293 (breaks IPA tests on Fedora 24) 2017-02-28 11:46:21 -08:00
Adam Williamson
461f3a6132 Update testing: log packages in update and installed packages
Summary:
This adds some logging related to the update testing workflow,
so we have some idea what we actually tested. We log precisely
which packages were actually downloaded from the update - this
is important as updates can be edited and when examining results
we'll want to know which packages actually got used. We also
add a new module which runs at the end of postinstall and tries
to figure out which packages from the update were installed in
the course of the test. This still isn't a guarantee the test
actually *tested them* in any way, but it at least means they
got installed successfully and didn't interfere with the test.

Test Plan:
Run the update test workflow, check the logs get
uploaded and seem accurate (sometimes some RPM garbage messages
wind up in the package log, I'm not too worried about that at
present). Run the compose test workflow and check it didn't
break.

Reviewers: jsedlak

Reviewed By: jsedlak

Subscribers: tflink

Differential Revision: https://phab.qa.fedoraproject.org/D1149
2017-02-23 14:51:19 -08:00
Adam Williamson
92d588f245 Add support for testing updates
Summary:
This adds an entirely new workflow for testing distribution
updates. The `ADVISORY` variable is introduced: when set,
`main.pm` will load an early post-install test that sets up
a repository containing the packages from the specified update,
runs `dnf -y update`, and reboots. A new templates file is
added, `templates-updates`, which adds two new flavors called
`updates-server` and `updates-workstation`, each containing
job templates for appropriate post-install tests. Scheduler is
expected to post `ADVISORY=(update ID) HDD_1=(base image)
FLAVOR=updates-(server|workstation)`, where (base image) is one
of the stable release base disk images produced by `createhdds`
and usually used for upgrade testing. This will result in the
appropriate job templates being loaded.

We rejig postinstall test loading and static network config a
bit so that this works for both the 'compose' and 'updates' test
flows: we have to ensure we bring up networking for the tap
tests before we try and install the updates, but still allow
later adjustment of the configuration. We take advantage of the
openQA feature that was added a few months back to run the same
module multiple times, so the `_advisory_update` module can
reboot after installing the updates and the modules that take
care of bootloader, encryption and login get run again. This
looks slightly wacky in the web UI, though - it doesn't show the
later runs of each module.

We also use the recently added feature to specify `+HDD_1` in
the test suites which use a disk image uploaded by an earlier
post-install test, so the test suite value will take priority
over the value POSTed by the scheduler for those tests, and we
will use the uploaded disk image (and not the clean base image
POSTed by the scheduler) for those tests.

My intent here is to enhance the scheduler, adding a consumer
which listens out for critpath updates, and runs this test flow
for each one, then reports the results to ResultsDB where Bodhi
could query and display them. We could also add a list of other
packages to have one or both sets of update tests run on it, I
guess.

Test Plan:
Try a post something like:
HDD_1=disk_f25_server_3_x86_64.img DISTRI=fedora VERSION=25
FLAVOR=updates-server ARCH=x86_64 BUILD=FEDORA-2017-376ae2b92c
ADVISORY=FEDORA-2017-376ae2b92c CURRREL=25 PREVREL=24

Pick an appropriate `ADVISORY` (ideally, one containing some
packages which might actually be involved in the tests), and
matching `FLAVOR` and `HDD_1`. The appropriate tests should run,
a repo with the update packages should be created and enabled
(and dnf update run), and the tests should work properly. Also
test a regular compose run to make sure I didn't break anything.

Reviewers: jskladan, jsedlak

Reviewed By: jsedlak

Subscribers: tflink

Differential Revision: https://phab.qa.fedoraproject.org/D1143
2017-02-22 11:33:32 -08:00
Adam Williamson
186678e98b Make log upload work when installed system hits emergency mode
Summary:
This is to handle cases like #1414904 , where the system boots
to emergency mode. We really need logs to try and debug this.

Test Plan:
Force a test to hit emergency mode somehow (right now
you can just run base_services_start on Rawhide over and over
until you hit #1414904, but there's probably an easier way to
do it, I think there's a systemd boot arg to tell it which target
to boot for e.g.) and check logs get uploaded. Also check this
doesn't break log upload for a 'normal' failure.

Reviewers: garretraziel_but_actually_jsedlak_who_uses_stupid_nicknames

Reviewed By: garretraziel_but_actually_jsedlak_who_uses_stupid_nicknames

Subscribers: tflink

Differential Revision: https://phab.qa.fedoraproject.org/D1103
2017-02-01 12:30:21 +01:00
Adam Williamson
b67f604894 Move all remaining utility functions into exporter modules
Summary:
This adds a couple of new exporter modules, renames main_common
to utils (this is a better name: openSUSE's main_common is
functions used in main.pm, utils is what they call their module
full of miscellaneous commonly-used functions), and moves a
bunch of utility functions that were previously needlessly
implemented as instance methods in base classes into the
exporter modules. That means we can get rid of all the annoying
$self-> syntax for calling them.

We get rid of `fedorabase` entirely, as it's no longer useful
for anything. Other base classes keep the 'standard' methods
(like `post_fail_hook`) and methods which actually need to be
methods (like `root_console`, whose behaviour is different in
anacondatest and installedtest).

Test Plan:
Do a full test suite run and check everything lines
up. There should be no functional differences from before at all,
this is just a re-org.

Reviewers: jskladan, garretraziel_but_actually_jsedlak_who_uses_stupid_nicknames

Reviewed By: garretraziel_but_actually_jsedlak_who_uses_stupid_nicknames

Subscribers: tflink

Differential Revision: https://phab.qa.fedoraproject.org/D1080
2017-01-17 23:15:44 -08:00
Adam Williamson
a091185af1 Handle 'non-ASCII in password' warning for Russian installs
Summary:
Since 26.17, anaconda shows a warning when the user password
contains non-ASCII characters, and requires a second Done click
to confirm. This change should handle that.

On the 'catch cases where password typing went wrong and re-try'
bit: to keep that, but not re-type the password *every single
time* on the Russian install test, we'd have to make the needle
match the text of the warning. This is problematic because then
that needle will be able to break without us easily noticing;
that's why I wanted to keep the 'warning bar' needle text-free.
Unfortunately, that means we have to skip the protection for
switched-layout installs.

Note the protection was actually not working for any non-English
install anyhow, because the needle had `LANGUAGE-english` as a
tag. We never noticed that. Failed password typing is pretty
rare now, so we can live without the protection - it's just nice
to have it for the English install tests because there's so many
of them.

Test Plan:
Run the Russian install with a recent Rawhide image,
check it clicks 'Done' twice. Note, it will still fail, because
of RHBZ #1413813.

Reviewers: jskladan, garretraziel_but_actually_jsedlak_who_uses_stupid_nicknames

Reviewed By: garretraziel_but_actually_jsedlak_who_uses_stupid_nicknames

Subscribers: tflink

Differential Revision: https://phab.qa.fedoraproject.org/D1084
2017-01-17 23:13:47 -08:00
Adam Williamson
062d9f8f5e Add jobs to gather memory usage data
Summary:
This adds a new test, memory_check, which just does a default
package set install with `inst.debug` parameter then uploads
the memory usage file (`/tmp/memory.dat`) at the end. We can
have check-compose use the data to analyze changes in memory
usage over time.

Test Plan:
Fire off the Workstation network install image tests
and make sure the memory usage test runs and works on all three
machines. This is live on staging already.

Reviewers: jskladan, garretraziel_but_actually_jsedlak_who_uses_stupid_nicknames

Reviewed By: garretraziel_but_actually_jsedlak_who_uses_stupid_nicknames

Subscribers: tflink

Differential Revision: https://phab.qa.fedoraproject.org/D1082
2017-01-16 09:30:14 -08:00
Adam Williamson
460a6e5439 Simplify the services test fix
I forgot `script_run` actually returns the exit code. That makes
it easier.
2017-01-11 16:03:09 -08:00
Adam Williamson
c9ede993e5 Fix base_services_start
The old version wasn't working - it was passing even though two
services fail to start in Workstation currently. I'm really not
sure why the old approach wasn't working, but it wasn't, and I
rather hate `script_output` anyway, so here's a different way
of doing it which relies on `eval`ing `assert_script_output`
instead. (I really should send a PR for a non-fatal version of
assert_script_output...)
2017-01-11 13:49:17 -08:00
Adam Williamson
dac3c2dd38 Use systemctl --no-legend when collecting service info
Without this, when there are failed services, we get an extra
column to the left of the service names with a unicode dot for
each failed service, which is awkward and screws up the parsing.
2017-01-11 11:58:37 -08:00
Adam Williamson
c4f32ab5ad add an Asian (Japanese) language install test
Summary:
Include some basic testing of Japanese input, and split the
input testing (including Russian) into a separate module, since
it's not really part of 'login' testing.

Test Plan:
Run the test, and the Russian and French tests too to
make sure they didn't break. Tested on staging. Note the Japanese
test soft fails, intentionally, at present, as I discovered a bug
while working on it:
https://bugzilla.gnome.org/show_bug.cgi?id=776189

Reviewers: jskladan, garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D1072
2016-12-21 08:41:00 -08:00
Adam Williamson
7135a4e05d iSCSI: test user authentication
Summary:
This isn't in the criteria, but it's commonly used, so we ought
to test this way. Require authentication for the iSCSI target
and have the test provide the appropriate auth info.

Test Plan:
Run the iscsi test and check it works (you need the
recent fixes for support_server to make *that* work). Nothing
else should be affected.

Reviewers: jskladan, garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D1070
2016-12-20 08:34:30 -08:00
Adam Williamson
0fa6138448 Have non-English tests do graphical install and login
Summary:
The non-English tests so far did not test that graphical login
worked as expected, which is a fairly large hole. With this
change, they should do a Workstation install and test login to
both GNOME and the console works as expected. KDE is not yet
tested.

As part of this we tweak the implementation of keyboard layout
switching in graphical environments to use a generic function
in main_common which can handle both anaconda and desktops
(just GNOME at present, but should extend easily to any desktop
with a known switcher key and a visible layout indicator),
replacing the anacondatest class method. I kinda don't like that
the test has to specifically tell the function when it's in
anaconda, but I don't think I want to start experimenting with
a global 'test phase' openQA variable or anything like that at
present.

Fixes T842.

Test Plan:
Run the French and Russian install tests and check
they work as expected. Also run an English Workstation install
if you like, and make sure that didn't break. This change is
live on staging ATM, seems to work fine.

Reviewers: jskladan, garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Maniphest Tasks: T842

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D1071
2016-12-16 09:40:29 -08:00
Adam Williamson
d8ba53aefc workaround #1402427 in support_server
support_server was failing because of #1402427. I re-generated
the disk image with latest F25 so the fixed selinux-policy is
used, but even then, it seems we have to run 'restorecon' on
rpcbind manually before starting nfs.
2016-12-13 16:38:22 -08:00
Jan Sedlák
44cf1cd89c reintroduce rescue on UEFI
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D1060
2016-12-13 09:18:34 +01:00
Adam Williamson
5c94924c59 guh, main_common 2016-12-08 12:20:09 -08:00
Adam Williamson
6dc20de0b0 move 'loadkeys us' call into avc_crash test for now
Ugh, there's no good way to do this ATM. If this doesn't work
I might just skip the test for this case.
2016-12-08 12:18:45 -08:00
Adam Williamson
cc7fe4b971 Tweaks to AVC test loading / run
Committing without review as this causes failures...try to make
sure we only run the AVC test when it makes sense, and fix
running it on the French install test.
2016-12-08 12:03:26 -08:00
Adam Williamson
7687a3aa34 Add a non-fatal console AVC / crash post-install check
Summary:
This has all console tests check for AVCs (with ausearch) and
crashes (with coredumpctl) at post-install stage. It's non-
fatal as this doesn't really mean the test failed, but we want
to spot when there are unexpected AVCs or crashes.

Test Plan:
Run some console tests, check it works right. I only
tested with one test, since so many are broken on Rawhide ATM

Reviewers: jskladan, garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D1066
2016-12-08 08:58:29 -08:00
Adam Williamson
62ef578758 Extend a timeout in the graphical update test
The 'refresh' button takes a bit longer to appear on F24 respin
tests.
2016-12-07 17:27:35 -08:00
Adam Williamson
6f90ff0143 collect_data: run top *before* everything else
Committing without review as this is pretty trivial. Running
top *after* all the other collect_data tasks leads to some
meaningless fluctuations in the output; I intended to look for
activity caused by stuff running 'as usual', not activity from
the other collect_data tasks.
2016-12-02 15:01:40 -08:00
Adam Williamson
571cf3c035 bump the initial ARM boot timeout a bit 2016-11-09 07:21:03 -08:00
Adam Williamson
65cadb11df Collect some data on freshly-installed systems after some tests
Summary:
I've been wanting to do this for a while; I think it'll let us
check for some significant changes between composes. This should
cause runs of a few test cases to collect and upload info on:

* installed packages
* free memory
* disk space
* active services
* 1 minute of CPU usage info (via top)

immediately after install and initial login. In some cases this
will be useful / interesting simply to look at directly, but
we can also have check-compose analyze the data and include
significant changes in its reports.

Test Plan:
Run affected tests, make sure the data collection
works.

Reviewers: jskladan, garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D1046
2016-11-09 07:20:17 -08:00
Adam Williamson
7b31b8263e Force GNOME to notify updates, re-enable test on Workstation
Summary:
GNOME's update notification criteria are pretty braindead: it
fires the update check timer once on login then once every hour
thereafter, but only actually checks for and notifies of updates
once a day if it's after 6am(?!?!?!). So we have to do a bunch
of fiddling around to ensure we reliably get a notification.
Move the clock to 6am if it's earlier than that, and reset the
'last update check' timer to 48 hours ago, then log in to GNOME
after that.

Note: I thought this still wasn't fully reliable, but I've looked
into all the recent failures of either test on staging and
there's only one which was really 'no update notification came
up', and the logs clearly indicate PK did run an update check,
so I don't think that was a test bug (I think something went
wrong with the update check). The other failures are all 'GNOME
did something wacky', plus one case where the needle didn't quite
match because I think the match area is slightly too tall; I'll
fix that in a second.

Test Plan:
Run the tests on both KDE and GNOME and check they
work properly now (assuming nothing unrelated breaks, like KDE
crashing...)

Reviewers: jskladan, garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D1039
2016-10-27 16:23:59 -07:00
Adam Williamson
1ae1b6e2cb wait longer for system to shutdown in _console_shutdown
We're not really *testing* shutdown here, we're just shutting
down to make sure the uploaded disk image is clean. So we don't
really mind if shutdown takes a while. It often seems to take
longer than 1 minute on KDE installs and cause a soft fail, so
let's bump the timeout to 3 minutes.
2016-10-26 14:03:15 -07:00
Adam Williamson
b129bf5487 don't wait for dnf system-upgrade reboot command to return
because it won't.
2016-10-21 17:43:04 -07:00
Adam Williamson
f95e1d55a8 give IPA server decommission longer to complete (RHBZ #1387425)
It's suddenly taking longer in Rawhide - reported to RHBZ - so
let it take longer.
2016-10-20 13:44:08 -07:00
Adam Williamson
dcb68d93c8 drop our implementation of script_run in favour of os-autoinst
Summary:
os-autoinst implements `script_run` itself now, we aren't
required to implement it ourselves any more. os-autoinst's
implementation is better than ours, as it allows for verifying
the script actually ran (via the redirect-output-to-serial-
console trick).

So this drops our implementation so we'll just use the upstream
one. Where I judged we don't want to bother with the 'check
the command actually ran' feature I've adjusted our direct
`script_run` calls to pass a wait time of 0, which skips the
'wait for command to run' stuff entirely and just does a simple
'type the string and hit enter'.

Because of how the inheritance works, our `assert_script_run`
calls already used the os-autoinst `script_run`, rather than
the one from our distribution.

This should prevent `prepare_test_packages` sometimes going
wrong right after removing the python3-kickstart package, as
we'll properly wait for that removal to complete now (before
we weren't, we'd just start typing the next command while it
was still running, which could result in lost keypresses).

Test Plan:
Check all tests still run OK (I've tried this on
staging and it seems fine).

Reviewers: jskladan, garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D1034
2016-10-20 09:24:48 -07:00
Adam Williamson
3d84e2a8c2 type root password faster on ostree installs
Summary:
Since we started using `type_very_safely` for typing the root
password, we starting hitting a race issue. If we complete the
root password spoke so slowly that the software deployment
process completes in the meantime, anaconda will wait until we
complete the spoke then immediately flip to the 'post-install
configuration' step, at which point access to the USER CREATION
spoke is blocked.

We don't hit this case on regular RPM installs or live installs
as the deployment phase still takes a while for both of those,
but we are sometimes hitting it for the Atomic ostree install
image, as the software deployment phase is pretty fast there.
We *could* just not bother creating a user and testing we can
log in as a user for that test, but I don't like that approach,
we *should* be testing that user creation and login works OK
for ostree installs. So instead, let's just type the root
password a bit less safely for ostree installs; this will be
more vulnerable to typing errors but hopefully will avoid the
race problem.

Test Plan:
Run a few Atomic installs, see if they hit the race.
Might need to run other tests at the same time, and you may not
be able to hit it, this is obviously dependent on the I/O of
the worker host...

Reviewers: jskladan, garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D1022
2016-10-13 18:39:33 -07:00
Adam Williamson
fc1dc167f9 support_server: give the DVD copy a bit longer to complete
seems like it took more than 2 minutes in F25 testing today.
2016-10-11 18:11:34 -07:00
Adam Williamson
fdeb333ff3 fix desktop_notifications to use assert_screen not check
whoops...this test should have been failing...
2016-09-30 09:21:01 -07:00
Adam Williamson
bacb6f1f7b redo console_login with multiple matches, move to main_common
Summary:
Since we can match on multiple needles, we can drop the loop
from console_login and instead do it this way, which is simpler
and should work better on ARM (the timeouts will scale and
allow ARM to be slow here). Also move it to main_common as
there's no logical reason for it to be a class method.

Also remove the `check` arg. `check` was only set to 0 by two
tests, _console_shutdown and anacondatest's _post_fail_hook.

For _console_shutdown, I think I just wanted to give it the
best possible chance of succeeding. But we're really not going
to lose anything significant by checking, the only case where
check=>0 would've helped is if the 'good' needle had stopped
matching, and all sorts of other tests will fail in that case.

anacondatest was only using it to save a screenshot of whatever
was on the tty if it didn't reach a root console, which doesn't
seem that useful, and we'll get screenshots from check_screen
and assert_screen anyway.

Test Plan:
Run all tests, check they behave as expected and
none inappropriately fails on console login.

Reviewers: jskladan, garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D1016
2016-09-30 08:42:45 -07:00
Adam Williamson
8a6529526f fix ARM initial-setup handling for extra choice
As with the text install test, there's an additional choice on
the 'Time settings' path compared to whatever image garret
developed the test on; right after 'Time settings' you have to
pick 'Set timezone' or 'Configure NTP servers'. So adjust the
test to handle this, just like we did there.
2016-09-28 13:15:56 -07:00
Adam Williamson
e9ce14a891 consolidate login waits, use postinstall not entrypoint for base
Summary:
I started out wanting to fix an issue I noticed today where
graphical upgrade tests were failing because they didn't wait
for the graphical login screen properly; the test was sitting
at the 'full Fedora logo' state of plymouth for a long time,
so the current boot_to_login_screen's wait_still_screen was
triggered by it and the function wound up failing on the
assert_screen, because it was still some time before the real
login screen appeared.

So I tweaked the boot_to_login_screen implementation to work
slightly differently (look for a login screen match, *then* -
if we're dealing with a graphical login - wait_still_screen
to defeat the 'old GPU buffer showing login screen' problem
and assert the login screen again). But while working on it,
I figured we really should consolidate all the various places
that handle the bootloader -> login, we were doing it quite
differently in all sorts of different places. And as part of
that, I converted the base tests to use POSTINSTALL (and thus
go through the shared _wait_login tests) instead of handling
boot themselves. As part of *that*, I tweaked main.pm to not
require all POSTINSTALL tests have the _postinstall suffix on
their names, as it really doesn't make sense, and renamed the
tests.

Test Plan: Run all tests, see if they work.

Reviewers: jskladan, garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D1015
2016-09-27 11:48:15 -07:00
Adam Williamson
683b819bf9 extend the wait_still_screen time in upgrade_pre too
I have a better fix for this coming, but it's a big change that
requires review, so for now, do this. The tests are actually
failing because the 30 second wait_still_screen is too short(!)
- it's triggering while the test is sitting at the 'full Fedora
logo' bootsplash screen then doing a 30 second assert_screen
graphical_login, which is failing because it actually takes
more than 60 seconds to get from the 'full F' screen to the
login screen. So let's just make the wait_still_screen 45
seconds for now.
2016-09-24 16:39:52 -07:00
Adam Williamson
032d8f76a3 give upgrade_preinstall a bit longer for desktop startup
90 seems too tight. See:
https://openqa.fedoraproject.org/tests/36079
2016-09-24 14:28:20 -07:00
Adam Williamson
ddc91efeff detect (rather than guessing) desktop vt
use 'ps' output for Xorg and Xwayland. We'd need some new
openQA var to get this right by 'guessing', as it's vt1 for
Workstation when running live - so long as autologin worked -
but vt2 after install. We'd need a var or some other thing to
detect which case we're running in. LIVE doesn't do it, it's
set even when running a post-install test from a live image.

So instead let's just do it a bit more cleverly. This also
gives us a bit of insurance against changes in GDM, SDDM etc.
behaviour, so long as Xwayland or Xorg is running (and we can
add additional processes to the list, like gnome-shell, if
needed/appropriate). We assume the *final* listed process -
i.e. the most recently-started one - will be the desktop;
this covers gdm's behaviour of starting up on vt1 then running
the user session on vt2. We can make this function more complex
and add args if we ever get to the point where we have multi-
user tests running or anything (e.g. allow to pass a username
and only look for that user's processes).

Landing without review as this broke the live variant of the
test on Workstation in production (kinda not sure why it worked
in testing, or I didn't notice that it failed, but never mind).
I've tested it on staging.
2016-09-24 13:04:04 -07:00
Adam Williamson
e33b635f41 handle auth request for unsigned updates on KDE
When a package is unsigned, KDE will prompt for authentication.
Let's handle this. But count it as a soft fail, because
puiterwijk claims that Rawhide packages will be autosigned
from next week, so this *should* not happen and would indicate
an unsigned package in the repos. We make the KDE 'update
complete' needle area smaller because the wider area includes
some transparency and so will only match if the update applet
is open; this area will match whether it's open (no auth case)
or not open (auth case - the applet seems to disappear after
you provide the password in the auth prompt).

Pushing without review as the test is in production so I want
to make sure it works correctly.

(Also, hey, check out that array match for assert_screen and
that match_has_tag! This is gonna make some things so much
easier...thanks upstream)
2016-09-23 18:20:28 -07:00
Adam Williamson
a067d0655f add a desktop notifications test
Summary:
this more or less covers desktop_error_checks and desktop_
update_notification, though it can't really distinguish
between them easily. All we know is that if both the live and
postinstall versions of this test pass, both of those tests
pass. Any fails will have to be investigated manually.

Test Plan:
Run the tests for both KDE and Workstation, see
what happens. Workstation will fail for F25 and Rawhide at
present, due to SELinux/abrt notifications.

Reviewers: jskladan, garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D1004
2016-09-23 16:03:13 -07:00
Adam Williamson
e8b20ec73f add a desktop_update_graphical test
Summary:
Very similar to the CLI update test, but using the desktops'
update applications. This is based off the CLI update test
branch as it uses the shared functions that branch introduced.
We do not use the fake update packages, as they don't really
do anything useful for these tests; for dnf they can help us
distinguish between issues with the dnf mechanism and issues
with the repos, but we can't really tell that in the graphical
case. So we only use the python3-kickstart package here.

Test Plan:
Run the test on both KDE and GNOME and ensure it
performs as intended. I've been testing it on staging, so you
can see it there.

Reviewers: jskladan, garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D1010
2016-09-22 11:38:51 -07:00
Adam Williamson
44ec3d84c3 add a base_update_cli test
Summary:
this uses a couple of test repos with fake packages to test the
basic dnf mechanisms are working, then messes around with the
python3-kickstart package a bit to try and test the default repo
configuration is working, keys are in place and so on. We use
python3-kickstart because we should be able to rely on the copy
of that package in the 'stable' repo being installable (or else
the compose would have failed), but it shouldn't be vital to
the operation of the system.

Test Plan: Run the test, see if it works.

Reviewers: jskladan, garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D1006
2016-09-22 10:57:12 -07:00
Adam Williamson
1cdd8e18b7 make sure we get logs from failed kickstart installs
Summary:
by waiting for the bootloader in _boot_to_anaconda rather than
_console_wait_login, we can ensure that we use the anaconda
post-fail hook and thus get logs uploaded when a kickstart
install fails.

Test Plan:
Run a kickstart install test that fails and check
anaconda logs get uploaded. Then run one that works and make
sure it...still works.

Reviewers: jskladan, garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D1005
2016-09-20 10:51:51 -07:00
Jan Sedlák
c0b9bdb543 add anaconda rescue test on encrypted disk
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D995
2016-09-16 14:44:03 +02:00
Adam Williamson
ec00f3c540 fix install_text test's timezone spoke handling
I guess this was developed on F24, or an older F25? There is
an additional choice showing up in production: before the tz
choice, it asks if you want to set the tz or configure NTP. So
we have to pick tz.
2016-09-13 12:58:22 -07:00
Adam Williamson
e1ec1997af try to be safer when typing in X: slower, more checks
Summary:
the main thing this does is try and type slower in X - this
should cover nearly everywhere we type anything in X, and make
it type slower. We also add a bit more safety checking to some
old tests which didn't have it (mainly _do_install_and_reboot)
- wait_still_screen after typing to make sure all the keypresses
were registered before continuing.

This is an attempt to mitigate the problems we've seen where
the wrong text gets typed into the wrong places and the tests
break.

This branch is live on staging atm. It still has *some* issues,
but I do think it's an improvement.

Test Plan:
run the tests (probably several times), compare to
runs without the change, see if it's better or worse...

Reviewers: jskladan, garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D993
2016-09-12 10:24:30 -07:00
Adam Williamson
1c5fffa859 add a database server role test (and client test)
Summary:
Pretty straightforward tests which deploy the database
server role and exercise it a bit.

Test Plan: Run the tests, check they work properly.

Reviewers: jskladan, garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D991
2016-09-08 14:19:09 -07:00
Adam Williamson
9af88e3f23 destroy kerberos cache before firefox in freeipa password change
In F25+, Firefox seems to do kerberos auth automatically, so if
we go to the FreeIPA admin URL while kerberos-authed as test4,
we are logged in right away as test4 - neat! But not what we
wanted. So let's kdestroy.
2016-09-08 10:28:31 -07:00
Adam Williamson
03af57e0da start firefox at 1024x768 resolution
when we run firefox in a bare X session, by default we get an
800x600 firefox in a 1024x768 X server with some dead black
space to the right and bottom of the screen. Now it turns out
that if the mouse is in the dead space, Firefox will not get
any keystrokes we send.

This didn't used to be a problem, but I made it into one with
this os-autoinst change:

https://github.com/os-autoinst/os-autoinst/pull/559

that makes os-autoinst move the cursor to 1023,767 after each
`assert_and_click`, instead of 0x0 as it did before, unless the
cursor has previously been explicitly place somewhere. So in
this case it gets moved to the dead space, and Firefox stops
responding to keypresses after the first `assert_and_click`.

We could equally well fix this by setting the cursor to 0,0
after running Firefox, but I like this more as it makes sure
we won't run into the same problem some other way, and makes
the videos and screenshots look nicer.

This fixes the realmd_join_cockpit test that's been failing
ever since I installed an os-autoinst with that fix. Committing
without review as it's a straightforward fix and I want the
test working again...
2016-09-08 09:15:43 -07:00
Jan Sedlák
db95bccd52 add anaconda text UI test
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D980
2016-09-07 10:34:54 +02:00
Adam Williamson
1c882f612e lengthen that last wait_still_screen a bit...
...per https://openqa.stg.fedoraproject.org/tests/38809 , one
second isn't long enough.
2016-09-05 17:50:06 -07:00
Adam Williamson
21a5f23206 add a short wait_still_screen to _graphical_wait_login
looks like we had a good ol' 'sliding-in-from-the-side' fail
in https://openqa.fedoraproject.org/tests/32163 .
2016-09-04 09:46:58 -07:00
Adam Williamson
82b8b0a053 handle REPOSITORY_GRAPHICAL being http as well as https
I assumed the 'compose location' sent by fedmsg was https, but
in fact it's http (you get redirected to https when you access
it). Could just change the default back to 4, but why not make
it properly robust. Sending without review so this doesn't go
wrong all weekend.
2016-09-02 08:51:36 -07:00
Adam Williamson
ef689e75a9 use compose repository (not master repo) for most tests
Summary:
we have a long-standing problem with all the tests that hit
the repositories. The tests are triggered as soon as a compose
completes. At this point in time, the compose is not synced to
the mirrors, where the default 'fedora' repo definition looks;
the sync happens after the compose completes, and there is also
a metadata sync step that must happen after *that* before any
operation that uses the 'fedora' repository definition will
actually use the packages from the new compose. Thus all net
install tests and tests that installed packages have been
effectively testing the previous compose, not the current one.

We have some thoughts about how to fix this 'properly' (such
that the openQA tests wouldn't have to do anything special,
but their 'fedora' repository would somehow reflect the compose
under test), but none of them is in place right now or likely
to happen in the short term, so in the mean time this should
deal with most of the issues. With this change, everything but
the default_install tests for the netinst images should use
the compose-under-test's Everything tree instead of the 'fedora'
repository, and thus should install and test the correct
packages.

This relies on a corresponding change to openqa_fedora_tools
to set the LOCATION openQA setting (which is simply the base
location of the compose under test).

Test Plan:
Do a full test run, check (as far as you can) tests run sensibly
and use appropriate repositories.

Reviewers: jskladan, garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D989
2016-09-01 08:22:59 -07:00
Adam Williamson
9311146f95 drop dictionary error workaround
Summary:
the dictionary error bug was fixed some time back, so drop this
workaround for it.

Test Plan:
Run all tests for F25 and Rawhide and verify they don't need
this workaround any longer.

Reviewers: jskladan, garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D988
2016-08-30 11:18:46 -07:00
Adam Williamson
67cb387e9a drop RHBZ #1349721 workaround, as the bug is fixed
Summary: Pretty simple!

Test Plan:
Check the upgrade tests work the same as before the
change.

Reviewers: jskladan, garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D979
2016-08-29 11:45:34 -07:00
Adam Williamson
88e63f1593 fix upgrade test (broken by recent nogpgcheck commit)
in the recent commit to always use nogpgcheck I inadvertently
broke the upgrade tests, by dropping the `--releasever` from
the `dnf system-upgrade download` command. So fix that.
2016-08-12 10:08:46 -07:00
Adam Williamson
b7abdf81a9 always use --nogpgcheck when installing packages
Summary:
Except when running on the pre-upgrade release in the upgrade
tests (where GPG check should always be OK).

Currently we always need to use --nogpgcheck on Rawhide, and we
must also use it on Branched prior to the Bodhi activation
point. At present we don't really have any simple way to know
when the Bodhi activation point has kicked in. We could assume
that it's safe to do GPG checking for 'candidate' (not nightly)
composes, but even that isn't 100% safe and isn't really the
*right* thing to do. So I think for now it's best to just always
use --nogpgcheck , until we come up with a decent way to check
for Bodhi enablement, or releng figures things out so we can
rely on packages being signed in Rawhide and in Branched before
Bodhi enablement.

Test Plan:
Check the tests all still run, make sure I didn't
miss any dnf calls.

Reviewers: jskladan, garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D964
2016-08-10 07:56:13 -07:00
Adam Williamson
a625eac820 support_server: give the ISO copy a bit longer to complete
this is a big copy, could take longer than the default.
2016-08-09 17:11:53 -07:00
Adam Williamson
a8ddc002f8 add QA:Testcase_desktop_browser test
Summary:
pretty simple stuff here. The distinction between 'firefox' and
'browser' is that the 'browser' needles I expect would also be
correct for other default browsers, while the 'firefox' needles
are specific to Firefox. We need '-kde' variants of some Firefox
needles where interface text is included, because the font is
Cantarell in GNOME but whatever the default 'sans' font is in
KDE - I suppose we should really use -thatfontsname rather than
-kde, but I can't think what it's called...

I couldn't do the 'log in to FAS' bit of the test since we don't
really have a sane way to provide a password while not exposing
it to the public.

Test Plan:
Run the test, check it works - for both KDE and
Workstation.

Reviewers: jskladan, garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D938
2016-08-03 13:22:29 -07:00