Commit Graph

122 Commits

Author SHA1 Message Date
Adam Williamson
7e8a3010bb Fix bracket matching in previous commit
Syntax checks? In a text editor? THAT'S CRAZY TALK

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-04-09 17:19:03 -07:00
Adam Williamson
51b46bab7b g-i-s suppression change now applies to FAW 28+
Since gnome-initial-setup-3.28.0-5.fc28 , the g-i-s screens
that are supposed to be suppressed as part of
https://fedoraproject.org/wiki/Changes/ReduceInitialSetupRedundancy
are now suppressed on FAW installs as well as traditional ones.
So adjust the logic accordingly.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-04-09 16:50:16 -07:00
Adam Williamson
23192baa5d Use better pattern for checking if there are coredumps
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-04-04 14:13:39 -07:00
Adam Williamson
cdf33cc2ae Catch and upload coredumps in post-fail hook
We were doing this in a post-install test, but not on failures.
We need it to figure out why Firefox is crashing on aarch64...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-04-04 12:10:33 -07:00
Adam Williamson
89bdea410f Rejig how we count the g-i-s screens we'll encounter
Trying to keep track of what these magic numbers mean is really
getting messy, so let's do it a bit more explicitly, using the
page names g-i-s uses internally, and lots of comments. This
should make it clearer and more maintainable when stuff changes.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-03-29 19:06:51 -07:00
Adam Williamson
baa7ac4e39 Handle KDE update test when KDE has aleady found updates
We do the 'desktop update' test for KDE via the notification
icon thingy, and it behaves differently depending on whether it
has already detected there are updates or not. The test only
works at present in the case where it *hasn't* - it expects the
notification icon to be in the extended panel and it expects to
see a 'refresh' button, neither of which is the case if it's
already noticed there are updates to install.

We should also force PackageKit to update its list of available
updates after we set up our 'special' update, otherwise on this
path KDE will only install the updates it found *before* we did
our stuff, and the test will fail as our special update won't be
there.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-03-28 19:53:19 -07:00
Adam Williamson
75d1b80db7 Bump a magic timeout a bit for aarch64
aarch64 managed to hit the problem this 'magic timeout' tries
to avoid, so let's extend it :(

e.g. https://openqa.stg.fedoraproject.org/tests/267174

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-03-27 11:45:22 -07:00
Adam Williamson
81540165e8 Adjust repo_setup for add-on Modularity
I believe this should do all the right repo modifications for
add-on Modularity (i.e. F28+ Server installs, for now).

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-03-26 13:27:31 -07:00
Adam Williamson
2c26ce1ede Simply repo_setup by removing MODULAR bit
We don't need all the MODULAR stuff any more as we no longer do
oldModularity composes like that.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-03-26 12:56:10 -07:00
Adam Williamson
669df14566 FreeIPA: handle unexpected auto-login to webUI
There are cases where we get logged back into the FreeIPA web UI
automatically by a stale kerberos ticket or something. If we're
logged in as the *right* user, let's just treat this as a soft
failure and continue with the test.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-03-21 15:43:40 -07:00
Adam Williamson
e685f00bd0 More tweaking for UEFI boot params on aarch64
Seems we're dealing with a bug, here.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-03-20 15:07:02 -07:00
Adam Williamson
20c361999b Fix post-install bootloader args on aarch64
Seems aarch64 needs 12 'down' key presses like ppc64, not 13
like x86_64. Tweak how this is done a bit; the ternary wasn't
elegant any more with the aarch64 change, so just get rid of it.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-03-20 14:05:00 -07:00
Adam Williamson
d162e4df16 Disable 'updates' repo for Branched update tests
This works around RHBZ #1552814, and it's not incorrect really
because the repo is always empty for Branched. I didn't do it
before because we might theoretically start using the repo for
Branched at some point in the future, and if we did that we'd
probably want it enabled for this test. But to get F28 update
tests working, let's just turn it off for now.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-03-10 11:01:35 -08:00
Adam Williamson
f1913e8305 Change g-i-s 'nexts' count for F28+
Several screens are now skipped in g-i-s pre-login mode, we
have to adjust the number we expect.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-03-09 18:37:01 -08:00
Adam Williamson
0915b857f9 More tweaking for this damn no root password spoke situation
Previous approach wouldn't work for tests that run after the
install test...let's just set a password from a chroot after
install completes. Don't really like this as it changes the
'real' install process a bit, but it's the least invasive short
term fix at least. We can maybe do something more sudo-y later
with a bit more thought.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-03-08 20:31:14 -08:00
Adam Williamson
83c32fe04e More tweaking for Workstation live scenario
It's really INSTALL_NO_USER, not USER_LOGIN='false'. Also, we
need to make root_console work with no root password, sigh.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-03-08 19:22:21 -08:00
Michel Normand
18602d72f3 Change baseurl for PowerPC repo as "fedora-secondary"
in _repo_setup_updates function

Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2017-11-28 08:18:16 +01:00
Adam Williamson
2a7b2923f7 Handle BGO#790811 (GNOME 'getting started' crash)
There's a bug causing the 'getting started' screen to crash.
This doesn't really make the system unusable, so treating it
as a soft failure seems appropriate, especially as this will
unblock all the post-install tests on Workstation.
2017-11-24 16:36:31 -08:00
Adam Williamson
da7eed3be6 Handle unbranded Cockpit login screen, record soft failure
There's a bug in current Modular Server which results in Cockpit
having an unbranded login screen, let's handle this but record
it as a soft failure.
2017-11-07 16:01:07 -08:00
Adam Williamson
442c143144 Try to fix quoting in printf 2017-11-07 15:00:08 -08:00
Adam Williamson
ce07c418d0 Fix (hackily) install of X, Firefox and haveged on Modular
Modular composes don't include these packages, but we need them
to run the web UI tests for FreeIPA and Cockpit. This is the
most reasonable hack I can come up with for now: just use a
non-modular fedora repo to source these packages when doing
Modular compose testing.

If we ever reach an all-Modular future, these packages should
be available in Modular composes I guess, but for now they are
not.
2017-11-07 14:20:36 -08:00
Adam Williamson
72395f1f67 Remove stray apostrophe in _repo_setup_compose 2017-11-06 17:42:30 -08:00
Adam Williamson
5e628551d8 Use Server not Everything variant for http repos on modular
Modular Server composes don't contain an Everything variant, so
we can't use it for the installation repo.
2017-11-06 16:25:40 -08:00
Adam Williamson
833502c880 Don't comment out metalink lines in repo files for modular
We can't easily do the 'point to the compose' fixup.
2017-11-06 16:20:40 -08:00
Adam Williamson
aaba51768e Don't use dnf config-manager on modular images
It's not there and can't be installed, at least at present.
2017-11-06 16:12:55 -08:00
Michel Normand
4610408963 Do not try to login if already done
the comment in console_login was already valid
but there was a missing return after test.

exemple of incorrect output is:
https://openqa.stg.fedoraproject.org/tests/182970#step/_collect_data/4

Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2017-10-18 18:53:11 +02:00
Michel Normand
e6fe996613 Revert "Modify "install_multi" test for PowerPC"
This reverts commit 8b2977f1d618316ded61420df4fc7d2afd07cbf4.

The initial commit was required for PowerPC
until qemu 2.7.1-6 (in f25) not required anymore
since qemu 2.9.0-5 (in f26)
2017-09-06 08:43:04 +02:00
Michel Normand
b1662c5f9d Add a 1s delay between each down key in do_bootloader
and call save_screenshot to visually check
for debug purpose only
Also change for PowerPC the number of down key to 12
(rather than 12)
Seems to be mandatory since 20170327.

Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2017-09-06 08:43:04 +02:00
Guy Menanteau
9c8a4949fa Modify "install_multi" test for PowerPC
PowerPC arches have the empty disk automatically
mounted on the second position in anaconda (vdb).
Thus, trig installation on second disk.
Change disk checking to point on correct disk.

Warning: this is a workaround specific correction
addressing a specific case.
This will have to be improved/changed with a more
generic code as suggested by Adam Williamson in
https://pagure.io/fedora-qa/os-autoinst-distri-fedora/pull-request/1#comment-31858
proposal for a next commit :)

Signed-off-by: Guy Menanteau <menantea@linux.vnet.ibm.com>
Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2017-09-06 08:43:04 +02:00
Michel Normand
6659e264b7 update repo_setup for PowerPC f25
to avoid upgrade_server test to fail with:
"Repository fedora-source has no mirror or baseurl set."

Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
2017-09-06 08:43:04 +02:00
Guy Menanteau
06839422e5 First PowerPC specific changes (not templates)
* New OFW variable to identify Open Firmware (used by PowerPC)
* Few needles changes for PowerPC support
* as requested do not change the timers value below for PowerPC
  tests/install_source_graphical.pm (300 to 600)
  tests/_boot_to_anaconda.pm (300 to 1200)
  This will be handled by TIMEOUT_SCALE in templates

Signed-off-by: Guy Menanteau <menantea@linux.vnet.ibm.com>
2017-09-06 08:43:04 +02:00
Adam Williamson
c2bb74c64d Explicitly install 'koji' package in update repo setup
bodhi-client doesn't depend on the 'koji' package but does need
it to do 'bodhi updates download', which we want to do. So we
must explicitly install it here.
2017-09-04 11:05:00 -07:00
Adam Williamson
b6c23ff4cf Hide cursor at start of prelogin gnome-initial-setup
For the same reason we do it at the start of GDM (recent mutter
dumps a mouse cursor in the middle of the screen on startup).
2017-09-02 16:16:54 -07:00
Adam Williamson
9b91c839a7 Treat failure to source bash profile as a soft fail
There's a bug in current Rawhide causing sourcing of /etc/bashrc
to fail when logging in as a regular user. This results in the
bash prompt looking different, which is currently a hard fail,
and causes most tests to die. It's better to treat this as a
soft fail so the rest of the test can run. So add a needle to
spot this case, and a little finish function the console login
function calls whenever it's successfully logged in, to check
whether it got the no-profile prompt and register a soft fail.
2017-09-02 12:49:56 -07:00
Adam Williamson
ad6186daf1 Use super not alt-f1 for the 'clean desktop' workaround
Using alt-f1 might cause problems if we do it while we're at a
TTY (it'll switch to TTY1, which may not be what we want).
2017-07-10 15:05:36 -07:00
Adam Williamson
aca7de2861 Change up 'clean desktop' check again (use a util function)
Well, that OCR needle isn't working out so great, as it seems
to match when it shouldn't:

https://openqa.fedoraproject.org/tests/119217#step/_graphical_wait_login/5

So let's try another approach. Ditch the OCR needle and have a
function for checking we're at a clean desktop. It does the
normal needle match, but if we're on GNOME, it also tries
hitting alt+f1 and seeing if we're at the overview; if so, it
hits alt+f1 again (to go back to the desktop) and returns.
2017-07-10 11:47:07 -07:00
Adam Williamson
c3fd611f2d Finally revert anaconda root_console workaround
RHBZ #1222413 was fixed long ago. This workaround is, I think,
the cause of openQA failures to run commands properly with an
extraneous '2' at the start of the command (e.g. 116864).
2017-07-04 09:48:40 -07:00
Jan Sedlák
5e2a76efde fix blivet so that correct unit for size is selected
Differential Revision: https://phab.qa.fedoraproject.org/D1193
2017-05-05 09:29:55 +02:00
Jan Sedlák
140c5f0a42 Add custom partitioning tests for blivet
Differential Revision: https://phab.qa.fedoraproject.org/D1188
2017-04-24 14:23:35 +02:00
Adam Williamson
654534e694 Work around RHBZ #1439429 when running Firefox on X 2017-04-05 18:22:09 -07:00
Adam Williamson
1444c5030b Add tests with no user created during install
Summary:
This adds a new test suite, run for Workstation and KDE live
images, which does not create a user during install. It then
expects initial-setup (KDE) or gnome-initial-setup (Workstation)
to appear after install, creates a user, and proceeds with
normal boot.

Note the ARM image test already covers the initial-setup text
mode, and the ARM minimal image is the only case where that
actually matters (it's not included in Server).

Test Plan:
Run the new tests, check they work. Run all old
tests, check the changes didn't break them.

Reviewers: jsedlak, jskladan

Reviewed By: jsedlak

Subscribers: tflink

Differential Revision: https://phab.qa.fedoraproject.org/D1185
2017-04-05 09:43:26 -07:00
Adam Williamson
7977b41804 Give update download yet more time (we have some big updates) 2017-03-24 14:10:29 -07:00
Adam Williamson
ba3a5152c1 Improve FreeIPA debug logging a bit
Committing without review as this is pretty trivial and I've
had it on staging for the last few days without issue. Just gets
us somewhat better info for debugging FreeIPA issues.
2017-03-16 12:36:33 -07:00
Adam Williamson
e4a8929465 Delete openh264 repo definition during repo_setup
This repo is causing problems for Branched update tests. The
repo is not available for 26 at all yet. This shouldn't be a
problem as the repo is disabled by default, but it seems that
some things - at least realmd, as used in the FreeIPA enrolment
tests - still try to update the repo's metadata when installing
packages, and fail because it 404s.

Since none of our tests actually needs this repo AFAIK, let's
just delete it in repo_setup.
2017-03-15 10:02:25 -07:00
Adam Williamson
bb1be2f0b1 Disable updates-testing for updates tests on Branched 2017-03-13 14:49:27 -07:00
Adam Williamson
da01d15406 Fix repo setup for Branched update tests (I hope)
Branched update tests are all failing because the baseurl in
fedora.repo is incorrect for Branched. This is a rather hacky
fix for this problem. It relies on the scheduler setting the
DEVELOPMENT variable when the update is for Branched (I named
the variable DEVELOPMENT rather than BRANCHED to be more
future-proof).

Alternative options I rejected were:

i) stick with MM links
ii) do something 'clever' to retrieve the URLs from MM

Rejected i) because the timing problem where the infra repo gets
updated before MM has the updated repodata checksums is just too
much of a problem; whenever that happens, dnf will refuse to use
the metadata from the infra repo and go pull it from an external
mirror, which can wind up timing out.

Rejected ii) because it seemed too fancy and not really any more
robust than just doing this and adapting it if Things Change In
Future (TM).
2017-03-13 12:43:54 -07:00
Adam Williamson
8eea2a5d1f Bump the timeout on the initial 'dnf -y update' for update tests 2017-03-03 15:55:31 -08:00
Adam Williamson
461f3a6132 Update testing: log packages in update and installed packages
Summary:
This adds some logging related to the update testing workflow,
so we have some idea what we actually tested. We log precisely
which packages were actually downloaded from the update - this
is important as updates can be edited and when examining results
we'll want to know which packages actually got used. We also
add a new module which runs at the end of postinstall and tries
to figure out which packages from the update were installed in
the course of the test. This still isn't a guarantee the test
actually *tested them* in any way, but it at least means they
got installed successfully and didn't interfere with the test.

Test Plan:
Run the update test workflow, check the logs get
uploaded and seem accurate (sometimes some RPM garbage messages
wind up in the package log, I'm not too worried about that at
present). Run the compose test workflow and check it didn't
break.

Reviewers: jsedlak

Reviewed By: jsedlak

Subscribers: tflink

Differential Revision: https://phab.qa.fedoraproject.org/D1149
2017-02-23 14:51:19 -08:00
Adam Williamson
92d588f245 Add support for testing updates
Summary:
This adds an entirely new workflow for testing distribution
updates. The `ADVISORY` variable is introduced: when set,
`main.pm` will load an early post-install test that sets up
a repository containing the packages from the specified update,
runs `dnf -y update`, and reboots. A new templates file is
added, `templates-updates`, which adds two new flavors called
`updates-server` and `updates-workstation`, each containing
job templates for appropriate post-install tests. Scheduler is
expected to post `ADVISORY=(update ID) HDD_1=(base image)
FLAVOR=updates-(server|workstation)`, where (base image) is one
of the stable release base disk images produced by `createhdds`
and usually used for upgrade testing. This will result in the
appropriate job templates being loaded.

We rejig postinstall test loading and static network config a
bit so that this works for both the 'compose' and 'updates' test
flows: we have to ensure we bring up networking for the tap
tests before we try and install the updates, but still allow
later adjustment of the configuration. We take advantage of the
openQA feature that was added a few months back to run the same
module multiple times, so the `_advisory_update` module can
reboot after installing the updates and the modules that take
care of bootloader, encryption and login get run again. This
looks slightly wacky in the web UI, though - it doesn't show the
later runs of each module.

We also use the recently added feature to specify `+HDD_1` in
the test suites which use a disk image uploaded by an earlier
post-install test, so the test suite value will take priority
over the value POSTed by the scheduler for those tests, and we
will use the uploaded disk image (and not the clean base image
POSTed by the scheduler) for those tests.

My intent here is to enhance the scheduler, adding a consumer
which listens out for critpath updates, and runs this test flow
for each one, then reports the results to ResultsDB where Bodhi
could query and display them. We could also add a list of other
packages to have one or both sets of update tests run on it, I
guess.

Test Plan:
Try a post something like:
HDD_1=disk_f25_server_3_x86_64.img DISTRI=fedora VERSION=25
FLAVOR=updates-server ARCH=x86_64 BUILD=FEDORA-2017-376ae2b92c
ADVISORY=FEDORA-2017-376ae2b92c CURRREL=25 PREVREL=24

Pick an appropriate `ADVISORY` (ideally, one containing some
packages which might actually be involved in the tests), and
matching `FLAVOR` and `HDD_1`. The appropriate tests should run,
a repo with the update packages should be created and enabled
(and dnf update run), and the tests should work properly. Also
test a regular compose run to make sure I didn't break anything.

Reviewers: jskladan, jsedlak

Reviewed By: jsedlak

Subscribers: tflink

Differential Revision: https://phab.qa.fedoraproject.org/D1143
2017-02-22 11:33:32 -08:00
Adam Williamson
186678e98b Make log upload work when installed system hits emergency mode
Summary:
This is to handle cases like #1414904 , where the system boots
to emergency mode. We really need logs to try and debug this.

Test Plan:
Force a test to hit emergency mode somehow (right now
you can just run base_services_start on Rawhide over and over
until you hit #1414904, but there's probably an easier way to
do it, I think there's a systemd boot arg to tell it which target
to boot for e.g.) and check logs get uploaded. Also check this
doesn't break log upload for a 'normal' failure.

Reviewers: garretraziel_but_actually_jsedlak_who_uses_stupid_nicknames

Reviewed By: garretraziel_but_actually_jsedlak_who_uses_stupid_nicknames

Subscribers: tflink

Differential Revision: https://phab.qa.fedoraproject.org/D1103
2017-02-01 12:30:21 +01:00