Commit graph

718 commits

Author SHA1 Message Date
Jan Sedlák
df7a4582d0 add another cantarell18 needle
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D646
2015-11-13 10:05:02 +01:00
Adam Williamson
8d0fc963df longer install timeout for Rawhide
Summary:
When Rawhide's using a debug kernel, the install is pretty slow,
and often times out. This gives us a 33% longer timeout for the
install process when running on Rawhide. This will slow Rawhide
runs down a bit when there are genuine failures at this point,
but it seems kind of unavoidable.

Test Plan: Do a Rawhide run and see if we get fewer false fails.

Reviewers: jskladan, garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D643
2015-11-11 08:48:46 -08:00
Adam Williamson
29b9ffc405 add Cantarell 0.0.18 variant needles
Cantarell 0.0.18 showed up in Rawhide recently and changes
rendering again, so we need some more variant needles. Once
Cantarell change quiets down a bit we'll have to clean these
all up.
2015-11-10 12:35:30 -08:00
Jan Sedlák
8e2bcbdca1 add CDMODEL=scsi-cd when iSCSI is used
There is a bug (it is fixed in newest unstable version though) in openQA
4.2-1.2 that when you use iSCI HDD device, it fails to boot. Workaround is to
set CDMODEL=scsi-cd (and it won't hurt even when this gets fixed in stable).

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D642
2015-11-09 09:50:24 +01:00
Jan Sedlák
da7bdda462 add azerty user logged in needle
Since we are now logging to user account every time,
I've encountered new bug - we are missing needle for azerty
user logged in. This DR adds it.

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D640
2015-11-09 09:49:21 +01:00
Jan Sedlák
ec6b3ff4a3 use validate_script_output instead of typing and needles matching
Use validate_script_output and regex matching instead
of type_string and needles.

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D637
2015-11-04 14:38:36 +01:00
Josef Skladanka
2e1dc5877b Login as user in console every time
Instead of removing the ` QA:Testcase_Anaconda_user_creation` from all
the testsuites, make OpenQA login in console (as user) each time.

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D636
2015-11-04 10:10:04 +01:00
Jan Sedlák
7a4eff8acb add new krunner needles
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D626
2015-10-27 13:52:00 +01:00
Adam Williamson
52ceed6f39 upgrade: disable screen blanking before long-running commands
Summary:
Updating the stable release prior to doing the update can take
a long time if the image hasn't been updated for a while, and
the upgrade download process itself can take a long time. If
the screen blanks out in either case, either the following
needle match may fail (if we're waiting for a needle) or 'still
screen' may be detected early (if we're waiting for a still
screen), so let's disable screen blanking to avoid it.

Test Plan: Run the upgrade tests and see if they work.

Reviewers: garretraziel, jskladan

Reviewed By: jskladan

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D628
2015-10-26 18:02:22 -07:00
Adam Williamson
1f9685d058 another Cantarell 0.0.17.2 variant
Summary: didn't catch this before (as it depends on the image tested)

Test Plan: Test a Server ISO, the ext3 test fails without this

Reviewers: jskladan, garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D622
2015-10-23 11:27:49 -07:00
Petr Schindler
079efe72e8 Adds variable PART_TABLE_TYPE to machines
PART_TABLE_TYPE variable says which type of partition table type
should be on attached HDDs.
Some tests with uefi have to use disks with gpt.
Tests are amended to use right disks.

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D623
2015-10-21 09:36:40 +02:00
Adam Williamson
2f1216c3b0 use lowercase $VERSION for upgrade version check
Summary:
For Rawhide, the text in os-release-fedora is 'rawhide' not
'Rawhide', so lc the $VERSION value when checking. For other
releases it's digits, so the lc won't change anything - lc('23')
is '23'.

Test Plan: Run the Rawhide upgrade tests, they should succeed.

Reviewers: jskladan, garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D620
2015-10-16 23:38:09 -07:00
Adam Williamson
676175023d add another cantarell variant needle
Summary:
So another couple of Cantarell updates hit Rawhide and changed
rendering a bit more, leading to this needle breaking. This is
for Cantarell 0.0.17.2. No other needles seem to have problems.

Test Plan:
Already in place on BOS - note the 2015-10-15 Rawhide
tests that initially failed but now pass.

Reviewers: jskladan, garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D621
2015-10-16 23:37:48 -07:00
Josef Skladanka
75b50ae17f Moved needles to the right dir 2015-10-15 09:58:04 +02:00
Josef Skladanka
20ada247ed Moved needles to the right dir 2015-10-15 09:57:33 +02:00
Josef Skladanka
f29aa030fd Cleaning up adamw's mess
Moved all the mess from repo root to a directory. Since there is no need
for all the needles to be in the repo root - no magic happening here, at
the deployment, the needles are copied in the needles directory anyway.

Landing without review, as this seems to be the trend now ;)
2015-10-14 09:30:57 +02:00
Adam Williamson
8d3c5a862e add versions of multiple needles for updated Cantarell
Cantarell 0.0.17, which landed in 23 Branched on 2015-10-09,
changes several elements of the font's appearance and leads to
many failed needle matches. This commit contains variants of
all appropriate needles.

We cannot simply replace all the needles as we are doing the
22 stable nightly tests as well. Once all tests we run are on
releases which have 0.0.17, we could make the cantarell17 files
into the 'standard' ones and remove the old ones.

Landing without review as this breaks all tests and the change
is straightforward, simply a case of updating the needles with
the interactive editor.
2015-10-09 22:20:09 -07:00
Adam Williamson
800870dd0d schedule i686 and x86_64 default_install tests for server DVD
Summary: simple enough. scheduler should already have the necessary bits.

Test Plan:
Kick off a run and see if we get tests, and the results are
reported to the wiki (Final TC1 should work).

Reviewers: jskladan, garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D607
2015-10-06 15:27:27 -07:00
Jan Sedlák
26fd74490c yet another KDE fixes - login manager changed
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D598
2015-09-30 13:31:03 +02:00
Adam Williamson
b97c019ae9 revise language tag handling to be easier to use (T617)
Summary:
T617 makes some good points about the language tags; this is my
suggestion for an improvement. It requires a bit of cleverness
in unregister_prefix_tags(), but the upshot is that you don't
need to know to set any special tags when creating needles, a
needle with no language-related tags will be considered as valid
for all languages. You have to explicitly add LANGUAGE- tag(s)
to a needle for the language filtering to 'kick in' in any way.
If a needle has at least one LANGUAGE- tag, it will be filtered
unless it has the appropriate tag for the job's specified
language (default is still 'english').

With this approach, only needles which we specifically want to
*only* match their tagged language(s) need the tags, so we can
drop all those -ALL tags.

We're using LANGUAGE- instead of ENV-LANGUAGE- now because the
ENV- tag names denote tags that are treated slightly specially
by openQA, and this is not one. We cannot cleanly use
ENV-INSTLANG because openQA has a hardwired default of 'en_US'
for that.

Test Plan:
Check both English and French tests still work as
intended.

Reviewers: jskladan, garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D589
2015-09-29 15:52:50 -07:00
Adam Williamson
d4ddfe167a add a default_install test for cloud atomic installer image
Summary:
We have these 'atomic installer' images (so far just Cloud),
and maxamillion wanted to get them tested. Turns out it's
pretty trivial - they look much like other installs. Only
little wrinkle is they have a reduced hub (no repository
needles) like live images, but are not like live images in
any other way, so I rejigged the 'small hub needle filtering'
handling a bit.

There will be an accompanying diff for tools, and also some
changes in fedfind (these images are getting built nightly
for *current stable*, and it'd be good to test those).

Because we'd like to test the 22 nightlies, I had to add some
needles for 'olddpi' versions of a few screens. See 2e4c1c2 -
the 22 Atomic installer images still have the old GTK+ code
meaning they run at 96.09dpi. I only retook the necessary
needles for the default-install test, if we add any others we
made need to retake a few more needles.

Test Plan:
Schedule jobs for a compose with the atomic installer
image. You will need the matching openqa_fedora_tools diff and
the very latest git fedfind. Check the test for that image runs,
all other tests run as usual, excessive images are not
downloaded, and the atomic installer is not used for running
universal tests.

Reviewers: jskladan, garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D595
2015-09-29 11:36:11 -07:00
Jan Sedlák
533dd806e2 fix KDE x11_start_program needle, add Gnome needle
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D592
2015-09-17 09:49:55 +02:00
Adam Williamson
f0f4482976 needle clean up: no ENV-OFW, remove unused needles
Summary:
This removes a couple of unused needles. password_required
needles were dupes of console_password_required, and no tests
actually use password_required. tmp_failed_proc just isn't
used by anything at all (I think it was at some point used in
fedup tests).

This also drops the ENV-OFW-1 tag from all needles, which I
think got in because someone based a needle on a SUSE needle
which has that tag, then it spread. It's of no use or meaning
in Fedora.

Test Plan: Check all tests still run as normal.

Reviewers: garretraziel, jskladan

Reviewed By: jskladan

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D579
2015-09-15 16:05:00 -07:00
Adam Williamson
fd177dd39e strip 32-bit test set a bit
Summary:
So test runs are getting very long on BOS, and we have UEFI
tests coming. Try to help out by reducing the 32-bit test load
a bit. I tried to strategically drop the tests that are least
likely to differ, e.g. different storage layouts (but not
filesystem/device types), kickstart delivery, and only doing
one upgrade test.

Test Plan:
Check the templates file loads and there are no
obvious errors. See if you agree with the tests I cut.

Reviewers: garretraziel, jskladan

Reviewed By: jskladan

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D580
2015-09-15 16:04:30 -07:00
Jan Sedlák
7a608985dc fix KDE needle language tag
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D582
2015-09-15 13:51:47 +02:00
Petr Schindler
338b4bf513 Adds uefi support to tests where it makes sense
What changed:
* There is a new needle for uefi bootloader.
* UEFI postinstallation phase
* UEFI tests and machine added to template
2015-09-15 11:04:01 +02:00
Adam Williamson
5d650b070d correct 32-bit kde live test entry in templates 2015-09-14 18:22:05 -07:00
Adam Williamson
b3aa968575 add a french (encrypted) test
Summary:
this handles Non-English European Language Install. Basically
it's a bunch of new screenshots for existing tag names, plus
a bit of configurability in _boot_to_anaconda and tweaking some
existing needles to do non-text matches. The weird 'half-the-
icon' needles are for cases where there may or may not be a
warning triangle but we want to click it either way (saves
duplicating the needle).

This also sets up a convention for tagging what languages a
needle is appropriate for. If it's specifically appropriate for
one or more languages, a tag ENV-LANGUAGE-(LANGUAGE) should be
applied for each language, where (LANGUAGE) is the install
language in upper-case ('LANGUAGE' variable, which should also
be the string that will be typed into the language selection
screen). If the needle ought to be used for *all* languages -
i.e. it's not a text match, or any text in the match is known
not to be translated - the tag ENV-INSTLANG-ALL should be
applied.

To back this, main.pm now unregisters all needles that are not
tagged with either ENV-LANGUAGE-ALL or the tag for the language
actually being used (if the LANGUAGE var is not set, we assume
english). The point of this is to check the install is actually
translated; if we allow all needles to match, the test would
pass even if no translations appeared at all.

Test Plan:
Run all tests and make sure you get the expected
results. You can schedule a run against 23 Beta TC1 to see the
French test fails 'correctly' when translations are missing.

Reviewers: jskladan, garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D577
2015-09-14 18:08:58 -07:00
Jan Sedlák
7017486d43 add KDE live default install test
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D576
2015-09-14 08:52:37 +02:00
Jan Sedlák
81bedd8de7 update README - use wiki instead of PhaseSeparation.md
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D571
2015-09-11 12:36:20 +02:00
Adam Williamson
68acecb6d4 convert upgrade tests to dnf-plugin-system-upgrade
Summary:
This is a first cut which more or less works for now. Issues:

1) We're not really testing the BUILD, here. All the test does
is try and upgrade to the specified VERSION - so it'll be using
the latest 'stable' for the given VERSION at the time the test
runs. This isn't really that terrible, but especially for TC/RC
validation, we might want to make things a bit more elaborate
and set up the repo for the actual BUILD (and disable the main
repos).

2) We'd actually need --nogpgcheck for non-Rawhide, at one
specific point in the release cycle - after Branching but
before Bodhi activation (which is when we can be sure all
packages are signed). This won't matter until 24 branches, and
maybe releng will have it fixed by then...if not, I'll tweak
it.

3) We don't really test that the upgrade actually *happened*
for desktop, at the moment - the only thing in the old test
that really checked that was where we checked for the fedup
boot menu entry, but that has no analog in dnf. What we should
probably do is check that GUI login works, then switch to a
console and check /etc/fedora-release just as the minimal test
does.

Test Plan:
Run the tests. Note that creating the desktop disk
image doesn't work ATM, so I can't verify the desktop test
works, but the minimal one seems to (with D565). There'll be
a matching diff for openqa_fedora_tools to update the test
case names there.

Reviewers: jskladan, garretraziel

Reviewed By: jskladan, garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D567
2015-09-10 14:49:13 -07:00
Adam Williamson
c9ec17b2b7 make Done button match area smaller to handle dotted line
Summary:
To understand this change see test 1092 on BOS. When you click
Done once and then get the 'click Done again to proceed' warning,
the Done button seems to grows a little dotted line (I think
this indicates it's selected), which causes the match to drop to
94% and fail. To counter that, shrink the match area to the bit
of the button inside the dotted line, which will be the same
whether it's there or not.

Test Plan:
See if no_swap stops failing sometimes now, maybe,
at last? Yeesh. Also see if all other tests still run OK.

Reviewers: jskladan, garretraziel

Reviewed By: garretraziel

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D549
2015-09-08 16:31:43 -07:00
Jan Sedlák
f8f242b7e0 add guided shrink test
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D561
2015-09-08 15:54:22 +02:00
Jan Sedlák
0942cd72a6 document utility methods
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D554
2015-09-04 11:00:31 +02:00
Adam Williamson
89b717919e small fix: wait a sec between clicks of 'Done' in no_swap
I've seen no_swap fail several times with the 'Click for details
or press Done again to continue.' message showing (e.g. job
967 on BOS). I think this is just because sometimes we try and
click Done too fast, so introduce a 1-second sleep between Done
clicks to try and solve that.
2015-08-31 17:02:27 -07:00
Jan Sedlák
45ec69e504 separate 32bit and 64bit upgrade tests
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D535
2015-08-27 13:49:09 +02:00
Adam Williamson
95bc2657fb add a 'no swap partition' test
Summary:
since we did this live at Flock today, I figured I'd tidy it
up and submit it. This is an 'optional' test, but some people
do run this way so it'd be nice to have it. This adds another
little helper method in anacondatest.pm, for deleting partitions,
which works much like the others added in previous commits.

Test Plan: Schedule a test run, see if the test runs and works.

Reviewers: jskladan, garretraziel

Reviewed By: jskladan, garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D503
2015-08-19 17:41:41 -04:00
Jan Sedlák
9092b31863 attach dnf.librepo.log in Anaconda post_fail_hook
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D513
2015-08-19 12:33:58 +02:00
Jan Sedlák
3f8f37a348 use disk_f22 instead disk_f21 in templates
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D502
2015-08-17 17:40:06 +02:00
Jan Sedlák
f71faaa43d add updates.img via local media test
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D496
2015-08-17 17:36:40 +02:00
Adam Williamson
5d0afd612b add 32-bit tests to templates
Summary:
This duplicates basically the entire test suite for 32-bit. We
could choose to run only a few tests for 32-bit if we wanted,
but I figure we may as well do as much testing as we can. Only
a few of the results will actually wind up 'counting' separately
in the wiki, but we can always see the results for all the tests
in openQA itself.

Next we can duplicate the whole set again for UEFI!

Test Plan:
Schedule a full test run and make sure all the tests
run for both arches. Also check result submission works
correctly. Requires the corresponding change to tools (or
you can just use one of the trigger commands which lets you
specify arch with a parameter).

Reviewers: jskladan, garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D500
2015-08-15 21:10:14 -04:00
Jan Sedlák
5438b452d2 mention main.pm architecture in README
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D495
2015-08-11 13:30:42 +02:00
Adam Williamson
8fde224dc5 add ext3, btrfs and LVM thin tests, tweaks to custom methods
Summary:
This adds three new custom storage tests and some needles to
support them, and tweaks the custom storage methods a bit to
address some things that cropped up in writing the tests. A
new method is added for changing the filesystem, as that's
a distinct operation from changing the device type.

This also restores the previous behaviour of select_disks()
where it handled selecting custom partitioning when needed.
Turns out it's pretty common to use regex'es in perl! Who'd'a
thought.

A corresponding commit to add the tests to openqa_fedora_tools
is coming.

There's no post-install step for the tests yet; I'll try and
write those up and add them soon.

Test Plan:
Do a full run, including the new tests, on Alpha RC2 and check
all are scheduled correctly and run correctly. The LVM thinp
test is expected to fail as it catches a genuine bug.

Reviewers: jskladan, garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D490
2015-08-10 11:01:12 -07:00
Jan Sedlák
cd57a721bc add VARIABLES file
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D488
2015-08-10 09:25:04 +02:00
Jan Sedlák
0b1d4d1f92 fix small bug with selecting custom partitioning 2015-08-06 16:15:48 +02:00
Jan Sedlák
4778d23f8c small fix 2015-08-06 13:53:43 +02:00
Jan Sedlák
335ce280f3 merge DISK_GUIDED and DISK_CUSTOM into PARTITIONING
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D486
2015-08-06 11:02:18 +02:00
Jan Sedlák
a18b6f8f69 fixes for current code
Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D485
2015-08-06 11:01:41 +02:00
Adam Williamson
3df993404c needle tweaks for eurlatgr font in anaconda
Summary:
I discovered another fun font issue today. Current anaconda
images don't use the intended 'default' console font, eurlatgr.
Neither do live images, but installed systems *do*.

The font they use is the system BIOS font, which in openQA
cases means the qemu firmware font. The easiest way to spot the
difference is the @ character; the shorter version is from the
system BIOS, the slightly taller one is what it looks like in
eurlatgr and latarcyrheb-sun16 (the old default).

In a test image I built, for some reason, I *did* get eurlatgr
in the tmux console, and that broke some needle matches. After
figuring all this out, bcl has sent a lorax patch to use
eurlatgr in the installer, so it makes sense to add these fixes
to the repo for when that kicks in.

We shrink the match on root_logged_in.json by one line. This
screenshot is taken from a post-install case where the prompt
appears in the middle of the screen, and has three black rows
above the prompt; in anaconda, when the prompt appears right at
the top of the screen, there's only *two* rows of black above
it, so the match fails. This fixes that. It's been working so
far because installs have been matching root_logged_in_
rawhide20150311, which is taken with the firmware font, but
once the installer starts using eurlatgr, that won't match any
more.

We also add a new needle for the anaconda_install_source_check
_repo_added tag, taken with eurlatgr. The existing screenshot
was taken either with the firmware font or with latarcyrheb.
They both use a curly glyph for a single quote ('), while
eurlatgr uses a straight line.

This also renames the root_logged_in variant needle to be
clearer about why it's there. We'll probably need variants of
some needles until we're sure lives, anaconda env, and installed
systems are all using eurlatgr. RHBZ #1250262 is a bug I filed
for the live images not using eurlatgr.

Test Plan:
Run the tests with both BIOS font and eurlatgr as
the anaconda font and make sure they all work. The latter
might be a bit tricky till the change lands upstream, I've no
idea how it worked out that way in my test boot.iso.

Reviewers: jskladan, garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D483
2015-08-05 09:15:41 -07:00
Adam Williamson
ce4cf5d022 drop the user_logged_in_20150721f23 variant needle
Summary:
I actually have no idea why I created this, now. Examining it
closely there is no difference in the matched area between it
and user_logged_in. Both are evidently using the eurlatgr font
(tall @ sign). I tested that the kickstart_user_creation test
still works on F23 Alpha TC2 with this needle gone, and the
kickstart tests are the only ones that use it.

Test Plan: Run the kickstart tests and verify they still work.

Reviewers: jskladan, garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D484
2015-08-05 03:25:55 -07:00