Adam Williamson 764c6dbd95 Notice when update package should have been installed but wasn't ... We hit an interesting case in update testing recently: https://bodhi.fedoraproject.org/updates/FEDORA-2018-115068f60e An earlier version of that update failed testing. When we dug into it a bit, we found that the test was failing because an earlier version of the `pki-server` package was installed than the version that was in the update; when asked (as part of FreeIPA deployment) to install it, dnf had noticed that there were dependency issues with the version of the package from the update, but it happened to be able to install the version from the frozen 'stable' repo...so it just went ahead and did that. In this case, the 'missed' package resulted in a test failure, but it'd actually be possible for this to happen and the test to complete; we really ought to notice when this happens, and treat it as a test failure. So what this attempts to do is: at the end of all update tests, check for all installed packages with the same name as a package from the update, and compare their full NEVR to the one of the package from the update. If a package with the same name as one of the update packages is installed, but does not appear to be the *same NEVR*, we fail, and upload the lists of packages for manual investigation as to what the heck's going on. Signed-off-by: Adam Williamson <awilliam@redhat.com>		2018-12-12 22:17:29 -08:00
..
_advisory_post.pm	Notice when update package should have been installed but wasn't	2018-12-12 22:17:29 -08:00
_advisory_update.pm	Add a test of bootchain stuff for updates	2018-10-04 16:43:15 -07:00
_anaconda_network_static.pm	Remove test_flags comments, add ignore_failure flag	2017-04-10 15:00:10 -07:00
_boot_to_anaconda.pm	Have kickstart tests handle RHBZ#1618928 too	2018-08-19 10:03:27 -04:00
_check_install_source.pm	Work around an anaconda logging bug that showed up today	2018-06-28 15:13:06 -07:00
_collect_data.pm	Move collect_data and console_shutdown to tty4...	2018-03-09 22:06:18 -08:00
_console_avc_crash.pm	Add a softfail workaround for #1600823	2018-07-13 00:39:53 -07:00
_console_login.pm	Fix some minor syntax errors in previous commit	2018-06-13 10:11:53 -07:00
_console_shutdown.pm	Disable hidden grub menu for uploaded base installs	2018-10-12 15:36:55 -07:00
_console_wait_login_2.pm	Workaround issues with loading same module more than once	2017-04-18 10:38:16 -07:00
_console_wait_login.pm	Remove test_flags comments, add ignore_failure flag	2017-04-10 15:00:10 -07:00
_do_install_and_reboot.pm	Use longer timeout for root console switch after liveinst	2018-10-06 08:44:34 -07:00
_graphical_input_2.pm	Workaround issues with loading same module more than once	2017-04-18 10:38:16 -07:00
_graphical_input.pm	Tweak _graphical_input to work with transparent top bar	2017-07-10 14:41:57 -07:00
_graphical_wait_login_2.pm	Workaround issues with loading same module more than once	2017-04-18 10:38:16 -07:00
_graphical_wait_login.pm	Drop the workaround for #1625572	2018-09-16 08:39:30 -07:00
_memcheck.pm	Remove test_flags comments, add ignore_failure flag	2017-04-10 15:00:10 -07:00
_post_network_static.pm	Remove test_flags comments, add ignore_failure flag	2017-04-10 15:00:10 -07:00
_setup_browser.pm	Move install of Firefox into its own milestone module	2018-03-23 09:10:51 -07:00
_software_selection.pm	Check default package selection is correct	2018-01-05 15:18:09 -08:00
_support_server.pm	support_server: workaround RHBZ#1554390 (breaks dnsmasq on F28)	2018-05-03 14:00:41 -07:00
base_selinux.pm	Remove test_flags comments, add ignore_failure flag	2017-04-10 15:00:10 -07:00
base_service_manipulation.pm	Remove test_flags comments, add ignore_failure flag	2017-04-10 15:00:10 -07:00
base_services_start.pm	Move the new base_services_start check to the start	2018-07-13 14:02:31 -07:00
base_system_logging.pm	add base system logging test	2017-06-01 11:06:04 +02:00
base_update_cli.pm	Remove test_flags comments, add ignore_failure flag	2017-04-10 15:00:10 -07:00
database_client.pm	Bump up some more DNF timeouts due to #1516990	2017-11-23 16:01:58 -08:00
desktop_browser.pm	Change up 'clean desktop' check again (use a util function)	2017-07-10 11:47:07 -07:00
desktop_notifications.pm	Fix tty switching for desktop_notifications	2018-10-12 21:16:33 -07:00
desktop_terminal.pm	Change up 'clean desktop' check again (use a util function)	2017-07-10 11:47:07 -07:00
desktop_update_graphical.pm	Tweak previous commit to avoid some bugs	2018-10-31 12:45:11 -07:00
disk_custom_blivet_btrfs.pm	Add "PPC PReP Boot" filesystem for PowerPC blivet tests	2017-09-06 08:43:04 +02:00
disk_custom_blivet_ext3.pm	Add "PPC PReP Boot" filesystem for PowerPC blivet tests	2017-09-06 08:43:04 +02:00
disk_custom_blivet_lvmthin.pm	Add "PPC PReP Boot" filesystem for PowerPC blivet tests	2017-09-06 08:43:04 +02:00
disk_custom_blivet_no_swap.pm	Add "PPC PReP Boot" filesystem for PowerPC blivet tests	2017-09-06 08:43:04 +02:00
disk_custom_blivet_software_raid.pm	Add "PPC PReP Boot" filesystem for PowerPC blivet tests	2017-09-06 08:43:04 +02:00
disk_custom_blivet_xfs.pm	Add "PPC PReP Boot" filesystem for PowerPC blivet tests	2017-09-06 08:43:04 +02:00
disk_custom_btrfs_postinstall.pm	add postinstalls to custom partitioning tests	2017-05-09 09:11:34 +02:00
disk_custom_btrfs.pm	Remove test_flags comments, add ignore_failure flag	2017-04-10 15:00:10 -07:00
disk_custom_ext3_postinstall.pm	add postinstalls to custom partitioning tests	2017-05-09 09:11:34 +02:00
disk_custom_ext3.pm	Remove test_flags comments, add ignore_failure flag	2017-04-10 15:00:10 -07:00
disk_custom_iscsi_postinstall.pm	Remove test_flags comments, add ignore_failure flag	2017-04-10 15:00:10 -07:00
disk_custom_iscsi.pm	Remove test_flags comments, add ignore_failure flag	2017-04-10 15:00:10 -07:00
disk_custom_lvmthin_postinstall.pm	add postinstalls to custom partitioning tests	2017-05-09 09:11:34 +02:00
disk_custom_lvmthin.pm	Remove test_flags comments, add ignore_failure flag	2017-04-10 15:00:10 -07:00
disk_custom_no_swap_postinstall.pm	add postinstalls to custom partitioning tests	2017-05-09 09:11:34 +02:00
disk_custom_no_swap.pm	Remove test_flags comments, add ignore_failure flag	2017-04-10 15:00:10 -07:00
disk_custom_software_raid_postinstall.pm	add postinstalls to custom partitioning tests	2017-05-09 09:11:34 +02:00
disk_custom_software_raid.pm	Remove test_flags comments, add ignore_failure flag	2017-04-10 15:00:10 -07:00
disk_custom_xfs_postinstall.pm	add postinstalls to custom partitioning tests	2017-05-09 09:11:34 +02:00
disk_custom_xfs.pm	Remove test_flags comments, add ignore_failure flag	2017-04-10 15:00:10 -07:00
disk_guided_delete_all.pm	Remove test_flags comments, add ignore_failure flag	2017-04-10 15:00:10 -07:00
disk_guided_delete_partial_postinstall.pm	Remove test_flags comments, add ignore_failure flag	2017-04-10 15:00:10 -07:00
disk_guided_delete_partial.pm	Remove test_flags comments, add ignore_failure flag	2017-04-10 15:00:10 -07:00
disk_guided_empty.pm	Try a different workaround for RHBZ #1444225	2017-07-10 12:24:41 -07:00
disk_guided_encrypted_postinstall_2.pm	Workaround issues with loading same module more than once	2017-04-18 10:38:16 -07:00
disk_guided_encrypted_postinstall.pm	Remove test_flags comments, add ignore_failure flag	2017-04-10 15:00:10 -07:00
disk_guided_encrypted.pm	Remove test_flags comments, add ignore_failure flag	2017-04-10 15:00:10 -07:00
disk_guided_free_space_postinstall.pm	Remove test_flags comments, add ignore_failure flag	2017-04-10 15:00:10 -07:00
disk_guided_multi_empty_all_postinstall.pm	Remove test_flags comments, add ignore_failure flag	2017-04-10 15:00:10 -07:00
disk_guided_multi_empty_all.pm	Remove test_flags comments, add ignore_failure flag	2017-04-10 15:00:10 -07:00
disk_guided_multi_postinstall.pm	Revert "Modify "install_multi" test for PowerPC"	2017-09-06 08:43:04 +02:00
disk_guided_multi.pm	Remove test_flags comments, add ignore_failure flag	2017-04-10 15:00:10 -07:00
disk_guided_shrink_postinstall.pm	Remove test_flags comments, add ignore_failure flag	2017-04-10 15:00:10 -07:00
disk_guided_shrink.pm	Remove test_flags comments, add ignore_failure flag	2017-04-10 15:00:10 -07:00
firewall_configured.pm	Remove test_flags comments, add ignore_failure flag	2017-04-10 15:00:10 -07:00
firewall_disabled.pm	Remove test_flags comments, add ignore_failure flag	2017-04-10 15:00:10 -07:00
freeipa_client.pm	Remove test_flags comments, add ignore_failure flag	2017-04-10 15:00:10 -07:00
freeipa_password_change.pm	Work around Firefox 'close multiple tabs' warning	2018-10-30 18:34:37 -07:00
freeipa_webui.pm	Work around Firefox 'close multiple tabs' warning	2018-10-30 18:34:37 -07:00
install_arm_image_deployment.pm	Remove test_flags comments, add ignore_failure flag	2017-04-10 15:00:10 -07:00
install_source_graphical.pm	Fix install_mirrorlist_graphical test	2017-10-16 14:23:56 -07:00
install_text.pm	Remove test_flags comments, add ignore_failure flag	2017-04-10 15:00:10 -07:00
modularity_enable_disable_module.pm	Create openqa tests to test modularity.	2018-09-26 23:09:36 -07:00
modularity_install_module.pm	Create openqa tests to test modularity.	2018-09-26 23:09:36 -07:00
modularity_module_list.pm	Create openqa tests to test modularity.	2018-09-26 23:09:36 -07:00
modularity_update_norepo.pm	Create openqa tests to test modularity.	2018-09-26 23:09:36 -07:00
realmd_join_cockpit.pm	Crank sss debuglevel up to 9, and also do it for cockpit test	2018-11-30 14:58:53 -08:00
realmd_join_sssd.pm	Crank sss debuglevel up to 9, and also do it for cockpit test	2018-11-30 14:58:53 -08:00
rescue_mode_encrypted.pm	Brown paper bag fix: need use utils for type_safely	2018-06-24 17:21:08 -07:00
role_deploy_database_server.pm	Stop using rolekit for database server role on F29+	2018-07-10 17:54:30 -07:00
role_deploy_domain_controller_check.pm	Add a check that FreeIPA is actually up after upgrade	2018-10-05 23:53:17 -07:00
role_deploy_domain_controller.pm	Add a FreeIPA replication job set	2018-08-02 09:59:40 -07:00
server_cockpit_basic.pm	Adjust cockpit log needles for Rawhide change	2017-07-10 10:35:27 -07:00
server_cockpit_default.pm	Work around Firefox 'close multiple tabs' warning	2018-10-30 18:34:37 -07:00
server_filesystem_default.pm	Add a check that correct filesystem was used on Server installs	2018-01-05 15:18:05 -08:00
server_firewall_default.pm	Remove test_flags comments, add ignore_failure flag	2017-04-10 15:00:10 -07:00
uefi_postinstall.pm	Load us console layout in uefi_postinstall too	2018-06-25 11:28:12 -07:00
upgrade_boot.pm	Use mirrorlist instead of baseurl for updates tests	2018-05-18 16:41:13 -07:00
upgrade_postinstall.pm	Remove test_flags comments, add ignore_failure flag	2017-04-10 15:00:10 -07:00
upgrade_preinstall.pm	Extend some boot timeouts in upgrade tests	2018-05-11 09:02:21 -07:00
upgrade_run.pm	Use mirrorlist instead of baseurl for updates tests	2018-05-18 16:41:13 -07:00