diskimage-builder/diskimage_builder/elements/pkg-map/bin/pkg-map

209 lines
6.7 KiB
Text
Raw Normal View History

#!/usr/bin/env python3
# Copyright 2014 Red Hat Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import argparse
import json
import logging
import os
import sys
Fix package-installs-v2 output The current output for package-installs-v2 is inscrutable [1] The problem starts with process_output() which is not capturing stderr. This means that any stderr output is dislocated from any stdout output around it. This is *really* confusing as you get a bunch of seemingly meaningless stderr output from any calls before you see any stdout (e.g. in [1] you can see random yum error output that should have been with the yum call)). The simplest thing to do is to redirect stderr to stdout which keeps everything in sync. This causes a slight problem, however, because pkg-map outputs both status information and errors on stderr. To work around this but maintain compatibility, we add a "--prefix" argument that prepends mapped packages from pkg-map with a value we can match on. The existing status/debug output from pkg-map is low-value; modify the call so that it will be traced only at higher debug levels (e.g. -x -x). The current loop is also calling pkg-map for every package in every element (this is why in [1] the same message is repeated over and over). This is unnecessary; it only needs to pkg-map once for each element, giving the package list as the arguments. Create package lists by element and pass those to pkg-map. As a cleanup, there is no point in printing e.output if the process_output fails for the install because we are already tracing it; i.e. the output, even for failures, is already in the logs. Printing it again just duplicates the output. [2] is an extract showing what I feel is a much more understandable log output for a fairly complex install. [1] http://paste.openstack.org/show/595118/ [2] http://paste.openstack.org/show/595303/ Change-Id: Ia74602a5d2db032a476481caec0e45dab013d54f
2017-01-18 00:21:54 +00:00
log = logging.getLogger()
def os_family(distro):
family = None
if distro in ['fedora', 'rhel', 'rhel7', 'centos', 'centos7', 'openeuler',
'rocky']:
family = 'redhat'
elif distro in ['debian', 'ubuntu']:
family = 'debian'
elif distro == 'opensuse':
family = 'suse'
elif distro == 'gentoo':
family = 'gentoo'
return family
def os_major_release(release):
if release in ['8-stream']:
return '8'
return release
def main():
parser = argparse.ArgumentParser(
description="Translate package name to distro specific name."
" Exits with 1 if error is encountered, 2 if no pkg-map"
" file is found. Otherwise exits with 0.")
parser.add_argument('--element', default='',
help='The element (namespace) to use for translation.')
parser.add_argument('--pkg-map', default='',
help='Path to specific pkg-map file. '
'(Useful for testing)')
parser.add_argument('--distro', default=os.environ.get('DISTRO_NAME'),
help='The distro name to use for translation.'
' Defaults to DISTRO_NAME')
parser.add_argument('--release', default=os.environ.get('DIB_RELEASE'),
help='A more specfic name for distribution release')
parser.add_argument('--missing-ok', action="store_true",
help='Do not consider missing mappings an error.'
' Causes packages where no mapping is set to be'
' printed.')
Fix package-installs-v2 output The current output for package-installs-v2 is inscrutable [1] The problem starts with process_output() which is not capturing stderr. This means that any stderr output is dislocated from any stdout output around it. This is *really* confusing as you get a bunch of seemingly meaningless stderr output from any calls before you see any stdout (e.g. in [1] you can see random yum error output that should have been with the yum call)). The simplest thing to do is to redirect stderr to stdout which keeps everything in sync. This causes a slight problem, however, because pkg-map outputs both status information and errors on stderr. To work around this but maintain compatibility, we add a "--prefix" argument that prepends mapped packages from pkg-map with a value we can match on. The existing status/debug output from pkg-map is low-value; modify the call so that it will be traced only at higher debug levels (e.g. -x -x). The current loop is also calling pkg-map for every package in every element (this is why in [1] the same message is repeated over and over). This is unnecessary; it only needs to pkg-map once for each element, giving the package list as the arguments. Create package lists by element and pass those to pkg-map. As a cleanup, there is no point in printing e.output if the process_output fails for the install because we are already tracing it; i.e. the output, even for failures, is already in the logs. Printing it again just duplicates the output. [2] is an extract showing what I feel is a much more understandable log output for a fairly complex install. [1] http://paste.openstack.org/show/595118/ [2] http://paste.openstack.org/show/595303/ Change-Id: Ia74602a5d2db032a476481caec0e45dab013d54f
2017-01-18 00:21:54 +00:00
# This tool has traditionally output status and debug messages on
# stderr. The problem is if a caller has stderr > stdout then
# actual output gets messed in with the logs. This allows callers
# to disambiguate actual output by specifying a unique prefix.
parser.add_argument('--prefix', default='',
help='Output mapped packages with this prefix')
parser.add_argument('--debug', dest='debug', action="store_true",
help="Enable debugging output")
args, extra = parser.parse_known_args()
Fix package-installs-v2 output The current output for package-installs-v2 is inscrutable [1] The problem starts with process_output() which is not capturing stderr. This means that any stderr output is dislocated from any stdout output around it. This is *really* confusing as you get a bunch of seemingly meaningless stderr output from any calls before you see any stdout (e.g. in [1] you can see random yum error output that should have been with the yum call)). The simplest thing to do is to redirect stderr to stdout which keeps everything in sync. This causes a slight problem, however, because pkg-map outputs both status information and errors on stderr. To work around this but maintain compatibility, we add a "--prefix" argument that prepends mapped packages from pkg-map with a value we can match on. The existing status/debug output from pkg-map is low-value; modify the call so that it will be traced only at higher debug levels (e.g. -x -x). The current loop is also calling pkg-map for every package in every element (this is why in [1] the same message is repeated over and over). This is unnecessary; it only needs to pkg-map once for each element, giving the package list as the arguments. Create package lists by element and pass those to pkg-map. As a cleanup, there is no point in printing e.output if the process_output fails for the install because we are already tracing it; i.e. the output, even for failures, is already in the logs. Printing it again just duplicates the output. [2] is an extract showing what I feel is a much more understandable log output for a fairly complex install. [1] http://paste.openstack.org/show/595118/ [2] http://paste.openstack.org/show/595303/ Change-Id: Ia74602a5d2db032a476481caec0e45dab013d54f
2017-01-18 00:21:54 +00:00
# Logs have traditionally gone to stderr with this tool. Maintain
# compatibility
Fix package-installs-v2 output The current output for package-installs-v2 is inscrutable [1] The problem starts with process_output() which is not capturing stderr. This means that any stderr output is dislocated from any stdout output around it. This is *really* confusing as you get a bunch of seemingly meaningless stderr output from any calls before you see any stdout (e.g. in [1] you can see random yum error output that should have been with the yum call)). The simplest thing to do is to redirect stderr to stdout which keeps everything in sync. This causes a slight problem, however, because pkg-map outputs both status information and errors on stderr. To work around this but maintain compatibility, we add a "--prefix" argument that prepends mapped packages from pkg-map with a value we can match on. The existing status/debug output from pkg-map is low-value; modify the call so that it will be traced only at higher debug levels (e.g. -x -x). The current loop is also calling pkg-map for every package in every element (this is why in [1] the same message is repeated over and over). This is unnecessary; it only needs to pkg-map once for each element, giving the package list as the arguments. Create package lists by element and pass those to pkg-map. As a cleanup, there is no point in printing e.output if the process_output fails for the install because we are already tracing it; i.e. the output, even for failures, is already in the logs. Printing it again just duplicates the output. [2] is an extract showing what I feel is a much more understandable log output for a fairly complex install. [1] http://paste.openstack.org/show/595118/ [2] http://paste.openstack.org/show/595303/ Change-Id: Ia74602a5d2db032a476481caec0e45dab013d54f
2017-01-18 00:21:54 +00:00
level = logging.DEBUG if args.debug else logging.INFO
logging.basicConfig(stream=sys.stderr, level=level)
if not args.element and not args.pkg_map:
Fix package-installs-v2 output The current output for package-installs-v2 is inscrutable [1] The problem starts with process_output() which is not capturing stderr. This means that any stderr output is dislocated from any stdout output around it. This is *really* confusing as you get a bunch of seemingly meaningless stderr output from any calls before you see any stdout (e.g. in [1] you can see random yum error output that should have been with the yum call)). The simplest thing to do is to redirect stderr to stdout which keeps everything in sync. This causes a slight problem, however, because pkg-map outputs both status information and errors on stderr. To work around this but maintain compatibility, we add a "--prefix" argument that prepends mapped packages from pkg-map with a value we can match on. The existing status/debug output from pkg-map is low-value; modify the call so that it will be traced only at higher debug levels (e.g. -x -x). The current loop is also calling pkg-map for every package in every element (this is why in [1] the same message is repeated over and over). This is unnecessary; it only needs to pkg-map once for each element, giving the package list as the arguments. Create package lists by element and pass those to pkg-map. As a cleanup, there is no point in printing e.output if the process_output fails for the install because we are already tracing it; i.e. the output, even for failures, is already in the logs. Printing it again just duplicates the output. [2] is an extract showing what I feel is a much more understandable log output for a fairly complex install. [1] http://paste.openstack.org/show/595118/ [2] http://paste.openstack.org/show/595303/ Change-Id: Ia74602a5d2db032a476481caec0e45dab013d54f
2017-01-18 00:21:54 +00:00
log.error('Please specify an --element argument.')
sys.exit(1)
if args.element and args.pkg_map:
Fix package-installs-v2 output The current output for package-installs-v2 is inscrutable [1] The problem starts with process_output() which is not capturing stderr. This means that any stderr output is dislocated from any stdout output around it. This is *really* confusing as you get a bunch of seemingly meaningless stderr output from any calls before you see any stdout (e.g. in [1] you can see random yum error output that should have been with the yum call)). The simplest thing to do is to redirect stderr to stdout which keeps everything in sync. This causes a slight problem, however, because pkg-map outputs both status information and errors on stderr. To work around this but maintain compatibility, we add a "--prefix" argument that prepends mapped packages from pkg-map with a value we can match on. The existing status/debug output from pkg-map is low-value; modify the call so that it will be traced only at higher debug levels (e.g. -x -x). The current loop is also calling pkg-map for every package in every element (this is why in [1] the same message is repeated over and over). This is unnecessary; it only needs to pkg-map once for each element, giving the package list as the arguments. Create package lists by element and pass those to pkg-map. As a cleanup, there is no point in printing e.output if the process_output fails for the install because we are already tracing it; i.e. the output, even for failures, is already in the logs. Printing it again just duplicates the output. [2] is an extract showing what I feel is a much more understandable log output for a fairly complex install. [1] http://paste.openstack.org/show/595118/ [2] http://paste.openstack.org/show/595303/ Change-Id: Ia74602a5d2db032a476481caec0e45dab013d54f
2017-01-18 00:21:54 +00:00
log.error('Specify either --element or --pkg-map')
sys.exit(1)
if not args.distro:
Fix package-installs-v2 output The current output for package-installs-v2 is inscrutable [1] The problem starts with process_output() which is not capturing stderr. This means that any stderr output is dislocated from any stdout output around it. This is *really* confusing as you get a bunch of seemingly meaningless stderr output from any calls before you see any stdout (e.g. in [1] you can see random yum error output that should have been with the yum call)). The simplest thing to do is to redirect stderr to stdout which keeps everything in sync. This causes a slight problem, however, because pkg-map outputs both status information and errors on stderr. To work around this but maintain compatibility, we add a "--prefix" argument that prepends mapped packages from pkg-map with a value we can match on. The existing status/debug output from pkg-map is low-value; modify the call so that it will be traced only at higher debug levels (e.g. -x -x). The current loop is also calling pkg-map for every package in every element (this is why in [1] the same message is repeated over and over). This is unnecessary; it only needs to pkg-map once for each element, giving the package list as the arguments. Create package lists by element and pass those to pkg-map. As a cleanup, there is no point in printing e.output if the process_output fails for the install because we are already tracing it; i.e. the output, even for failures, is already in the logs. Printing it again just duplicates the output. [2] is an extract showing what I feel is a much more understandable log output for a fairly complex install. [1] http://paste.openstack.org/show/595118/ [2] http://paste.openstack.org/show/595303/ Change-Id: Ia74602a5d2db032a476481caec0e45dab013d54f
2017-01-18 00:21:54 +00:00
log.error('Please specify a --distro argument or set DISTRO_NAME.')
sys.exit(1)
if args.pkg_map:
Fix package-installs-v2 output The current output for package-installs-v2 is inscrutable [1] The problem starts with process_output() which is not capturing stderr. This means that any stderr output is dislocated from any stdout output around it. This is *really* confusing as you get a bunch of seemingly meaningless stderr output from any calls before you see any stdout (e.g. in [1] you can see random yum error output that should have been with the yum call)). The simplest thing to do is to redirect stderr to stdout which keeps everything in sync. This causes a slight problem, however, because pkg-map outputs both status information and errors on stderr. To work around this but maintain compatibility, we add a "--prefix" argument that prepends mapped packages from pkg-map with a value we can match on. The existing status/debug output from pkg-map is low-value; modify the call so that it will be traced only at higher debug levels (e.g. -x -x). The current loop is also calling pkg-map for every package in every element (this is why in [1] the same message is repeated over and over). This is unnecessary; it only needs to pkg-map once for each element, giving the package list as the arguments. Create package lists by element and pass those to pkg-map. As a cleanup, there is no point in printing e.output if the process_output fails for the install because we are already tracing it; i.e. the output, even for failures, is already in the logs. Printing it again just duplicates the output. [2] is an extract showing what I feel is a much more understandable log output for a fairly complex install. [1] http://paste.openstack.org/show/595118/ [2] http://paste.openstack.org/show/595303/ Change-Id: Ia74602a5d2db032a476481caec0e45dab013d54f
2017-01-18 00:21:54 +00:00
# specifying the pkg-map by hand is just for manual testing
element = "<%s>" % args.pkg_map
map_file = args.pkg_map
else:
Fix package-installs-v2 output The current output for package-installs-v2 is inscrutable [1] The problem starts with process_output() which is not capturing stderr. This means that any stderr output is dislocated from any stdout output around it. This is *really* confusing as you get a bunch of seemingly meaningless stderr output from any calls before you see any stdout (e.g. in [1] you can see random yum error output that should have been with the yum call)). The simplest thing to do is to redirect stderr to stdout which keeps everything in sync. This causes a slight problem, however, because pkg-map outputs both status information and errors on stderr. To work around this but maintain compatibility, we add a "--prefix" argument that prepends mapped packages from pkg-map with a value we can match on. The existing status/debug output from pkg-map is low-value; modify the call so that it will be traced only at higher debug levels (e.g. -x -x). The current loop is also calling pkg-map for every package in every element (this is why in [1] the same message is repeated over and over). This is unnecessary; it only needs to pkg-map once for each element, giving the package list as the arguments. Create package lists by element and pass those to pkg-map. As a cleanup, there is no point in printing e.output if the process_output fails for the install because we are already tracing it; i.e. the output, even for failures, is already in the logs. Printing it again just duplicates the output. [2] is an extract showing what I feel is a much more understandable log output for a fairly complex install. [1] http://paste.openstack.org/show/595118/ [2] http://paste.openstack.org/show/595303/ Change-Id: Ia74602a5d2db032a476481caec0e45dab013d54f
2017-01-18 00:21:54 +00:00
element = args.element
map_file = '/usr/share/pkg-map/%s' % element
log.info("Mapping for %s : %s" % (element, ' '.join(extra)))
if not os.path.exists(map_file):
if args.missing_ok:
Fix package-installs-v2 output The current output for package-installs-v2 is inscrutable [1] The problem starts with process_output() which is not capturing stderr. This means that any stderr output is dislocated from any stdout output around it. This is *really* confusing as you get a bunch of seemingly meaningless stderr output from any calls before you see any stdout (e.g. in [1] you can see random yum error output that should have been with the yum call)). The simplest thing to do is to redirect stderr to stdout which keeps everything in sync. This causes a slight problem, however, because pkg-map outputs both status information and errors on stderr. To work around this but maintain compatibility, we add a "--prefix" argument that prepends mapped packages from pkg-map with a value we can match on. The existing status/debug output from pkg-map is low-value; modify the call so that it will be traced only at higher debug levels (e.g. -x -x). The current loop is also calling pkg-map for every package in every element (this is why in [1] the same message is repeated over and over). This is unnecessary; it only needs to pkg-map once for each element, giving the package list as the arguments. Create package lists by element and pass those to pkg-map. As a cleanup, there is no point in printing e.output if the process_output fails for the install because we are already tracing it; i.e. the output, even for failures, is already in the logs. Printing it again just duplicates the output. [2] is an extract showing what I feel is a much more understandable log output for a fairly complex install. [1] http://paste.openstack.org/show/595118/ [2] http://paste.openstack.org/show/595303/ Change-Id: Ia74602a5d2db032a476481caec0e45dab013d54f
2017-01-18 00:21:54 +00:00
log.info("No package map for %s, done" % element)
for name in extra:
Fix package-installs-v2 output The current output for package-installs-v2 is inscrutable [1] The problem starts with process_output() which is not capturing stderr. This means that any stderr output is dislocated from any stdout output around it. This is *really* confusing as you get a bunch of seemingly meaningless stderr output from any calls before you see any stdout (e.g. in [1] you can see random yum error output that should have been with the yum call)). The simplest thing to do is to redirect stderr to stdout which keeps everything in sync. This causes a slight problem, however, because pkg-map outputs both status information and errors on stderr. To work around this but maintain compatibility, we add a "--prefix" argument that prepends mapped packages from pkg-map with a value we can match on. The existing status/debug output from pkg-map is low-value; modify the call so that it will be traced only at higher debug levels (e.g. -x -x). The current loop is also calling pkg-map for every package in every element (this is why in [1] the same message is repeated over and over). This is unnecessary; it only needs to pkg-map once for each element, giving the package list as the arguments. Create package lists by element and pass those to pkg-map. As a cleanup, there is no point in printing e.output if the process_output fails for the install because we are already tracing it; i.e. the output, even for failures, is already in the logs. Printing it again just duplicates the output. [2] is an extract showing what I feel is a much more understandable log output for a fairly complex install. [1] http://paste.openstack.org/show/595118/ [2] http://paste.openstack.org/show/595303/ Change-Id: Ia74602a5d2db032a476481caec0e45dab013d54f
2017-01-18 00:21:54 +00:00
print('%s%s' % (args.prefix, name))
sys.exit(0)
Fix package-installs-v2 output The current output for package-installs-v2 is inscrutable [1] The problem starts with process_output() which is not capturing stderr. This means that any stderr output is dislocated from any stdout output around it. This is *really* confusing as you get a bunch of seemingly meaningless stderr output from any calls before you see any stdout (e.g. in [1] you can see random yum error output that should have been with the yum call)). The simplest thing to do is to redirect stderr to stdout which keeps everything in sync. This causes a slight problem, however, because pkg-map outputs both status information and errors on stderr. To work around this but maintain compatibility, we add a "--prefix" argument that prepends mapped packages from pkg-map with a value we can match on. The existing status/debug output from pkg-map is low-value; modify the call so that it will be traced only at higher debug levels (e.g. -x -x). The current loop is also calling pkg-map for every package in every element (this is why in [1] the same message is repeated over and over). This is unnecessary; it only needs to pkg-map once for each element, giving the package list as the arguments. Create package lists by element and pass those to pkg-map. As a cleanup, there is no point in printing e.output if the process_output fails for the install because we are already tracing it; i.e. the output, even for failures, is already in the logs. Printing it again just duplicates the output. [2] is an extract showing what I feel is a much more understandable log output for a fairly complex install. [1] http://paste.openstack.org/show/595118/ [2] http://paste.openstack.org/show/595303/ Change-Id: Ia74602a5d2db032a476481caec0e45dab013d54f
2017-01-18 00:21:54 +00:00
else:
log.error('Required pkg-map for %s element does not exist.'
% args.element)
sys.exit(2)
with open(map_file) as fd:
try:
package_names = json.loads(fd.read())
Fix package-installs-v2 output The current output for package-installs-v2 is inscrutable [1] The problem starts with process_output() which is not capturing stderr. This means that any stderr output is dislocated from any stdout output around it. This is *really* confusing as you get a bunch of seemingly meaningless stderr output from any calls before you see any stdout (e.g. in [1] you can see random yum error output that should have been with the yum call)). The simplest thing to do is to redirect stderr to stdout which keeps everything in sync. This causes a slight problem, however, because pkg-map outputs both status information and errors on stderr. To work around this but maintain compatibility, we add a "--prefix" argument that prepends mapped packages from pkg-map with a value we can match on. The existing status/debug output from pkg-map is low-value; modify the call so that it will be traced only at higher debug levels (e.g. -x -x). The current loop is also calling pkg-map for every package in every element (this is why in [1] the same message is repeated over and over). This is unnecessary; it only needs to pkg-map once for each element, giving the package list as the arguments. Create package lists by element and pass those to pkg-map. As a cleanup, there is no point in printing e.output if the process_output fails for the install because we are already tracing it; i.e. the output, even for failures, is already in the logs. Printing it again just duplicates the output. [2] is an extract showing what I feel is a much more understandable log output for a fairly complex install. [1] http://paste.openstack.org/show/595118/ [2] http://paste.openstack.org/show/595303/ Change-Id: Ia74602a5d2db032a476481caec0e45dab013d54f
2017-01-18 00:21:54 +00:00
# log.debug(pprint.pformat(package_names))
except ValueError:
Fix package-installs-v2 output The current output for package-installs-v2 is inscrutable [1] The problem starts with process_output() which is not capturing stderr. This means that any stderr output is dislocated from any stdout output around it. This is *really* confusing as you get a bunch of seemingly meaningless stderr output from any calls before you see any stdout (e.g. in [1] you can see random yum error output that should have been with the yum call)). The simplest thing to do is to redirect stderr to stdout which keeps everything in sync. This causes a slight problem, however, because pkg-map outputs both status information and errors on stderr. To work around this but maintain compatibility, we add a "--prefix" argument that prepends mapped packages from pkg-map with a value we can match on. The existing status/debug output from pkg-map is low-value; modify the call so that it will be traced only at higher debug levels (e.g. -x -x). The current loop is also calling pkg-map for every package in every element (this is why in [1] the same message is repeated over and over). This is unnecessary; it only needs to pkg-map once for each element, giving the package list as the arguments. Create package lists by element and pass those to pkg-map. As a cleanup, there is no point in printing e.output if the process_output fails for the install because we are already tracing it; i.e. the output, even for failures, is already in the logs. Printing it again just duplicates the output. [2] is an extract showing what I feel is a much more understandable log output for a fairly complex install. [1] http://paste.openstack.org/show/595118/ [2] http://paste.openstack.org/show/595303/ Change-Id: Ia74602a5d2db032a476481caec0e45dab013d54f
2017-01-18 00:21:54 +00:00
log.error('Unable to parse %s' % map_file)
raise
# Parse mapping data in this form using release/distro/family/default
# Most specific takes priority; order is
# - release
# - distro
# - family
# - default
# An empty package list can be provided.
#
# Example for Nova and Glance (using fictitious name for Fedora and SUSE
# and package mapping for SUSE family)
# {
# "release": {
# "fedora" : {
# "23" : {
# "nova_package": "openstack-compute-foo"
# }
# }
# }
# "distro": {
# "fedora": {
# "nova_package": "openstack-compute",
# "glance_package": "openstack-image"
# }
# },
# "family": {
# "redhat": {
# "nova_package": "openstack-nova",
# "glance_package": "openstack-glance"
# },
# "suse": {
# "nova_package": ""
# }
# },
# "default": {
# "nova_package": "nova",
# "glance_package": "glance"
# }
# }
name_map = package_names.get('default', {})
if 'family' in package_names:
family_map = package_names['family'].get(os_family(args.distro))
if family_map:
name_map.update(family_map)
if 'distro' in package_names:
distro_map = package_names['distro'].get(args.distro)
if distro_map:
name_map.update(distro_map)
if 'release' in package_names:
try:
# release is a sub-concept of distro
release = os_major_release(args.release)
release_map = package_names['release'][args.distro][release]
name_map.update(release_map)
except KeyError:
pass
Fix package-installs-v2 output The current output for package-installs-v2 is inscrutable [1] The problem starts with process_output() which is not capturing stderr. This means that any stderr output is dislocated from any stdout output around it. This is *really* confusing as you get a bunch of seemingly meaningless stderr output from any calls before you see any stdout (e.g. in [1] you can see random yum error output that should have been with the yum call)). The simplest thing to do is to redirect stderr to stdout which keeps everything in sync. This causes a slight problem, however, because pkg-map outputs both status information and errors on stderr. To work around this but maintain compatibility, we add a "--prefix" argument that prepends mapped packages from pkg-map with a value we can match on. The existing status/debug output from pkg-map is low-value; modify the call so that it will be traced only at higher debug levels (e.g. -x -x). The current loop is also calling pkg-map for every package in every element (this is why in [1] the same message is repeated over and over). This is unnecessary; it only needs to pkg-map once for each element, giving the package list as the arguments. Create package lists by element and pass those to pkg-map. As a cleanup, there is no point in printing e.output if the process_output fails for the install because we are already tracing it; i.e. the output, even for failures, is already in the logs. Printing it again just duplicates the output. [2] is an extract showing what I feel is a much more understandable log output for a fairly complex install. [1] http://paste.openstack.org/show/595118/ [2] http://paste.openstack.org/show/595303/ Change-Id: Ia74602a5d2db032a476481caec0e45dab013d54f
2017-01-18 00:21:54 +00:00
# log.debug(pprint.pformat(name_map))
for name in extra:
pkg_name = name_map.get(name)
if pkg_name:
Fix package-installs-v2 output The current output for package-installs-v2 is inscrutable [1] The problem starts with process_output() which is not capturing stderr. This means that any stderr output is dislocated from any stdout output around it. This is *really* confusing as you get a bunch of seemingly meaningless stderr output from any calls before you see any stdout (e.g. in [1] you can see random yum error output that should have been with the yum call)). The simplest thing to do is to redirect stderr to stdout which keeps everything in sync. This causes a slight problem, however, because pkg-map outputs both status information and errors on stderr. To work around this but maintain compatibility, we add a "--prefix" argument that prepends mapped packages from pkg-map with a value we can match on. The existing status/debug output from pkg-map is low-value; modify the call so that it will be traced only at higher debug levels (e.g. -x -x). The current loop is also calling pkg-map for every package in every element (this is why in [1] the same message is repeated over and over). This is unnecessary; it only needs to pkg-map once for each element, giving the package list as the arguments. Create package lists by element and pass those to pkg-map. As a cleanup, there is no point in printing e.output if the process_output fails for the install because we are already tracing it; i.e. the output, even for failures, is already in the logs. Printing it again just duplicates the output. [2] is an extract showing what I feel is a much more understandable log output for a fairly complex install. [1] http://paste.openstack.org/show/595118/ [2] http://paste.openstack.org/show/595303/ Change-Id: Ia74602a5d2db032a476481caec0e45dab013d54f
2017-01-18 00:21:54 +00:00
log.debug("map %s -> %s" % (name, pkg_name))
print('%s%s' % (args.prefix, pkg_name))
elif name in name_map:
Fix package-installs-v2 output The current output for package-installs-v2 is inscrutable [1] The problem starts with process_output() which is not capturing stderr. This means that any stderr output is dislocated from any stdout output around it. This is *really* confusing as you get a bunch of seemingly meaningless stderr output from any calls before you see any stdout (e.g. in [1] you can see random yum error output that should have been with the yum call)). The simplest thing to do is to redirect stderr to stdout which keeps everything in sync. This causes a slight problem, however, because pkg-map outputs both status information and errors on stderr. To work around this but maintain compatibility, we add a "--prefix" argument that prepends mapped packages from pkg-map with a value we can match on. The existing status/debug output from pkg-map is low-value; modify the call so that it will be traced only at higher debug levels (e.g. -x -x). The current loop is also calling pkg-map for every package in every element (this is why in [1] the same message is repeated over and over). This is unnecessary; it only needs to pkg-map once for each element, giving the package list as the arguments. Create package lists by element and pass those to pkg-map. As a cleanup, there is no point in printing e.output if the process_output fails for the install because we are already tracing it; i.e. the output, even for failures, is already in the logs. Printing it again just duplicates the output. [2] is an extract showing what I feel is a much more understandable log output for a fairly complex install. [1] http://paste.openstack.org/show/595118/ [2] http://paste.openstack.org/show/595303/ Change-Id: Ia74602a5d2db032a476481caec0e45dab013d54f
2017-01-18 00:21:54 +00:00
log.debug("map %s -> <skip>" % (name))
continue
else:
if args.missing_ok:
Fix package-installs-v2 output The current output for package-installs-v2 is inscrutable [1] The problem starts with process_output() which is not capturing stderr. This means that any stderr output is dislocated from any stdout output around it. This is *really* confusing as you get a bunch of seemingly meaningless stderr output from any calls before you see any stdout (e.g. in [1] you can see random yum error output that should have been with the yum call)). The simplest thing to do is to redirect stderr to stdout which keeps everything in sync. This causes a slight problem, however, because pkg-map outputs both status information and errors on stderr. To work around this but maintain compatibility, we add a "--prefix" argument that prepends mapped packages from pkg-map with a value we can match on. The existing status/debug output from pkg-map is low-value; modify the call so that it will be traced only at higher debug levels (e.g. -x -x). The current loop is also calling pkg-map for every package in every element (this is why in [1] the same message is repeated over and over). This is unnecessary; it only needs to pkg-map once for each element, giving the package list as the arguments. Create package lists by element and pass those to pkg-map. As a cleanup, there is no point in printing e.output if the process_output fails for the install because we are already tracing it; i.e. the output, even for failures, is already in the logs. Printing it again just duplicates the output. [2] is an extract showing what I feel is a much more understandable log output for a fairly complex install. [1] http://paste.openstack.org/show/595118/ [2] http://paste.openstack.org/show/595303/ Change-Id: Ia74602a5d2db032a476481caec0e45dab013d54f
2017-01-18 00:21:54 +00:00
log.debug("pass -> %s" % (name))
print('%s%s' % (args.prefix, name))
else:
Fix package-installs-v2 output The current output for package-installs-v2 is inscrutable [1] The problem starts with process_output() which is not capturing stderr. This means that any stderr output is dislocated from any stdout output around it. This is *really* confusing as you get a bunch of seemingly meaningless stderr output from any calls before you see any stdout (e.g. in [1] you can see random yum error output that should have been with the yum call)). The simplest thing to do is to redirect stderr to stdout which keeps everything in sync. This causes a slight problem, however, because pkg-map outputs both status information and errors on stderr. To work around this but maintain compatibility, we add a "--prefix" argument that prepends mapped packages from pkg-map with a value we can match on. The existing status/debug output from pkg-map is low-value; modify the call so that it will be traced only at higher debug levels (e.g. -x -x). The current loop is also calling pkg-map for every package in every element (this is why in [1] the same message is repeated over and over). This is unnecessary; it only needs to pkg-map once for each element, giving the package list as the arguments. Create package lists by element and pass those to pkg-map. As a cleanup, there is no point in printing e.output if the process_output fails for the install because we are already tracing it; i.e. the output, even for failures, is already in the logs. Printing it again just duplicates the output. [2] is an extract showing what I feel is a much more understandable log output for a fairly complex install. [1] http://paste.openstack.org/show/595118/ [2] http://paste.openstack.org/show/595303/ Change-Id: Ia74602a5d2db032a476481caec0e45dab013d54f
2017-01-18 00:21:54 +00:00
log.error("%s has no valid mapping for package %s" %
(element, name))
sys.exit(1)
sys.exit(0)
if __name__ == '__main__':
main()
Fix package-installs-v2 output The current output for package-installs-v2 is inscrutable [1] The problem starts with process_output() which is not capturing stderr. This means that any stderr output is dislocated from any stdout output around it. This is *really* confusing as you get a bunch of seemingly meaningless stderr output from any calls before you see any stdout (e.g. in [1] you can see random yum error output that should have been with the yum call)). The simplest thing to do is to redirect stderr to stdout which keeps everything in sync. This causes a slight problem, however, because pkg-map outputs both status information and errors on stderr. To work around this but maintain compatibility, we add a "--prefix" argument that prepends mapped packages from pkg-map with a value we can match on. The existing status/debug output from pkg-map is low-value; modify the call so that it will be traced only at higher debug levels (e.g. -x -x). The current loop is also calling pkg-map for every package in every element (this is why in [1] the same message is repeated over and over). This is unnecessary; it only needs to pkg-map once for each element, giving the package list as the arguments. Create package lists by element and pass those to pkg-map. As a cleanup, there is no point in printing e.output if the process_output fails for the install because we are already tracing it; i.e. the output, even for failures, is already in the logs. Printing it again just duplicates the output. [2] is an extract showing what I feel is a much more understandable log output for a fairly complex install. [1] http://paste.openstack.org/show/595118/ [2] http://paste.openstack.org/show/595303/ Change-Id: Ia74602a5d2db032a476481caec0e45dab013d54f
2017-01-18 00:21:54 +00:00
# Tell emacs to use python-mode
# Local variables:
# mode: python
# End: