diskimage-builder/diskimage_builder/elements/pkg-map
Ian Wienand f068e6aa6e Fix package-installs-v2 output
The current output for package-installs-v2 is inscrutable [1]

The problem starts with process_output() which is not capturing
stderr.  This means that any stderr output is dislocated from any
stdout output around it.  This is *really* confusing as you get a
bunch of seemingly meaningless stderr output from any calls before you
see any stdout (e.g. in [1] you can see random yum error output that
should have been with the yum call)).  The simplest thing to do is to
redirect stderr to stdout which keeps everything in sync.

This causes a slight problem, however, because pkg-map outputs both
status information and errors on stderr.  To work around this but
maintain compatibility, we add a "--prefix" argument that prepends
mapped packages from pkg-map with a value we can match on.  The
existing status/debug output from pkg-map is low-value; modify the
call so that it will be traced only at higher debug levels (e.g. -x
-x).

The current loop is also calling pkg-map for every package in every
element (this is why in [1] the same message is repeated over and
over).  This is unnecessary; it only needs to pkg-map once for each
element, giving the package list as the arguments.  Create package
lists by element and pass those to pkg-map.

As a cleanup, there is no point in printing e.output if the
process_output fails for the install because we are already tracing
it; i.e. the output, even for failures, is already in the logs.
Printing it again just duplicates the output.

[2] is an extract showing what I feel is a much more understandable
log output for a fairly complex install.

[1] http://paste.openstack.org/show/595118/
[2] http://paste.openstack.org/show/595303/

Change-Id: Ia74602a5d2db032a476481caec0e45dab013d54f
2017-04-07 13:48:53 +10:00
..
bin Fix package-installs-v2 output 2017-04-07 13:48:53 +10:00
extra-data.d Move elements & lib relative to diskimage_builder package 2016-11-01 17:27:41 -07:00
element-deps Move elements & lib relative to diskimage_builder package 2016-11-01 17:27:41 -07:00
README.rst Move elements & lib relative to diskimage_builder package 2016-11-01 17:27:41 -07:00

=======
pkg-map
=======
Map package names to distro specific packages.

Provides the following:

 * bin/pkg-map::

    usage: pkg-map [-h] [--element ELEMENT] [--distro DISTRO]

    Translate package name to distro specific name.

    optional arguments:
      -h, --help         show this help message and exit
      --element ELEMENT  The element (namespace) to use for translation.
      --distro DISTRO    The distro name to use for translation. Defaults to
                         DISTRO_NAME
      --release RELEASE  The release to use for translation.  Defaults to
                         DIB_RELEASE

 * Any element may create its own pkg-map JSON config file using the
   one of 4 sections for the release/distro/family/ and or default.
   The family is set automatically within pkg-map based on the
   supplied distro name. Families include:

     + redhat: includes centos, fedora, and rhel distros
     + debian: includes debian and ubuntu distros
     + suse: includes the opensuse distro

   The release is a specification of distro; i.e. the distro and
   release must mach for a translation.

   The most specific section takes priority.

   An empty package list can be provided.

   Example for Nova and Glance (NOTE: using fictitious package names
   for Fedora and package mapping for suse family to provide a good
   example!)

   Example format::

    {
      "release": {
        "fedora": {
          "23": {
            "nova_package": "foo" "bar"
          }
        }
      },
      "distro": {
        "fedora": {
          "nova_package": "openstack-compute",
          "glance_package": "openstack-image"
        }
      },
      "family": {
        "redhat": {
          "nova_package": "openstack-nova",
          "glance_package": "openstack-glance"
        },
        "suse": {
          "nova_package": ""
        }
      },
      "default": {
        "nova_package": "nova",
        "glance_package": "glance"
      }
    }

   Example commands using this format:

   pkg-map --element nova-compute --distro fedora nova_package

   Returns: openstack-compute

   pkg-map --element nova-compute --distro rhel nova_package

   Returns: openstack-nova

   pkg-map --element nova-compute --distro ubuntu nova_package

   Returns: nova

   pkg-map --element nova-compute --distro opensuse nova_package

   Returns:

 * This output can be used to filter what other tools actually install
   (install-packages can be modified to use this for example)

 * Individual pkg-map files live within each element. For example
   if you are created an Apache element your pkg-map JSON file
   should be created at elements/apache/pkg-map.