Add pkg-map element.

Adds a new pkg-map element which can be used to install
packages based on an in element 'pkg-map' JSON config
file format.

As part of this change we expose DIB's IMAGE_ELEMENT variable
so that we can have access to the element names which are being
installed in a clean manner.

This change is intended to decouple elements from DIB
and allow new elements to support multiple distributions
(with various package naming schemes) without having to
constantly maintain DIB's various bin/map-packages files.

Change-Id: Ic0a061995e2ae708c95a535cba6237bff58f7d93

README.md

@@ -322,6 +322,10 @@ To set environment variables for other hooks, add a file to environment.d.
 This directory contains bash script snippets that are sourced before running
 scripts in each phase.
 
+DIB exposes an internal IMAGE\_ELEMENT variable which provides elements access
+to the full set of elements that are included in the image build. This can
+be used to process local in-element files across all the elements
+(pkg-map for example).
 
 ### Dependencies ###
 

elements/pkg-map/README.md

@@ -0,0 +1,68 @@
+Map package names to distro specific packages.
+
+Provides the following:
+
+ * bin/pkg-map
+
+   usage: pkg-map [-h] [--element ELEMENT] [--distro DISTRO]
+
+   Translate package name to distro specific name.
+
+   optional arguments:
+     -h, --help         show this help message and exit
+     --element ELEMENT  The element (namespace) to use for translation.
+     --distro DISTRO    The distro name to use for translation. Defaults to
+                        DISTRO_NAME
+
+ * Any element may create its own pkg-map JSON config file using
+   the one of 3 sections for the distro/family/ and or default.
+   The family is set automatically within pkg-map based on
+   the supplied distro name. Families include:
+     + redhat: includes centos, fedora, and rhel distros
+     + debian: includes debian and ubuntu distros
+     + suse: includes the opensuse distro
+
+   The most specific section takes priority.
+   Example for Nova and Glance (NOTE: using fictitious package names
+   for Fedora to provide a good example!)
+
+   Example format:
+    {
+      "distro": {
+        "fedora": {
+          "nova_package": "openstack-compute",
+          "glance_package": "openstack-image"
+        }
+      },
+      "family": {
+        "redhat": {
+          "nova_package": "openstack-nova",
+          "glance_package": "openstack-glance"
+        }
+      },
+      "default": {
+        "nova_package": "nova",
+        "glance_package": "glance"
+      }
+    }
+
+   Example commands using this format:
+
+   pkg-map --element nova-compute --distro fedora nova_package
+
+   Returns: openstack-compute
+
+   pkg-map --element nova-compute --distro rhel nova_package
+
+   Returns: openstack-nova
+
+   pkg-map --element nova-compute --distro ubuntu nova_package
+
+   Returns: nova
+
+ * This output can be used to filter what other tools actually install
+   (install-packages can be modified to use this for example)
+
+ * Individual pkg-map files live within each element. For example
+   if you are created an Apache element your pkg-map JSON file
+   should be created at elements/apache/pkg-map.

elements/pkg-map/bin/pkg-map

@@ -0,0 +1,105 @@
+#!/usr/bin/env python
+
+# Copyright 2014 Red Hat Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+
+import argparse
+import json
+import os
+import sys
+
+
+def os_family(distro):
+    family = None
+    if distro in ['centos', 'fedora', 'rhel']:
+        family = 'redhat'
+    elif distro in ['debian', 'ubuntu']:
+        family = 'debian'
+    elif distro == 'opensuse':
+        family = 'suse'
+
+    return family
+
+
+def main():
+    parser = argparse.ArgumentParser(
+        description="Translate package name to distro specific name.")
+    parser.add_argument('--element', default='',
+                        help='The element (namespace) to use for translation.')
+    parser.add_argument('--distro', default=os.environ.get('DISTRO_NAME'),
+                        help='The distro name to use for translation.'
+                        ' Defaults to DISTRO_NAME')
+    args, extra = parser.parse_known_args()
+
+    if not args.element:
+        print('Please specify an --element argument.')
+        sys.exit(1)
+
+    if not args.distro:
+        print('Please specify a --distro argument or set DISTRO_NAME.')
+        sys.exit(1)
+
+    map_file = '/usr/share/pkg-map/%s' % args.element
+    if not os.path.exists(map_file):
+        print('Map file for %s element does not exist.' % args.element)
+        sys.exit(1)
+
+    with open(map_file) as fd:
+        package_names = json.loads(fd.read())
+
+    # Parse mapping data in this form using distro/family/default
+    # Most specific takes priority (distro is most specific).
+    # Example for Nova and Glance (using fictitious name for Fedora)
+    # {
+    #   "distro": {
+    #     "fedora": {
+    #       "nova_package": "openstack-compute",
+    #       "glance_package": "openstack-image"
+    #     }
+    #   },
+    #   "family": {
+    #     "redhat": {
+    #       "nova_package": "openstack-nova",
+    #       "glance_package": "openstack-glance"
+    #     }
+    #   },
+    #   "default": {
+    #     "nova_package": "nova",
+    #     "glance_package": "glance"
+    #   }
+    # }
+    name_map = package_names.get('default', {})
+    if 'family' in package_names:
+        family_map = package_names['family'].get(os_family(args.distro))
+        if family_map:
+            name_map.update(family_map)
+    if 'distro' in package_names:
+        distro_map = package_names['distro'].get(args.distro)
+        if distro_map:
+            name_map.update(distro_map)
+    for name in extra:
+        pkg_name = None
+        if name in name_map:
+            pkg_name = name_map.get(name)
+        if pkg_name:
+            print(pkg_name)
+        else:
+            err_msg = 'Missing package name for distro/element: %s/%s'
+            print(err_msg % (args.distro, args.element))
+            sys.exit(1)
+
+    sys.exit(0)
+
+if __name__ == '__main__':
+    main()

elements/pkg-map/extra-data.d/10-create-pkg-map-dir

@@ -0,0 +1,13 @@
+#!/bin/bash
+set -eux
+set -o pipefail
+
+sudo mkdir -p $TMP_MOUNT_PATH/usr/share/pkg-map/
+
+for ELEMENT in $IMAGE_ELEMENT ; do
+    for DIR in ${ELEMENTS_PATH//:/ }; do
+        if [ -f "$DIR/$ELEMENT/pkg-map" ]; then
+            sudo cp $DIR/$ELEMENT/pkg-map $TMP_MOUNT_PATH/usr/share/pkg-map/$ELEMENT
+        fi
+    done
+done

lib/common-functions

@@ -241,7 +241,7 @@ function arg_to_elements() {
   fi
   echo "Building elements: $IMAGE_ELEMENT"
 
-  IMAGE_ELEMENT=$($SCRIPT_HOME/element-info --expand-dependencies $IMAGE_ELEMENT)
+  export IMAGE_ELEMENT=$($SCRIPT_HOME/element-info --expand-dependencies $IMAGE_ELEMENT)
 
   echo "Expanded element dependencies to: $IMAGE_ELEMENT"
 }