Add svc-map element.

Adds a new svc-map element which can be used to install
services based on an in element 'svc-map' YAML config
file format.

This change is intended to decouple elements from DIB
and allow new elements to support multiple distributions
(with various service naming schemes) without having to
constantly maintain DIB's various bin/map-services files.

This patch ensures all of the YAML files are compiled into
a single file without namespaces.  The YAML compilation
process occurs during image creation time and errors if
conflicting mappings are found.

Change-Id: Id11433ea342aace71a358936a7ca3151ec11d506

elements/svc-map/README.md

@@ -0,0 +1,103 @@
+Map service names to distro specific services.
+
+Provides the following:
+
+ * bin/svc-map
+
+   usage: svc-map [-h] SERVICE
+
+   Translate service name to distro specific name.
+
+   optional arguments:
+     -h, --help         show this help message and exit
+
+ * Any element may create its own svc-map YAML config file using
+   the one of 3 sections for the distro/family/ and or default.
+   The family is set automatically within svc-map based on
+   the supplied distro name. Families include:
+     + redhat: includes centos, fedora, and rhel distros
+     + debian: includes debian and ubuntu distros
+     + suse: includes the opensuse distro
+
+   The most specific section takes priority. Example for Nova and Glance
+   (NOTE: default is using the common value for redhat and suse families)
+
+   The key used for the service name should always be the same name used for
+   the source installation of the service.  The svc-map script will check for
+   the source name against systemd and upstart and return that name if it
+   exists instead of the mapped name.
+
+    Example format for Nova:
+
+      nova-api:
+        default: openstack-nova-api
+        debian: nova-api
+      nova-cert:
+        default: openstack-nova-cert
+        debian:  nova-cert
+      nova-compute:
+        default: openstack-nova-compute
+        debian: nova-compute
+      nova-conductor:
+        default: openstack-nova-conductor
+        debian: nova-conductor
+      nova-consoleauth:
+        default: openstack-nova-console
+        debian: nova-console
+
+
+    Example format for Glance
+
+      glance-api:
+        debian: glance-api
+        default: openstack-glance-api
+      glance-reg:
+        debian: glance-reg
+        default: openstack-glance-registry
+
+
+    If the distro is of the debian family the combined services file would be:
+
+        nova-cert: nova-cert
+        nova-compute: nova-compute
+        glance-api: glance-api
+        nova-conductor: nova-conductor
+        nova-api: nova-api
+        glance-reg: glance-reg
+        nova-consoleauth: nova-console
+
+
+    If the distro is of the suse or redhat families the combined services file would be:
+
+        nova-cert: openstack-nova-cert
+        nova-compute: openstack-nova-compute
+        glance-reg: openstack-glance-registry
+        nova-conductor: openstack-nova-conductor
+        glance-api: openstack-glance-api
+        nova-consoleauth: openstack-nova-console
+        nova-api: openstack-nova-api
+
+
+   Example commands using this format:
+
+   svc-map nova-compute
+
+   Returns: openstack-nova-compute
+
+   svc-map nova-compute
+
+   Returns: openstack-nova-compute
+
+   svc-map nova-compute
+
+   Returns: nova-compute
+
+ * This output can be used to filter what other tools actually install
+   (install-services can be modified to use this for example)
+
+ * If you pass more than one service argument, the result for each service
+   is printed on its own line.
+
+ * Individual svc-map files live within each element. For example
+   if you have created an Apache element your svc-map YAML file
+   should be created at elements/apache/svc-map.

elements/svc-map/bin/svc-map

@@ -0,0 +1,46 @@
+#!/usr/bin/env python
+
+# Copyright 2012 Hewlett-Packard Development Company, L.P.
+# Copyright 2014 Red Hat, Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+import os
+import sys
+import yaml
+
+
+def load_service_mapping(filepath="/usr/share/svc-map/services"):
+    if not os.path.isfile(filepath):
+        return {}
+    with open(filepath, 'r') as data_file:
+        return yaml.load(data_file.read())
+
+
+def main():
+    for arg in sys.argv[1:]:
+        # We need to support the service name being different when installing
+        # from source vs. packages. So, if the requested service file already
+        # exists, just use that.
+        if os.path.exists('/lib/systemd/system/%s.service' % arg):  # systemd
+            print(arg)
+        elif os.path.exists('/etc/init/%s.conf' % arg):  # upstart
+            print(arg)
+        else:
+            service_map = load_service_mapping()
+            name = service_map.get(arg, arg)
+            print(name)
+    sys.exit(0)
+
+
+if __name__ == '__main__':
+    main()

elements/svc-map/extra-data.d/10-merge-svc-map-files

@@ -0,0 +1,70 @@
+#!/usr/bin/env python
+import os
+import sys
+import yaml
+
+
+def os_family(distro):
+    """Given a distribution, returns a operating system family."""
+    family = None
+    if distro in ['centos', 'fedora', 'rhel', 'rhel7']:
+        family = 'redhat'
+    elif distro in ['debian', 'ubuntu']:
+        family = 'debian'
+    elif distro == 'opensuse':
+        family = 'suse'
+    return family
+
+
+def merge_data(source, destination, distro):
+    """Merges two dictionaries and filters on distro, family, or default
+    (in order)."""
+    result = dict()
+    result.update(destination)
+    family = os_family(distro)
+    for servicename, mapping in source.iteritems():
+        if servicename in result:
+            raise Exception("%s already found in services list" % servicename)
+        if distro in mapping:
+            result[servicename] = mapping.get(distro)
+        elif family in mapping:
+            result[servicename] = mapping.get(family)
+        elif 'default' in mapping:
+            result[servicename] = mapping.get('default')
+        else:
+            result[servicename] = servicename
+
+    return result
+
+
+def write_data_to_file(data, service_file_path):
+    """Writes yaml data to a specified path."""
+    with open(service_file_path, 'w') as destination:
+        yaml.dump(data, destination, default_flow_style=False)
+
+
+def main():
+    elements = os.environ.get("IMAGE_ELEMENT").split(' ')
+    element_paths = os.environ.get("ELEMENTS_PATH").split(':')
+    service_names = dict()
+    for element in elements:
+        for element_path in element_paths:
+            data_path = os.path.join(element_path, element, "svc-map")
+            if os.path.exists(data_path):
+                with open(data_path, 'r') as dataFile:
+                    data = yaml.load(dataFile.read())
+                    try:
+                        service_names = merge_data(
+                            data,
+                            service_names,
+                            os.environ.get("DISTRO_NAME"))
+                    except Exception as err:
+                        print("%s.  Check %s for duplicate \
+                            service name." % (err, element))
+                        sys.exit(1)
+
+    write_data_to_file(service_names, os.path.join("/tmp", 'svc-map-services'))
+
+
+if __name__ == "__main__":
+    main()

elements/svc-map/extra-data.d/11-copy-svc-map-file

@@ -0,0 +1,5 @@
+#!/bin/bash
+set -eux
+set -o pipefail
+
+sudo install -D -o root -g root -m 0644 /tmp/svc-map-services "$TMP_MOUNT_PATH/usr/share/svc-map/services"

elements/svc-map/tests/test_data_merge.py

@@ -0,0 +1,179 @@
+# Copyright 2014 Red Hat, Inc.
+#
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
+import imp
+service_map = imp.load_source('service_map',
+                              '../extra-data.d/10-merge-svc-map-files')
+
+from oslotest import base
+
+
+class TestDataMerge(base.BaseTestCase):
+
+    nova_api_services = {
+        u'nova-api': {
+            u'debian': u'nova-api',
+            u'default': u'openstack-nova-api'
+        },
+        u'nova-cert': {
+            u'debian': u'nova-cert',
+            u'default': u'openstack-nova-cert'
+        },
+        u'nova-compute': {
+            u'debian': u'nova-compute',
+            u'default': u'openstack-nova-compute'
+        },
+        u'nova-conductor': {
+            u'debian': u'nova-conductor',
+            u'default': u'openstack-nova-conductor'
+        },
+        u'nova-consoleauth': {
+            u'debian': u'nova-console',
+            u'default': u'openstack-nova-console'
+        }
+    }
+
+    glance_api_services = {
+        u'glance-api': {
+            u'debian': u'glance-api',
+            u'default': u'openstack-glance-api'
+        },
+        u'glance-reg': {
+            u'debian': u'glance-reg',
+            u'default': u'openstack-glance-registry'
+        }
+    }
+
+    cinder_api_services = {
+        u'cinder-api': {
+            u'debian': u'cinder-api',
+            u'default': u'openstack-cinder-api'
+        },
+        u'cinder-scheduler': {
+            u'debian': u'cinder-scheduler',
+            u'default': u'openstack-cinder-scheduler'
+        }
+    }
+
+    def test_merge_data_fedora(self):
+
+        fedora_nova_api_services = {
+            u'nova-api': u'openstack-nova-api',
+            u'nova-cert': u'openstack-nova-cert',
+            u'nova-compute': u'openstack-nova-compute',
+            u'nova-conductor': u'openstack-nova-conductor',
+            u'nova-consoleauth': u'openstack-nova-console'
+        }
+
+        fedora_nova_glance_services = {
+            u'nova-api': u'openstack-nova-api',
+            u'nova-cert': u'openstack-nova-cert',
+            u'nova-compute': u'openstack-nova-compute',
+            u'nova-conductor': u'openstack-nova-conductor',
+            u'nova-consoleauth': u'openstack-nova-console',
+            u'glance-api': u'openstack-glance-api',
+            u'glance-reg': u'openstack-glance-registry'
+        }
+
+        fedora_nova_glance_cinder_services = {
+            u'nova-api': u'openstack-nova-api',
+            u'nova-cert': u'openstack-nova-cert',
+            u'nova-compute': u'openstack-nova-compute',
+            u'nova-conductor': u'openstack-nova-conductor',
+            u'nova-consoleauth': u'openstack-nova-console',
+            u'glance-api': u'openstack-glance-api',
+            u'glance-reg': u'openstack-glance-registry',
+            u'cinder-api': u'openstack-cinder-api',
+            u'cinder-scheduler': u'openstack-cinder-scheduler',
+        }
+
+        result = dict()
+        result = service_map.merge_json(self.nova_api_services,
+                                        result,
+                                        "fedora")
+
+        self.assertDictEqual(result,
+                             fedora_nova_api_services,
+                             "Merge failed")
+
+        result = service_map.merge_json(self.glance_api_services,
+                                        result,
+                                        "fedora")
+
+        self.assertDictEqual(result,
+                             fedora_nova_glance_services,
+                             "Merge failed")
+
+        result = service_map.merge_json(self.cinder_api_services,
+                                        result,
+                                        "fedora")
+        self.assertDictEqual(result,
+                             fedora_nova_glance_cinder_services,
+                             "Merge failed")
+
+    def test_merge_data_ubuntu(self):
+
+        ubuntu_nova_api_services = {
+            u'nova-api': u'nova-api',
+            u'nova-cert': u'nova-cert',
+            u'nova-compute': u'nova-compute',
+            u'nova-conductor': u'nova-conductor',
+            u'nova-consoleauth': u'nova-console'
+        }
+
+        ubuntu_nova_glance_services = {
+            u'nova-api': u'nova-api',
+            u'nova-cert': u'nova-cert',
+            u'nova-compute': u'nova-compute',
+            u'nova-conductor': u'nova-conductor',
+            u'nova-consoleauth': u'nova-console',
+            u'glance-api': u'glance-api',
+            u'glance-reg': u'glance-reg'
+        }
+
+        ubuntu_nova_glance_cinder_services = {
+            u'nova-api': u'nova-api',
+            u'nova-cert': u'nova-cert',
+            u'nova-compute': u'nova-compute',
+            u'nova-conductor': u'nova-conductor',
+            u'nova-consoleauth': u'nova-console',
+            u'glance-api': u'glance-api',
+            u'glance-reg': u'glance-reg',
+            u'cinder-api': u'cinder-api',
+            u'cinder-scheduler': u'cinder-scheduler'
+        }
+
+        result = dict()
+        result = service_map.merge_json(self.nova_api_services,
+                                        result,
+                                        "ubuntu")
+
+        self.assertDictEqual(result,
+                             ubuntu_nova_api_services,
+                             "Merge failed")
+
+        result = service_map.merge_json(self.glance_api_services,
+                                        result,
+                                        "ubuntu")
+
+        self.assertDictEqual(result,
+                             ubuntu_nova_glance_services,
+                             "Merge failed")
+
+        result = service_map.merge_json(self.cinder_api_services,
+                                        result,
+                                        "ubuntu")
+
+        self.assertDictEqual(result,
+                             ubuntu_nova_glance_cinder_services,
+                             "Merge failed")

test-requirements.txt

@@ -2,5 +2,6 @@ hacking>=0.9.2,<0.10
 
 coverage>=3.6
 discover
+oslotest>=1.1.0
 testrepository>=0.0.18
 testtools>=0.9.34