Introduce openEuler distro

Add openeuler-minimal element and add CI functional tests for both
x86_64 and arm64.

OpenEuler is an open source community driven YUM/DNF distro like
Fedora. It references Fedora and CentOS a lot for the rpm packages
building. So somewhat it can be treated as a redhat family distro
and reuse the YUM/DNF related elements to help build openEuler images.

For more info about openEuler, see: https://openeuler.org/en

Depends-On: https://review.opendev.org/c/zuul/zuul-jobs/+/803413
Change-Id: I3e06e49b524364c3a4edeba8bce7a8c06b9c7b76

.zuul.d/jobs.yaml

@@ -31,6 +31,7 @@
         - ubuntu-minimal/focal-build-succeeds
         - containerfile/focal-build-succeeds
         - fedora-container/build-succeeds
+        - openeuler-minimal/20.03-LTS-SP2-build-succeeds
 
 - job:
     name: dib-functests-image
@@ -89,6 +90,7 @@
         - ubuntu-minimal/bionic-arm64-build-succeeds
         - ubuntu-minimal/focal-arm64-build-succeeds
         - centos-minimal/8-aarch64-build-succeeds
+        - openeuler-minimal/20.03-LTS-SP2-build-succeeds
 
 # Bionic / Python3 tests
 - job:

diskimage_builder/elements/openeuler-minimal/README.rst

@@ -0,0 +1,12 @@
+=================
+openeuler-minimal
+=================
+Create a minimal image from scratch.
+
+Use of this element will require 'yum' and 'yum-utils' to be installed on
+Ubuntu and Debian. Nothing additional is needed on Fedora, CentOS or
+openEuler.
+
+Set ``DIB_RELEASE`` to ``20.03-LTS-SP2``,``21.03`` or other release number
+to explicitly select the release. ``DIB_RELEASE`` defaults the latest LTS
+release.

diskimage_builder/elements/openeuler-minimal/element-deps

@@ -0,0 +1,4 @@
+cloud-init
+growroot
+openssh-server
+yum-minimal

diskimage_builder/elements/openeuler-minimal/element-provides

@@ -0,0 +1 @@
+operating-system

diskimage_builder/elements/openeuler-minimal/environment.d/10-openeuler-distro-name.bash

@@ -0,0 +1,3 @@
+export DISTRO_NAME=openeuler
+export DIB_RELEASE=${DIB_RELEASE:-20.03-LTS-SP2}
+export EFI_BOOT_DIR="EFI/openEuler"

diskimage_builder/elements/openeuler-minimal/environment.d/11-yum-dnf.bash

@@ -0,0 +1 @@
+export YUM=dnf

diskimage_builder/elements/openeuler-minimal/package-installs.yaml

@@ -0,0 +1,2 @@
+openEuler-latest-release:
+vim-minimal:

diskimage_builder/elements/openeuler-minimal/root.d/00-00-generate-repo

@@ -0,0 +1,29 @@
+#!/bin/bash
+
+if [ ${DIB_DEBUG_TRACE:-1} -gt 0 ]; then
+    set -x
+fi
+set -eu
+set -o pipefail
+
+# NOTE(xinliang): This script should run before script elements/yum-minimal/root.d/08-yum-chroot
+# So set the script name prefix as 00-
+yum_repos_dir="$(dirname $0)/../yum.repos.d"
+mkdir -p $yum_repos_dir
+
+repo_file=$(mktemp)
+cat > $repo_file << EOF
+[OS]
+name=openEuler-$DIB_RELEASE - OS
+baseurl=http://repo.openeuler.org/openEuler-$DIB_RELEASE/OS/\$basearch/
+enabled=1
+gpgcheck=0
+
+[update]
+name=openEuler-$DIB_RELEASE - update
+baseurl=http://repo.openeuler.org/openEuler-$DIB_RELEASE/update/\$basearch/
+enabled=1
+gpgcheck=0
+EOF
+
+mv $repo_file ${yum_repos_dir}/openEuler.repo

diskimage_builder/elements/openeuler-minimal/test-elements/20.03-LTS-SP2-build-succeeds/README.rst

@@ -0,0 +1 @@
+openEuler 20.03-LTS-SP2 test

diskimage_builder/elements/openeuler-minimal/test-elements/20.03-LTS-SP2-build-succeeds/element-deps

@@ -0,0 +1,3 @@
+block-device-efi
+openstack-ci-mirrors
+vm

diskimage_builder/elements/openeuler-minimal/test-elements/20.03-LTS-SP2-build-succeeds/environment.d/10-set-distro.bash

@@ -0,0 +1 @@
+export DIB_RELEASE='20.03-LTS-SP2'

diskimage_builder/elements/openeuler-minimal/test-elements/20.03-LTS-SP2-build-succeeds/test-output-formats

@@ -0,0 +1 @@
+qcow2

diskimage_builder/elements/openstack-ci-mirrors/environment.d/11-dib-distribution-mirror.bash

@@ -58,6 +58,8 @@ if [[ -d ${DIB_OS_CI_YUM_REPOS:-/not/a/path/} ]]; then
         fi
     elif [[ "${DISTRO_NAME}" == "centos" ]]; then
         export DIB_YUM_MINIMAL_BOOTSTRAP_REPOS=${DIB_OS_CI_YUM_REPOS}/centos-minimal/${DIB_RELEASE}/yum.repos.d
+    elif [[ "${DISTRO_NAME}" == "openeuler" ]]; then
+        export DIB_YUM_MINIMAL_BOOTSTRAP_REPOS=${DIB_OS_CI_YUM_REPOS}/openeuler-minimal/${DIB_RELEASE}/yum.repos.d
     fi
 
 fi

diskimage_builder/elements/pkg-map/README.rst

@@ -24,7 +24,7 @@ Provides the following:
    The family is set automatically within pkg-map based on the
    supplied distro name. Families include:
 
-     + redhat: includes centos, fedora, and rhel distros
+     + redhat: includes centos, fedora, openeuler and rhel distros
      + debian: includes debian and ubuntu distros
      + suse: includes the opensuse distro
 

diskimage_builder/elements/pkg-map/bin/pkg-map

@@ -26,7 +26,7 @@ log = logging.getLogger()
 
 def os_family(distro):
     family = None
-    if distro in ['fedora', 'rhel', 'rhel7', 'centos', 'centos7']:
+    if distro in ['fedora', 'rhel', 'rhel7', 'centos', 'centos7', 'openeuler']:
         family = 'redhat'
     elif distro in ['debian', 'ubuntu']:
         family = 'debian'

diskimage_builder/elements/redhat-common/README.rst

@@ -1,7 +1,7 @@
 =============
 redhat-common
 =============
-Image installation steps common to RHEL, CentOS, and Fedora.
+Image installation steps common to RHEL, CentOS, Fedora and openEuler.
 
 Requirements:
 

diskimage_builder/elements/redhat-common/pkg-map

@@ -4,6 +4,11 @@
       "libselinux-python": "python3-libselinux",
       "policycoreutils": "python3-policycoreutils",
       "policycoreutils-python": "policycoreutils-python-utils"
+    },
+    "openeuler": {
+      "libselinux-python": "python3-libselinux",
+      "policycoreutils": "python3-policycoreutils",
+      "policycoreutils-python": "policycoreutils-python-utils"
     }
   },
   "release": {

diskimage_builder/elements/yum-minimal/pkg-map

@@ -1,4 +1,12 @@
 {
+  "distro": {
+    "openeuler": {
+      "linux-firmware": "",
+      "linux-firmware-whence": "",
+      "lsb_release": "openeuler-lsb",
+      "redhat-rpm-config": "openEuler-rpm-config"
+    }
+  },
   "release": {
       "centos": {
           "7": {

diskimage_builder/elements/yum-minimal/root.d/08-yum-chroot

@@ -31,6 +31,8 @@ fi
 ARCH=${ARCH:-x86_64}
 if [ $ARCH = amd64 ]; then
     ARCH=x86_64
+elif [[ "arm64" == "$ARCH" ]]; then
+    ARCH="aarch64"
 fi
 # Calling elements will need to set DISTRO_NAME and DIB_RELEASE
 # TODO Maybe deal with DIB_DISTRIBUTION_MIRROR
@@ -75,6 +77,8 @@ function _install_repos {
     packages+="basesystem filesystem setup "
     if [[ ${DISTRO_NAME} = fedora && ${DIB_RELEASE} -gt 29 ]]; then
         packages+="fedora-release-cloud fedora-release-common "
+    elif [[ ${DISTRO_NAME} == 'openeuler' ]]; then
+        packages+="openEuler-release "
     else
         packages+="${DISTRO_NAME}-release "
     fi
@@ -99,6 +103,11 @@ function _install_repos {
         fi
     fi
 
+    # For openEuler, install other required repos and gpy-keys packages.
+    if [[ ${DISTRO_NAME} == 'openeuler' ]]; then
+        packages+="openEuler-repos openEuler-gpg-keys "
+    fi
+
     # By default, parent elements (fedora-minimal, centos-minimal)
     # have a yum.repos.d directory in the element with a default repo;
     # this is copied to TMP_HOOK_PATH by the usual hook-copying
@@ -151,6 +160,12 @@ function _install_repos {
         sudo cp ${DIB_YUM_MINIMAL_EXTRA_REPOS}/* \
             $TARGET_ROOT/etc/yum.repos.d/
     fi
+
+    # For openEuler, some repos like update are disabled by default.
+    # Ensure all the repo is enabled, so that we get the latest packages.
+    if [[ ${DISTRO_NAME} == 'openeuler' ]]; then
+        sudo sed -i 's/enabled=0/enabled=1/' $TARGET_ROOT/etc/yum.repos.d/*.repo
+    fi
 }
 
 # _install_pkg_manager packages...
@@ -292,7 +307,7 @@ _install_repos
 # it seems, as it seems like as part of the Fedora modular updates
 # (https://docs.fedoraproject.org/en-US/modularity/) we can pick up
 # seemingly mismatched libraries.
-if [[ ${DISTRO_NAME} == 'fedora' ]]; then
+if [[ ${DISTRO_NAME} =~ (fedora|openeuler) ]]; then
     _install_pkg_manager dnf dnf-plugins-core curl
 elif [[ ${DISTRO_NAME} == centos && $DIB_RELEASE > "7" ]]; then
     _install_pkg_manager dnf dnf-plugins-core curl libcurl

doc/source/user_guide/supported_distros.rst

@@ -19,4 +19,5 @@ Distributions which are supported as a target for an image:
 - RHEL 6, 7
 - 16.04 ("xenial"), 18.04 ("bionic")
 - Gentoo
+- openEuler 20.03-LTS-SP1+
 - openSUSE Leap 42.3, 15.0, 15.1 and Tumbleweed (opensuse-minimal only)

releasenotes/notes/add-openeuler-minimal-element-068dc7d9d94e5098.yaml

@@ -0,0 +1,6 @@
+---
+features:
+  - |
+    Add ``openeuler-minimal`` element. OpenEuler is an open source community driven
+    YUM/DNF distro like Fedora.
+    For more info about openEuler, see: https://openeuler.org/en.

roles/dib-setup-gate-mirrors/tasks/main.yaml

@@ -58,3 +58,18 @@
   with_nested:
     - [ 'default' ]
     - [ 'fedora.repo', 'fedora-updates.repo' ]
+
+- name: Create openeuler-minimal 20.03-LTS-SP2 directory
+  file:
+    path: "{{ dib_gate_mirror_repos }}/openeuler-minimal/20.03-LTS-SP2/yum.repos.d"
+    state: directory
+    mode: 0775
+    recurse: yes
+
+- name: Install openeuler-minimal 20.03-LTS-SP2 repo files
+  template:
+    dest: "{{ dib_gate_mirror_repos }}/openeuler-minimal/20.03-LTS-SP2/yum.repos.d/dib-mirror-{{ item }}"
+    mode: 0644
+    src: "openeuler-minimal/20.03-LTS-SP2/{{ item }}.j2"
+  with_items:
+    - openEuler.repo

roles/dib-setup-gate-mirrors/templates/openeuler-minimal/20.03-LTS-SP2/openEuler.repo.j2

@@ -0,0 +1,29 @@
+[OS]
+name=OS
+baseurl=http://{{ mirror_fqdn }}/openeuler/openEuler-20.03-LTS-SP2/OS/$basearch/
+enabled=1
+gpgcheck=0
+
+[everything]
+name=everything
+baseurl=http://{{ mirror_fqdn }}/openeuler/openEuler-20.03-LTS-SP2/everything/$basearch/
+enabled=1
+gpgcheck=0
+
+[EPOL]
+name=EPOL
+baseurl=http://{{ mirror_fqdn }}/openeuler/openEuler-20.03-LTS-SP2/EPOL/main/$basearch/
+enabled=1
+gpgcheck=0
+
+[EPOL-UPDATE]
+name=EPOL-UPDATE
+baseurl=http://{{ mirror_fqdn }}/openeuler/openEuler-20.03-LTS-SP2/EPOL/update/main/$basearch/
+enabled=1
+gpgcheck=0
+
+[update]
+name=update
+baseurl=http://{{ mirror_fqdn }}/openeuler/openEuler-20.03-LTS-SP2/update/$basearch/
+enabled=1
+gpgcheck=0