Add a yum-minimal element that just uses yum

The centos-minimal approach of using rinse does not, it turns out, work
on centos. That's a bummer. It's also rather heavyweight. Instead, with
minor machinations, we can just use yum itself pointed at a chroot.

Also adding fedora-minimal element which creates a fedora image using
the new yum-minimal approach.

Co-Authored-By: Gregory Haynes <greg@greghaynes.net>

Change-Id: I026fd9d323e786dae5bb67824c6501067e1ceaa3

elements/fedora-minimal/README.rst

@@ -0,0 +1,20 @@
+==============
+fedora-minimal
+==============
+Create a minimal image based on Fedora.
+
+Use of this element will require 'yum' and 'yum-utils' to be installed on
+Ubuntu and Debian. Nothing additional is needed on Fedora or CentOS. The
+element will need `python-lzma` everywhere.
+
+Due to a bug in the released version of urlgrabber, on many systems an
+installation of urlgrabber from git is required. The git repository
+can be found here: http://yum.baseurl.org/gitweb?p=urlgrabber.git;a=summary
+
+The `DIB_OFFLINE` or more specific `DIB_YUMCHROOT_USE_CACHE`
+variables can be set to prefer the use of a pre-cached root filesystem
+tarball.
+
+This element sets the `DIB_RELEASE` var to 'fedora'. The release of fedora
+to be installed can be controlled through the `DIB_RELEASE` variable, which
+defaults to '21'.

elements/fedora-minimal/element-deps

@@ -0,0 +1 @@
+yum-minimal

elements/fedora-minimal/element-provides

@@ -0,0 +1 @@
+operating-system

elements/fedora-minimal/environment.d/10-fedora-distro-name.bash

@@ -0,0 +1,2 @@
+export DISTRO_NAME=fedora
+export DIB_RELEASE=${DIB_RELEASE:-21}

elements/fedora-minimal/install.d/99-ramdisk

@@ -0,0 +1,17 @@
+#!/bin/bash
+
+if [ ${DIB_DEBUG_TRACE:-1} -gt 0 ]; then
+    set -x
+fi
+set -eu
+set -o pipefail
+
+initrd=$(find /boot -name initrd)
+kernel_version=$(rpm -qa | grep kernel | sort | head -n 1 | cut -d '-' -f 2,3)
+
+if [ "$(echo $initrd | wc -l)" -eq 1 ]; then
+    cp $initrd /boot/initrd-$kernel_version.img
+else
+    echo "Zero or multiple initrds found. This should not happen."
+    exit 1
+fi

elements/fedora-minimal/yum.repos.d/yum.repo

@@ -0,0 +1,16 @@
+[fedora]
+name=Fedora $releasever - $basearch
+failovermethod=priority
+#baseurl=http://download.fedoraproject.org/pub/fedora/linux/releases/$releasever/Everything/$basearch/os/
+metalink=https://mirrors.fedoraproject.org/metalink?repo=fedora-$releasever&arch=$basearch
+gpgcheck=0
+skip_if_unavailable=False
+
+[updates]
+name=Fedora $releasever - $basearch - Updates
+failovermethod=priority
+#baseurl=http://download.fedoraproject.org/pub/fedora/linux/updates/$releasever/$basearch/
+metalink=https://mirrors.fedoraproject.org/metalink?repo=updates-released-f$releasever&arch=$basearch
+enabled=1
+gpgcheck=0
+skip_if_unavailable=False

elements/yum-minimal/README.rst

@@ -0,0 +1,14 @@
+===========
+yum-minimal
+===========
+Base element for creating minimal yum-based images.
+
+This element is incomplete by itself, you'll want to use the centos-minimal
+or fedora-minimal elements to get an actual base image.
+
+Use of this element will require 'yum' and 'yum-utils' to be installed on
+Ubuntu and Debian. Nothing additional is needed on Fedora or CentOS.
+
+The `DIB_OFFLINE` or more specific `DIB_YUMCHROOT_USE_CACHE`
+variables can be set to prefer the use of a pre-cached root filesystem
+tarball.

elements/yum-minimal/element-deps

@@ -0,0 +1,4 @@
+dib-run-parts
+redhat-common
+rpm-distro
+yum

elements/yum-minimal/install.d/10-base-networking

@@ -0,0 +1,37 @@
+#!/bin/bash
+#
+# Copyright 2015 Hewlett-Packard Development Company, L.P.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+#
+
+if [ "${DIB_DEBUG_TRACE:-0}" -gt 0 ]; then
+    set -x
+fi
+set -eu
+set -o pipefail
+
+# allow networking init scripts inside the container to work without extra steps
+cat << EOF | sudo tee /etc/sysconfig/network > /dev/null
+NETWORKING=yes
+NETWORKING_IPV6=yes
+NOZEROCONF=yes
+EOF
+
+for interface in eth0 eth1; do
+    cat << EOF | tee /etc/sysconfig/network-scripts/ifcfg-$interface > /dev/null
+DEVICE=$interface
+BOOTPROTO=dhcp
+ONBOOT=on
+EOF
+done

elements/yum-minimal/install.d/15-base-fstab

@@ -0,0 +1,27 @@
+#!/bin/bash
+#
+# Copyright 2015 Hewlett-Packard Development Company, L.P.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+#
+
+if [ "${DIB_DEBUG_TRACE:-0}" -gt 0 ]; then
+    set -x
+fi
+set -eu
+set -o pipefail
+
+cat << EOF | tee /etc/fstab > /dev/null
+proc /proc proc nodev,noexec,nosuid 0 0
+LABEL=${DIB_ROOT_LABEL} / ext4 errors=remount-ro 0 1
+EOF

elements/yum-minimal/package-installs.yaml

@@ -0,0 +1,5 @@
+dracut:
+kernel:
+initscripts:
+man-pages:
+redhat-lsb-core:

elements/yum-minimal/pkg-map

@@ -0,0 +1,8 @@
+{
+  "family": {
+    "redhat": {
+      "lsb_release": "redhat-lsb-core"
+    }
+  }
+}
+

elements/yum-minimal/pre-install.d/03-yum-cleanup

@@ -0,0 +1,37 @@
+#!/bin/bash
+#
+# Copyright 2015 Hewlett-Packard Development Company, L.P.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+#
+
+if [ "${DIB_DEBUG_TRACE:-0}" -gt 0 ]; then
+    set -x
+fi
+set -eu
+set -o pipefail
+
+# effectively: febootstrap-minimize --keep-zoneinfo --keep-rpmdb --keep-services "$target"
+# locales
+rm -rf /usr/{{lib,share}/locale,{lib,lib64}/gconv,bin/localedef,sbin/build-locale-archive}
+# docs
+rm -rf /usr/share/{doc,info,gnome/help}
+# cracklib
+rm -rf /usr/share/cracklib
+# i18n
+rm -rf /usr/share/i18n
+# sln
+rm -rf /sbin/sln
+# ldconfig
+rm -rf /etc/ld.so.cache /var/cache/ldconfig
+mkdir -p --mode=0755 /var/cache/ldconfig

elements/yum-minimal/root.d/08-yum-chroot

@@ -0,0 +1,108 @@
+#!/bin/bash
+#
+# Copyright 2015 Hewlett-Packard Development Company, L.P.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+#
+if [ "${DIB_DEBUG_TRACE:-0}" -gt 0 ]; then
+    set -x
+fi
+set -eu
+set -o pipefail
+
+if [ -f ${TARGET_ROOT}/.extra_settings ] ; then
+    . ${TARGET_ROOT}/.extra_settings
+fi
+ARCH=${ARCH:-x86_64}
+if [ $ARCH = amd64 ]; then
+    ARCH=x86_64
+fi
+# Calling elements will need to set DISTRO_NAME and DIB_RELEASE
+DIB_YUMCHROOT_EXTRA_ARGS=${DIB_YUMCHROOT_EXTRA_ARGS:-}
+YUMCHROOT_TARBALL=$DIB_IMAGE_CACHE/yumchroot-${DISTRO_NAME}-${DIB_RELEASE}-${ARCH}.tar.gz
+# TODO Maybe deal with DIB_DISTRIBUTION_MIRROR
+http_proxy=${http_proxy:-}
+
+set -x
+if [ -n "$DIB_OFFLINE" -o -n "${DIB_YUMCHROOT_USE_CACHE:-}" ] && [ -f $YUMCHROOT_TARBALL ] ; then
+    echo $YUMCHROOT_TARBALL found in cache. Using.
+    sudo tar -C $TARGET_ROOT --numeric-owner -xzf $YUMCHROOT_TARBALL
+else
+    sudo mkdir -p $TARGET_ROOT/var/lib/rpm
+    sudo rpm --root $TARGET_ROOT --initdb
+
+    WORKING=$(mktemp --tmpdir=${TMP_DIR:-/tmp} -d)
+    EACTION="rm -r $WORKING"
+    trap "$EACTION" EXIT
+
+    yumdownloader \
+            --releasever=$DIB_RELEASE \
+            --setopt=reposdir=$TMP_HOOKS_PATH/yum.repos.d \
+            --destdir=$WORKING \
+        ${DISTRO_NAME}-release
+    RELEASE_RPMS="${DISTRO_NAME}-release"
+    if [ $DISTRO_NAME = fedora ] ; then
+        yumdownloader \
+                --releasever=$DIB_RELEASE \
+                --setopt=reposdir=$TMP_HOOKS_PATH/yum.repos.d \
+                --destdir=$WORKING \
+            ${DISTRO_NAME}-repos
+        RELEASE_RPMS="${RELEASE_RPMS} ${DISTRO_NAME}-repos"
+    fi
+    sudo rpm --root $TARGET_ROOT -ivh $WORKING/*rpm
+
+    YUM_CACHE=$DIB_IMAGE_CACHE/yum
+
+    # Install yum into the chroot but use the gpg keys we've installed
+    # directly into the chroot for the purpose
+    sudo sed -i "s,/etc/pki/rpm-gpg,$TARGET_ROOT/etc/pki/rpm-gpg,g" $TARGET_ROOT/etc/yum.repos.d/*repo
+    sudo yum -y \
+            --setopt=cachedir=$YUM_CACHE/$ARCH/$DIB_RELEASE \
+            --setopt=reposdir=$TARGET_ROOT/etc/yum.repos.d \
+            --installroot $TARGET_ROOT \
+        install yum
+
+    # Set gpg path back because subsequent actions will take place in the chroot
+    sudo sed -i "s,$TARGET_ROOT/etc/pki/rpm-gpg,/etc/pki/rpm-gpg,g" $TARGET_ROOT/etc/yum.repos.d/*repo
+
+    # We have to do this next bit outside of the chroot to get far enough
+    # that dib-run-parts can operate
+    sudo cp /etc/resolv.conf $TARGET_ROOT/etc/resolv.conf
+
+    # Same logic as in the yum element to provide for yum caching
+    # copied here because the sequencing is wrong otherwise
+    sudo mkdir -p $TMP_MOUNT_PATH/tmp/yum
+    sudo mount --bind $YUM_CACHE $TMP_MOUNT_PATH/tmp/yum
+    sudo chroot $TARGET_ROOT yum -y --releasever=$DIB_RELEASE \
+        --setopt=cachedir=/tmp/yum/$ARCH/$DIB_RELEASE \
+        install $RELEASE_RPMS
+
+    sudo chroot $TARGET_ROOT yum makecache
+    sudo chroot $TARGET_ROOT yum -y \
+        --setopt=cachedir=/tmp/yum/$ARCH/$DIB_RELEASE \
+        install passwd findutils sudo util-linux-ng
+
+    # RPM doesn't know whether files have been changed since install
+    # At this point though, we know for certain that we have changed no
+    # config files, so anything marked .rpmnew is just a bug.
+    for newfile in $(sudo find $TARGET_ROOT -type f -name '*rpmnew') ; do
+        sudo mv $newfile $(echo $newfile | sed 's/.rpmnew$//')
+    done
+    sudo rm $TARGET_ROOT/etc/resolv.conf
+    sudo umount $TMP_MOUNT_PATH/tmp/yum
+
+    echo Caching result in $YUMCHROOT_TARBALL
+    sudo tar --numeric-owner -C $TARGET_ROOT -zcf $YUMCHROOT_TARBALL --exclude='./tmp/*' .
+fi
+
+sudo rm -f ${TARGET_ROOT}/.extra_settings