diskimage-builder/elements/yum-minimal/root.d/08-yum-chroot
Ian Wienand 08d6a9f93d yum-minimal: add systemd to initial install
It seems in the grub cleanup in
Iafe3611f4eec3c6357587a6cae6a30a261686ead I managed to unintentionally
drop systemd from the yum-minimal builds.  By not pre-installing grub
we dropped some dependencies; the path is tortured ... grub2 ->
os-prober -> udev -> systemd-udev -> systemd (we don't even want
os-prober!  So this whole thing was working by accident).

This manifests in *very* confusing ways.

Currently centos-minimal builds are failing late in the build with
services unable to enabled.  dib-init-system was actually trying to
tell us that it didn't know what init was installed (because systemd
wasn't actually installed), but unfortunately it was not really
failing.  This meant the service files were not copied correctly from
other elements, and thus fail to be enabled.  I have corrected this
with I076c08190d40c315ad6a6d96a3823e9fc52630be which would at least
alert us earlier.

For Fedora 24, due to a bug in dracut dependencies [1], missing the
systemd-udev package fails the build of the initrd during the kernel
install.  This then results in an initrd-less, unbootable system (see
also Ibaaa81124098f3c6febe48e455d3e1cd0a5f1761).

Add these dependencies explicitly.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1398505

Change-Id: I24ce648485c3d6f3c27ab8f87a638516b3727017
2016-11-25 21:09:11 +11:00

296 lines
12 KiB
Bash
Executable File

#!/bin/bash
#
# Copyright 2015 Hewlett-Packard Development Company, L.P.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
#
# dib-lint: disable=safe_sudo
if [ "${DIB_DEBUG_TRACE:-0}" -gt 0 ]; then
set -x
fi
set -eu
set -o pipefail
if [ -f ${TARGET_ROOT}/.extra_settings ] ; then
. ${TARGET_ROOT}/.extra_settings
fi
ARCH=${ARCH:-x86_64}
if [ $ARCH = amd64 ]; then
ARCH=x86_64
fi
# Calling elements will need to set DISTRO_NAME and DIB_RELEASE
DIB_YUMCHROOT_EXTRA_ARGS=${DIB_YUMCHROOT_EXTRA_ARGS:-}
YUMCHROOT_TARBALL=$DIB_IMAGE_CACHE/yumchroot-${DISTRO_NAME}-${DIB_RELEASE}-${ARCH}.tar.gz
# TODO Maybe deal with DIB_DISTRIBUTION_MIRROR
http_proxy=${http_proxy:-}
YUM=${YUM:-yum}
WORKING=$(mktemp --tmpdir=${TMP_DIR:-/tmp} -d)
EACTION="rm -r $WORKING"
trap "$EACTION" EXIT
YUM_CACHE=$DIB_IMAGE_CACHE/yum
mkdir -p $YUM_CACHE
# Note, on Debian/Ubuntu, %_dbpath is set in the RPM macros as
# ${HOME}/.rpmdb/ -- this makes sense as RPM isn't the system
# packager. This path is relative to the "--root" argument
_RPM="rpm --dbpath=/var/lib/rpm"
# install the [fedora|centos]-[release|repo] packages inside the
# chroot, which are needed to bootstrap yum/dnf
#
# note this runs outside the chroot, where we're assuming the platform
# has yum/yumdownloader
function _install_repos {
local packages
local rc
# pre-install the base system packages via rpm. We previously
# just left it up to yum to drag these in when we "yum install
# yum" in the chroot in _install_pkg_manager. This raised a small
# problem that inside the empty chroot yum went ahead and did a
# mkdir for /var/run to put some pid file in, which then messed up
# the "filesystem" package making /var/run a symlink to /run
# ... which leads to odd issues with a running system.
#
# TODO: these packages still have some small %posttrans stuff that
# depends on other packages (see rhbz#1306489) ... maybe the idea
# is that they are only installed in one big transaction with the
# rest of the system? but we don't want to use yum to do this
# (see above) so ...
packages="basesystem filesystem setup "
packages+="${DISTRO_NAME}-release "
# after fedora21, this is split into into a separate -repos
# package
if [ $DISTRO_NAME = fedora ]; then
packages+="${DISTRO_NAME}-repos "
fi
# yumdownloader puts repo xml files and such into a directory
# ${TMPDIR}/yum-$USER-random. Since we don't need this once the
# initial download happens, redirect TMPDIR for this call so we
# can clean it up nicely
local temp_tmp
temp_tmp=$(mktemp -d)
TMPDIR=${temp_tmp} yumdownloader \
--releasever=$DIB_RELEASE \
--setopt=reposdir=$TMP_HOOKS_PATH/yum.repos.d \
--destdir=$WORKING \
${packages} && rc=$? || rc=$?
rm -rf ${temp_tmp}
if [[ ${rc} != 0 ]]; then
die "Failed to download initial packages: ${packages}"
fi
# --nodeps works around these wanting /bin/sh in some fedora
# releases, see rhbz#1265873
sudo $_RPM --root $TARGET_ROOT --nodeps -ivh $WORKING/*rpm
}
# _install_pkg_manager packages...
#
# install the package manager packages. This is done outside the chroot
# and with yum from the build system.
# TODO: one day build systems will be dnf only, but we don't handle
# that right now
function _install_pkg_manager {
# Install into the chroot, using the gpg keys from the release
# rpm's installed in the chroot
sudo sed -i "s,/etc/pki/rpm-gpg,$TARGET_ROOT/etc/pki/rpm-gpg,g" \
$TARGET_ROOT/etc/yum.repos.d/*repo
# See notes on $_RPM variable -- we need to override the
# $HOME-based dbpath set on debian/ubuntu here. Unfortunately,
# yum does not have a way to override rpm macros from the command
# line. So we modify the user's ~/.rpmmacros to set %_dbpath back
# to "/var/lib/rpm" (note, this is taken relative to the
# --installroot).
#
# Also note, we only want this done around this call -- this is
# the only place we are using yum outside the chroot, and hence
# picking up the base-system's default rpm macros. For example,
# the yumdownloader calls above in _install_repos want to use
# ~/.rpmdb/ ... there is nothing in the build-system /var/lib/rpm!
#
# Another issue we hit is having to set --releasedir here. yum
# determines $releasevar based on (more or less) "rpm -q
# --whatprovides $distroverpkg". By default, this is
# "redhat-release" (fedora-release provides redhat-release) but
# some platforms like CentOS override it in /etc/yum.conf (to
# centos-release in their case). You can't override this (see
# [1]), but setting --releasever works around this.
#
# [1] https://bugzilla.redhat.com/show_bug.cgi?id=1287333
(
flock -w 1200 9 || die "Can not lock .rpmmacros"
echo "%_dbpath /var/lib/rpm" >> $HOME/.rpmmacros
_lang_pack=""
if [ $DISTRO_NAME = "fedora" -a $DIB_RELEASE -le 23 ]; then
# _install_langs is a rpm macro that limits the translation
# files, etc installed by packages. For Fedora 23 [1], the
# glibc-common package will obey this to only install the
# listed locales, keeping things much smaller (we still have
# to clean up locales manually on centos7). We install just
# en_US because people often ssh in with that locale, but
# leave out everything else. Note that yum has an option to
# set this from the command-line [2], but the yum in trusty we
# are using is too old to have it. So we set it directly in
# the macros file
#
# [1] http://pkgs.fedoraproject.org/cgit/rpms/glibc.git/commit/glibc.spec?h=f23&id=91764bd9ec690d4b8a886c0a3a104aac12d340d2
# [2] http://yum.baseurl.org/gitweb?p=yum.git;a=commit;h=26128173b362474456e8f0642073ecb0322ed031
echo "%_install_langs C:en_US:en_US.UTF-8" >> $HOME/.rpmmacros
elif [ $DISTRO_NAME = "fedora" -a $DIB_RELEASE -ge 24 ]; then
# glibc on F24 has split locales into "langpack" packages.
# Yum doesn't understand the weak-dependencies glibc now
# uses to get the minimal-langpack and chooses a
# random(ish) one that satisfies the locale dependency
# (rhbz#1349258). Work-around this by explicitly requring
# the minimal and english (for en_US.UTF-8) pack.
_lang_pack="glibc-minimal-langpack glibc-langpack-en"
fi
sudo -E yum -y \
--disableexcludes=all \
--setopt=cachedir=$YUM_CACHE/$ARCH/$DIB_RELEASE \
--setopt=reposdir=$TARGET_ROOT/etc/yum.repos.d \
--releasever=$DIB_RELEASE \
--installroot $TARGET_ROOT \
install $@ ${_lang_pack} && rc=$? || rc=$?
# We modified the base system - make sure we clean up always!
rm $HOME/.rpmmacros.dib.lock
# sed makes it easy to remove last line, but not last n lines...
sed -i '$ d' $HOME/.rpmmacros; sed -i '$ d' $HOME/.rpmmacros;
if [ $rc != 0 ]; then
die "Initial yum install to chroot failed! Can not continue."
fi
) 9>$HOME/.rpmmacros.dib.lock
# Set gpg path back because subsequent actions will take place in
# the chroot
sudo sed -i "s,$TARGET_ROOT/etc/pki/rpm-gpg,/etc/pki/rpm-gpg,g" \
$TARGET_ROOT/etc/yum.repos.d/*repo
}
if [ -n "$DIB_OFFLINE" -o -n "${DIB_YUMCHROOT_USE_CACHE:-}" ] && [ -f $YUMCHROOT_TARBALL ] ; then
echo $YUMCHROOT_TARBALL found in cache. Using.
sudo tar -C $TARGET_ROOT --numeric-owner -xzf $YUMCHROOT_TARBALL
else
# Note this is not usually done for root.d elements (see
# lib/common-functions:mount_proc_dev_sys) but it's important that
# we have things like /dev/urandom around inside the chroot for
# the rpm [pre|post]inst scripts within the packages.
sudo mkdir -p $TARGET_ROOT/proc $TARGET_ROOT/dev $TARGET_ROOT/sys
sudo mount -t proc none $TARGET_ROOT/proc
sudo mount --bind /dev $TARGET_ROOT/dev
sudo mount --bind /dev/pts $TARGET_ROOT/dev/pts
sudo mount -t sysfs none $TARGET_ROOT/sys
# initalize rpmdb
sudo mkdir -p $TARGET_ROOT/var/lib/rpm
sudo $_RPM --root $TARGET_ROOT --initdb
# this makes sure that running yum/dnf in the chroot it can get
# out to download stuff
sudo mkdir $TARGET_ROOT/etc
sudo cp /etc/resolv.conf $TARGET_ROOT/etc/resolv.conf
# Bind mount the external yum cache inside the chroot. Same logic
# as in the yum element to provide for yum caching copied here
# because the sequencing is wrong otherwise
sudo mkdir -p $TMP_MOUNT_PATH/tmp/yum
sudo mount --bind $YUM_CACHE $TMP_MOUNT_PATH/tmp/yum
_install_repos
# install dnf for >= f22
if [ $DIB_RELEASE -ge 22 ]; then
_install_pkg_manager dnf dnf-plugins-core yum
else
_install_pkg_manager yum
fi
# we just installed yum/dnf with "outside" tools (yum/rpm) which
# might have created /var/lib/[yum|rpm] (etc) that are slighlty
# incompatible. Refresh everything with the in-chroot tools
sudo -E chroot $TARGET_ROOT rpm --rebuilddb
sudo -E chroot $TARGET_ROOT ${YUM} clean all
# populate the lang reduction macro in the chroot
echo "%_install_langs C:en_US:en_US.UTF-8" | \
sudo tee -a $TARGET_ROOT/etc/rpm/macros.langs > /dev/null
# bootstrap the environment within the chroot; bring in new
# metadata with an update and install some base packages we need.
sudo -E chroot $TARGET_ROOT ${YUM} -y update
sudo -E chroot $TARGET_ROOT ${YUM} -y \
--setopt=cachedir=/tmp/yum/$ARCH/$DIB_RELEASE \
install systemd passwd findutils sudo util-linux-ng
# This package is split out from systemd on >F24, dracut is
# missing the dependency and will fail to make an initrd without
# it; see
# https://bugzilla.redhat.com/show_bug.cgi?id=1398505
if [ $DISTRO_NAME = "fedora" -a $DIB_RELEASE -ge 24 ]; then
sudo -E chroot $TARGET_ROOT ${YUM} -y \
--setopt=cachedir=/tmp/yum/$ARCH/$DIB_RELEASE \
install systemd-udev
fi
# Put in a dummy /etc/resolv.conf over the temporary one we used
# to bootstrap. systemd has a bug/feature [1] that it will assume
# you want systemd-networkd as the network manager and create a
# broken symlink to /run/... if the base image doesn't have one.
# This broken link confuses things like dhclient.
# [1] https://bugzilla.redhat.com/show_bug.cgi?id=1197204
echo -e "# This file intentionally left blank\n" | \
sudo tee $TARGET_ROOT/etc/resolv.conf
# set the most reliable UTF-8 locale
echo -e 'LANG="en_US.UTF-8"' | \
sudo tee $TARGET_ROOT/etc/locale.conf
# default to UTC
sudo -E chroot $TARGET_ROOT ln -sf /usr/share/zoneinfo/UTC \
/etc/localtime
# cleanup
# TODO : move this into a exit trap; and reconsider how
# this integrates with the global exit cleanup path.
sudo umount $TMP_MOUNT_PATH/tmp/yum
sudo umount $TARGET_ROOT/proc
sudo umount $TARGET_ROOT/dev/pts
sudo umount $TARGET_ROOT/dev
sudo umount $TARGET_ROOT/sys
# RPM doesn't know whether files have been changed since install
# At this point though, we know for certain that we have changed no
# config files, so anything marked .rpmnew is just a bug.
for newfile in $(sudo find $TARGET_ROOT -type f -name '*rpmnew') ; do
sudo mv $newfile $(echo $newfile | sed 's/.rpmnew$//')
done
echo Caching result in $YUMCHROOT_TARBALL
sudo tar --numeric-owner \
-C $TARGET_ROOT \
-zcf $YUMCHROOT_TARBALL --exclude='./tmp/*' .
fi
sudo rm -f ${TARGET_ROOT}/.extra_settings