sig_cloud/diskimage-builder
2
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge "Remove yum chroot caching"

Browse Source
This commit is contained in:
Jenkins 2017-01-17 01:10:41 +00:00 committed by Gerrit Code Review
commit f3d0d6e8ec
5 changed files with 106 additions and 121 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

12
elements/centos-minimal/README.rst
View File

@ -6,13 +6,9 @@ Create a minimal image based on CentOS 7.
Use of this element will require 'yum' and 'yum-utils' to be installed on Use of this element will require 'yum' and 'yum-utils' to be installed on
Ubuntu and Debian. Nothing additional is needed on Fedora or CentOS. Ubuntu and Debian. Nothing additional is needed on Fedora or CentOS.
The `DIB_OFFLINE` or more specific `DIB_YUMCHROOT_USE_CACHE` By default, ``DIB_YUM_MINIMAL_CREATE_INTERFACES`` is set to enable the
variables can be set to prefer the use of a pre-cached root filesystem creation of ``/etc/sysconfig/network-scripts/ifcfg-eth[0|1]`` scripts to
tarball. enable DHCP on the ``eth0`` & ``eth1`` interfaces. If you do not have
By default, `DIB_YUM_MINIMAL_CREATE_INTERFACES` is set to enable the
creation of `/etc/sysconfig/network-scripts/ifcfg-eth[0|1]` scripts to
enable DHCP on the `eth0` & `eth1` interfaces. If you do not have
these interfaces, or if you are using something else to setup the these interfaces, or if you are using something else to setup the
network such as cloud-init, glean or network-manager, you would want network such as cloud-init, glean or network-manager, you would want
to set this to `0`. to set this to ``0``.

10
elements/fedora-minimal/README.rst
View File

@ -11,10 +11,6 @@ Due to a bug in the released version of urlgrabber, on many systems an
installation of urlgrabber from git is required. The git repository installation of urlgrabber from git is required. The git repository
can be found here: http://yum.baseurl.org/gitweb?p=urlgrabber.git;a=summary can be found here: http://yum.baseurl.org/gitweb?p=urlgrabber.git;a=summary
The `DIB_OFFLINE` or more specific `DIB_YUMCHROOT_USE_CACHE` This element sets the ``DIB_RELEASE`` var to 'fedora'. The release of
variables can be set to prefer the use of a pre-cached root filesystem fedora to be installed can be controlled through the ``DIB_RELEASE``
tarball. variable, which defaults the latest supported release.
This element sets the `DIB_RELEASE` var to 'fedora'. The release of fedora
to be installed can be controlled through the `DIB_RELEASE` variable, which
defaults to '21'.

4
elements/yum-minimal/README.rst
View File

@ -9,10 +9,6 @@ or fedora-minimal elements to get an actual base image.
Use of this element will require 'yum' and 'yum-utils' to be installed on Use of this element will require 'yum' and 'yum-utils' to be installed on
Ubuntu and Debian. Nothing additional is needed on Fedora or CentOS. Ubuntu and Debian. Nothing additional is needed on Fedora or CentOS.
The `DIB_OFFLINE` or more specific `DIB_YUMCHROOT_USE_CACHE`
variables can be set to prefer the use of a pre-cached root filesystem
tarball.
If you wish to have DHCP networking setup for eth0 & eth1 via If you wish to have DHCP networking setup for eth0 & eth1 via
/etc/sysconfig/network-config scripts/ifcfg-eth[0|1], set the /etc/sysconfig/network-config scripts/ifcfg-eth[0|1], set the
environment variable `DIB_YUM_MINIMAL_CREATE_INTERFACES` to `1`. environment variable `DIB_YUM_MINIMAL_CREATE_INTERFACES` to `1`.

160
elements/yum-minimal/root.d/08-yum-chroot
View File

@ -31,8 +31,6 @@ if [ $ARCH = amd64 ]; then
ARCH=x86_64 ARCH=x86_64
fi fi
# Calling elements will need to set DISTRO_NAME and DIB_RELEASE # Calling elements will need to set DISTRO_NAME and DIB_RELEASE
DIB_YUMCHROOT_EXTRA_ARGS=${DIB_YUMCHROOT_EXTRA_ARGS:-}
YUMCHROOT_TARBALL=$DIB_IMAGE_CACHE/yumchroot-${DISTRO_NAME}-${DIB_RELEASE}-${ARCH}.tar.gz
# TODO Maybe deal with DIB_DISTRIBUTION_MIRROR # TODO Maybe deal with DIB_DISTRIBUTION_MIRROR
http_proxy=${http_proxy:-} http_proxy=${http_proxy:-}
YUM=${YUM:-yum} YUM=${YUM:-yum}
@ -189,107 +187,97 @@ function _install_pkg_manager {
$TARGET_ROOT/etc/yum.repos.d/*repo $TARGET_ROOT/etc/yum.repos.d/*repo
} }
if [ -n "$DIB_OFFLINE" -o -n "${DIB_YUMCHROOT_USE_CACHE:-}" ] && [ -f $YUMCHROOT_TARBALL ] ; then # Note this is not usually done for root.d elements (see
echo $YUMCHROOT_TARBALL found in cache. Using. # lib/common-functions:mount_proc_dev_sys) but it's important that
sudo tar -C $TARGET_ROOT --numeric-owner -xzf $YUMCHROOT_TARBALL # we have things like /dev/urandom around inside the chroot for
else # the rpm [pre|post]inst scripts within the packages.
# Note this is not usually done for root.d elements (see sudo mkdir -p $TARGET_ROOT/proc $TARGET_ROOT/dev $TARGET_ROOT/sys
# lib/common-functions:mount_proc_dev_sys) but it's important that sudo mount -t proc none $TARGET_ROOT/proc
# we have things like /dev/urandom around inside the chroot for sudo mount --bind /dev $TARGET_ROOT/dev
# the rpm [pre|post]inst scripts within the packages. sudo mount --bind /dev/pts $TARGET_ROOT/dev/pts
sudo mkdir -p $TARGET_ROOT/proc $TARGET_ROOT/dev $TARGET_ROOT/sys sudo mount -t sysfs none $TARGET_ROOT/sys
sudo mount -t proc none $TARGET_ROOT/proc
sudo mount --bind /dev $TARGET_ROOT/dev
sudo mount --bind /dev/pts $TARGET_ROOT/dev/pts
sudo mount -t sysfs none $TARGET_ROOT/sys
# initalize rpmdb # initalize rpmdb
sudo mkdir -p $TARGET_ROOT/var/lib/rpm sudo mkdir -p $TARGET_ROOT/var/lib/rpm
sudo $_RPM --root $TARGET_ROOT --initdb sudo $_RPM --root $TARGET_ROOT --initdb
# this makes sure that running yum/dnf in the chroot it can get # this makes sure that running yum/dnf in the chroot it can get
# out to download stuff # out to download stuff
sudo mkdir $TARGET_ROOT/etc sudo mkdir $TARGET_ROOT/etc
sudo cp /etc/resolv.conf $TARGET_ROOT/etc/resolv.conf sudo cp /etc/resolv.conf $TARGET_ROOT/etc/resolv.conf
# Bind mount the external yum cache inside the chroot. Same logic # Bind mount the external yum cache inside the chroot. Same logic
# as in the yum element to provide for yum caching copied here # as in the yum element to provide for yum caching copied here
# because the sequencing is wrong otherwise # because the sequencing is wrong otherwise
sudo mkdir -p $TMP_MOUNT_PATH/tmp/yum sudo mkdir -p $TMP_MOUNT_PATH/tmp/yum
sudo mount --bind $YUM_CACHE $TMP_MOUNT_PATH/tmp/yum sudo mount --bind $YUM_CACHE $TMP_MOUNT_PATH/tmp/yum
_install_repos _install_repos
# install dnf for >= f22 # install dnf for >= f22
if [ $DIB_RELEASE -ge 22 ]; then if [ $DIB_RELEASE -ge 22 ]; then
_install_pkg_manager dnf dnf-plugins-core yum _install_pkg_manager dnf dnf-plugins-core yum
else else
_install_pkg_manager yum _install_pkg_manager yum
fi fi
# we just installed yum/dnf with "outside" tools (yum/rpm) which # we just installed yum/dnf with "outside" tools (yum/rpm) which
# might have created /var/lib/[yum|rpm] (etc) that are slighlty # might have created /var/lib/[yum|rpm] (etc) that are slighlty
# incompatible. Refresh everything with the in-chroot tools # incompatible. Refresh everything with the in-chroot tools
sudo -E chroot $TARGET_ROOT rpm --rebuilddb sudo -E chroot $TARGET_ROOT rpm --rebuilddb
sudo -E chroot $TARGET_ROOT ${YUM} clean all sudo -E chroot $TARGET_ROOT ${YUM} clean all
# populate the lang reduction macro in the chroot # populate the lang reduction macro in the chroot
echo "%_install_langs C:en_US:en_US.UTF-8" | \ echo "%_install_langs C:en_US:en_US.UTF-8" | \
sudo tee -a $TARGET_ROOT/etc/rpm/macros.langs > /dev/null sudo tee -a $TARGET_ROOT/etc/rpm/macros.langs > /dev/null
# bootstrap the environment within the chroot; bring in new # bootstrap the environment within the chroot; bring in new
# metadata with an update and install some base packages we need. # metadata with an update and install some base packages we need.
sudo -E chroot $TARGET_ROOT ${YUM} -y update sudo -E chroot $TARGET_ROOT ${YUM} -y update
sudo -E chroot $TARGET_ROOT ${YUM} -y \ sudo -E chroot $TARGET_ROOT ${YUM} -y \
--setopt=cachedir=/tmp/yum/$ARCH/$DIB_RELEASE \ --setopt=cachedir=/tmp/yum/$ARCH/$DIB_RELEASE \
install systemd passwd findutils sudo util-linux-ng install systemd passwd findutils sudo util-linux-ng
# This package is split out from systemd on >F24, dracut is # This package is split out from systemd on >F24, dracut is
# missing the dependency and will fail to make an initrd without # missing the dependency and will fail to make an initrd without
# it; see # it; see
# https://bugzilla.redhat.com/show_bug.cgi?id=1398505 # https://bugzilla.redhat.com/show_bug.cgi?id=1398505
if [ $DISTRO_NAME = "fedora" -a $DIB_RELEASE -ge 24 ]; then if [ $DISTRO_NAME = "fedora" -a $DIB_RELEASE -ge 24 ]; then
sudo -E chroot $TARGET_ROOT ${YUM} -y \ sudo -E chroot $TARGET_ROOT ${YUM} -y \
--setopt=cachedir=/tmp/yum/$ARCH/$DIB_RELEASE \ --setopt=cachedir=/tmp/yum/$ARCH/$DIB_RELEASE \
install systemd-udev install systemd-udev
fi
# Put in a dummy /etc/resolv.conf over the temporary one we used
# to bootstrap. systemd has a bug/feature [1] that it will assume
# you want systemd-networkd as the network manager and create a
# broken symlink to /run/... if the base image doesn't have one.
# This broken link confuses things like dhclient.
# [1] https://bugzilla.redhat.com/show_bug.cgi?id=1197204
echo -e "# This file intentionally left blank\n" | \
sudo tee $TARGET_ROOT/etc/resolv.conf
# set the most reliable UTF-8 locale
echo -e 'LANG="en_US.UTF-8"' | \
sudo tee $TARGET_ROOT/etc/locale.conf
# default to UTC
sudo -E chroot $TARGET_ROOT ln -sf /usr/share/zoneinfo/UTC \
/etc/localtime
# cleanup
# TODO : move this into a exit trap; and reconsider how
# this integrates with the global exit cleanup path.
sudo umount $TMP_MOUNT_PATH/tmp/yum
sudo umount $TARGET_ROOT/proc
sudo umount $TARGET_ROOT/dev/pts
sudo umount $TARGET_ROOT/dev
sudo umount $TARGET_ROOT/sys
# RPM doesn't know whether files have been changed since install
# At this point though, we know for certain that we have changed no
# config files, so anything marked .rpmnew is just a bug.
for newfile in $(sudo find $TARGET_ROOT -type f -name '*rpmnew') ; do
sudo mv $newfile $(echo $newfile | sed 's/.rpmnew$//')
done
echo Caching result in $YUMCHROOT_TARBALL
sudo tar --numeric-owner \
-C $TARGET_ROOT \
-zcf $YUMCHROOT_TARBALL --exclude='./tmp/*' .
fi fi
# Put in a dummy /etc/resolv.conf over the temporary one we used
# to bootstrap. systemd has a bug/feature [1] that it will assume
# you want systemd-networkd as the network manager and create a
# broken symlink to /run/... if the base image doesn't have one.
# This broken link confuses things like dhclient.
# [1] https://bugzilla.redhat.com/show_bug.cgi?id=1197204
echo -e "# This file intentionally left blank\n" | \
sudo tee $TARGET_ROOT/etc/resolv.conf
# set the most reliable UTF-8 locale
echo -e 'LANG="en_US.UTF-8"' | \
sudo tee $TARGET_ROOT/etc/locale.conf
# default to UTC
sudo -E chroot $TARGET_ROOT ln -sf /usr/share/zoneinfo/UTC \
/etc/localtime
# cleanup
# TODO : move this into a exit trap; and reconsider how
# this integrates with the global exit cleanup path.
sudo umount $TMP_MOUNT_PATH/tmp/yum
sudo umount $TARGET_ROOT/proc
sudo umount $TARGET_ROOT/dev/pts
sudo umount $TARGET_ROOT/dev
sudo umount $TARGET_ROOT/sys
# RPM doesn't know whether files have been changed since install
# At this point though, we know for certain that we have changed no
# config files, so anything marked .rpmnew is just a bug.
for newfile in $(sudo find $TARGET_ROOT -type f -name '*rpmnew') ; do
sudo mv $newfile $(echo $newfile | sed 's/.rpmnew$//')
done
sudo rm -f ${TARGET_ROOT}/.extra_settings sudo rm -f ${TARGET_ROOT}/.extra_settings

9
releasenotes/notes/yum-cache-removal-148c33012515e56e.yaml Normal file
View File

@ -0,0 +1,9 @@
---
deprecations:
- The ``DIB_YUMCHROOT_USE_CACHE`` variable has been removed and the
Fedora and CentOS ``-minimal`` initial chroot will always be
created by the package manager. The default creation of a chroot
tarball is stopped for these elements. This unused option was
unsafe; there is no guarantee that the base system will not change
even between runs. Getting the package manager to reuse the cache
for the initial chroot install is future work.