diskimage-builder/diskimage_builder/elements/pip-and-virtualenv/install.d/pip-and-virtualenv-source-install/04-install-pip

188 lines
6.9 KiB
Plaintext
Raw Normal View History

#!/bin/bash
if [ ${DIB_DEBUG_TRACE:-0} -gt 0 ]; then
set -x
fi
set -eu
set -o pipefail
Remove centos and rhel elements Several people have popped up in IRC recently with failures in these elements. Without Python 2.7 available in the image they are unsupported (OpenStack hasn't supported it for a long time). Remove these to avoid further confusion. The centos/centos7 DISTRO split that has happened with centos-minimal is unfortunate but I don't think it helps to rename centos7/rhel7 ATM. To summarise; DISTRO=centos7 means image based build, DISTRO=centos && DIB_RELEASE=7 means the minimal build. In the future, I think it is important that the minimal builds and image builds set the same DISTRO. This reflects that "upper" layers shouldn't care about the exact building of the lower layers. I see CentOS 8 going one of two ways 1) the changes are so significant, we start separate centos8 / centos8-minimal elements. They both set DISTRO=centos8 (and DIB_RELEASE to point-release maybe?). This means we have to update all "if DISTRO == centos || DISTRO == centos7" branches to also check for "centos8". Evenually (!) "centos" goes away for versioned DISTRO only 2) we restore centos element with DISTRO=centos and DIB_RELEASE=8, and centos-minimal remains the same. This means we have to audit all "if DISTRO == centos" calls to make sure they're appropriate for version 8 (stick a "&& DIB_RELEASE=7" on them all basically). I'm not sure we can fully decide until we start to see excatly how the distro switching/matching bits look, but (2) is consistent with Ubuntu and probably the preferred solution. Some "rhel" parts have been cleaned up. More could be done in rhel-common, but given our lack of coverage of that I'd prefer to leave it for now. Change-Id: I6ea784116ef59ca22878c8512c963f29c815a00a
2017-06-28 00:55:53 +00:00
if [[ $DISTRO_NAME =~ (opensuse|fedora|centos|centos7|rhel7) ]]; then
# Default packages
_do_py3=0
_extra_repo=''
case "$DISTRO_NAME" in
centos*|rhel7)
# note python2-pip in epel
_extra_repo="--enablerepo=epel"
packages="python-virtualenv python2-pip"
if [[ "$(rpm -q --qf '[%{obsoletes}\n]' python2-setuptools)" == "python-setuptools" ]]; then
# If OpenStack release is installed, then python-setuptools is
# obsoleted by python2-setuptools
packages+=" python2-setuptools"
else
packages+=" python-setuptools"
fi
;;
fedora)
_do_py3=1
packages="python2-virtualenv python2-pip python2-setuptools"
packages+=" python3-virtualenv python3-pip python3-setuptools"
;;
opensuse)
case "$DIB_RELEASE" in
42*)
packages="python-virtualenv python-pip python-setuptools"
;;
tumbleweed|15*)
# XXX: python3?
packages="python2-virtualenv python2-pip python2-setuptools"
;;
esac
esac
# force things to happen so our assumptions hold
pip_args="-U --force-reinstall"
# GENERAL WARNING : mixing packaged python libraries with
# pip-installed versions always creates issues. Upstream
# openstack-infra uses this a lot (especially devstack) but be
# warned: here be dragons :)
# Firstly we want to install the system packages. Otherwise later
# on somebody does a "yum install python-virtualenv" and goes and
# overwrites the pip installed version with the packaged version,
# leading to all sorts of weird version issues.
if [[ $DISTRO_NAME = opensuse ]]; then
zypper -n install $packages
else
${YUM:-yum} ${_extra_repo} install -y $packages
fi
# pip10 (unlike earlier versions) will not uninstall distutils
# installed packages (note this is only a subset of packages that
# don't use setuptools for various reasons). We give it a little
# help by clearing out the files from the packages we are about to
# re-install so pip doesn't think anything is installed. See:
# https://github.com/pypa/pip/issues/4805
for pkg in $packages; do
rpm -ql $pkg | xargs rm -rf
done
# install the latest python2 pip; this overwrites packaged pip
python /tmp/get-pip.py ${pip_args}
# Install latest setuptools; there is a slight chicken-egg issue in
# that pip requires setuptools for some operations like building a
# wheel. But this simple install should be fine.
pip install ${pip_args} setuptools
if [[ $_do_py3 -eq 1 ]]; then
# Repeat above for python3
# python2 on fedora always installs into /usr/bin. Move pip2
# binary out, as we want "pip" in the final image to be
# python2 for historical reasons.
mv /usr/bin/pip /usr/bin/pip2
# You would think that installing python3 bits first, then
# python2 would work -- alas get-pip.py doesn't seem to leave
# python3 alone:
# https://github.com/pypa/pip/issues/4435
python3 /tmp/get-pip.py ${pip_args}
pip3 install ${pip_args} setuptools
# on < 27, this installed pip3 to /usr/bin/pip. On >=27 it's
# /usr/local/bin/pip. reclaim /usr/bin/pip back to pip2 and
# remove the /usr/local/bin/pip (i.e. python3 version) if it
# exists, so that "pip" calls pip2 always. if we want pip3 we
# call it explicitly.
ln -sf /usr/bin/pip2 /usr/bin/pip
rm -f /usr/local/bin/pip
fi
# now install latest virtualenv. it vendors stuff it needs so
# doesn't have issues with other system packages.
# python[2|3]-virtualenv package has installed versioned scripts
# (/usr/bin/virtualenv-[2|3]) but upstream does not! (see [2]).
# For consistency, reinstall so we're just left with python2's
# version. Note this is a rather moot point, the usual way we get
# a python3 environment is to call "virtualenv -p python3 foo" and
# that works to create a python3 virtualenv, even if using
# python2's version. Thus we probably don't *really* need to
# "pip3 install virtualenv". What we don't want is "virtualenv
# foo" creating a python3 virtualenv by default, because that
# confuses a lot of legacy code.
#
#[2] http://pkgs.fedoraproject.org/cgit/rpms/python-virtualenv.git/tree/python-virtualenv.spec#n116)
pip install ${pip_args} virtualenv
mv /usr/bin/virtualenv /usr/bin/virtualenv2
if [[ $_do_py3 -eq 1 ]]; then
pip3 install ${pip_args} virtualenv
fi
# Reclaim virtualenv to virtualenv2; similar to above, on fedora
# >27 the pip3 version has gone into /usr/local/bin; remove it so
# only /usr/bin/virtualenv exists
ln -sf /usr/bin/virtualenv2 /usr/bin/virtualenv
rm -f /usr/local/bin/virtualenv
# at this point, we should have the latest
# pip/setuptools/virtualenv packages for python2 & 3, and
# "/usr/bin/pip" and "/usr/bin/virtualenv" should be python2
# versions.
if [[ $DISTRO_NAME = opensuse ]]; then
for pkg in virtualenv pip setuptools; do
cat - >> /etc/zypp/locks <<EOF
type: package
match_type: glob
case_sensitive: on
solvable_name: python-$pkg
EOF
done
else
# Add this to exclude so that we don't install a later package
# over it if it updates. Note that fedora-minimal, bootstrapped
# via yum, can have an old yum.conf around, so look for dnf first.
if [[ -f /etc/dnf/dnf.conf ]]; then
conf=/etc/dnf/dnf.conf
elif [[ -f /etc/yum.conf ]]; then
conf=/etc/yum.conf
else
die "No conf to modify?"
fi
echo "exclude=$packages" >> ${conf}
fi
elif [[ $DISTRO_NAME = gentoo ]]; then
packages="dev-python/pip dev-python/virtualenv"
emerge -U $packages
else
# pre-install packages so dependencies are there. We will
# overwrite with latest below.
packages="python-pip python3-pip python-virtualenv"
# Unfortunately older ubuntu (trusty) doesn't have a
# python3-virtualenv package -- it seems it wasn't ready at the
# time and you had to use "python -m venv". Since then virtualenv
# has gained 3.4 support so the pip install below will work
if [[ ${DIB_PYTHON_VERSION} == 3 ]]; then
packages+=" python3-virtualenv"
fi
apt-get -y install $packages
# force things to happen so our assumptions hold
pip_args="-U --force-reinstall"
# These install into /usr/local/bin so override any packages, even
# if installed later.
python3 /tmp/get-pip.py $pip_args
python2 /tmp/get-pip.py $pip_args
pip3 install $pip_args virtualenv
pip install $pip_args virtualenv
fi