By default we create a tarball of any debootstrap rootfs we create. For
the majority of use cases this is a large performance hit for no
benefit. Lets make this an opt-in feature.
Change-Id: I58fc485aacacaa17243bf9ce760ed91256d1f182
python3 is a hard requirement of dnf so can't be removed [1]
grubby is also required for kernel installs on Fedora. For too much
detail see I1a6e45d04755515286b3d49f8280c16b527e2f48; but the kernel,
via dracut, now has this as a "recommends" due to people removing it
and making unbootable systems.
[1] http://logs.openstack.org/76/248976/2/check/gate-dib-dsvm-functests-devstack-f21/734c8bd/console.html
Change-Id: I5867ecd57834eece9477aa9ea4b8bdd70e238084
The debian-minimal element creates /etc/apt/sources.list solely with
the 'main' component. I need to add 'non-free' and 'contribs'. I tried
to pass them via DIB_DEBIAN_COMPONENTS but it is not recognized.
Make debian-minimal to honor DIB_DEBIAN_COMPONENTS. Note that it is
comma separated for 'debootstrap', so replace commas with spaces to fit
the needs of sources.list.
Example usage:
DIB_DEBIAN_COMPONENTS='main,non-free,contrib'
Will debootstrap with the three components then when debian-minimal is
realized pre install a sources.list that has:
deb http://example.org/debian jessie main contrib non-free
Change-Id: I1dca2e8ffd31044a6b441ccb277298601e62f67c
Gentoo provides eclean-dist via the gentoolkit package
eclean-dist isn't needed anyway was /usr/portage is removed anyway
Removed redundant package update in cleanup.
Change-Id: Icf4f9ed549b9a6d923448d927d7c42bcf8d6091b
This action has been reordered so cleanup can occur before copy to blockdev
occurs. Documentation has been added about the ordering of this element in
relation to cleanup actions as well.
Change-Id: I3f9334a3669ee588d7fa7129202c97fa22fdb050
While it does save 1MiB of space, it might also pull python packages
depending on it. E.g. it makes impossible to install python-hardware
on the IPA image for advanced introspection.
Change-Id: Iab80dde63e6de62a5e45dcf404b4f9f633e50ac3
We were getting some subtle issues in fedora-minimal builds that
turned out to be because /var/run was not a symlink to /run.
Upon further investigation, it turns out that yum is creating a
/var/run directory for it's pid file when it starts working in the
empty chroot (which I verified by stracing it)
---
5905 stat("/home/ubuntu/tmp/dib-tmp/image.Ac4VZZsl/mnt/var/run", 0x7ffddffa0330) = -1 ENOENT (No such file or directory)
5905 mkdir("/home/ubuntu/tmp/dib-tmp/image.Ac4VZZsl/mnt/var/run", 0755) = 0
5905 open("/home/ubuntu/tmp/dib-tmp/image.Ac4VZZsl/mnt/var/run/yum.pid", O_WRONLY|O_CREAT|O_EXCL, 0644) = 6
---
Because this happens *before* we install "filesystem" (the package),
we mess up it's symlinking.
To work-around this, pre-install the trio of base packages (setup,
basesystem, filesystem) with rpm from outside the chroot.
Change-Id: I411b6ec9d91d95d3a0f98e76853086af3b70abe8
The Ubuntu Xenial cloud server images set the mode of
/var/lib/apt/lists/partial to 700, so when mounted it's inaccessible to
an unprivileged user, resulting in an error:
find: `/tmp/image.aDQKdkRi/mnt/var/lib/apt/lists/partial': Permission denied
There's no reason an image should come with anything already in
/var/lib/apt/lists/partial, so just avoid trying to descend into that
directory when fixing the apt translations packages.
Change-Id: Id27f0166bfb09d67200f337a5ffff2f2037b7c1c
This is a slight refactor I found useful when debugging. The
udevadmin info query will be visbile in the output of "set -x" which
helps, and is the logs/journal.
We can also reduce some calls by keeping the value and just grepping
with a herefile.
This also does some error checking and bails out if it does not see
what it needs to continue.
Change-Id: I39c4d262f9c5ce53f6b83d95b1363a74834cf2c8
Tucked away in systemd-udev-settle.service is the following comment
# This service can dynamically be pulled-in by legacy services which
# cannot reliably cope with dynamic device configurations, and
# wrongfully expect a populated /dev during bootup.
The info that the growroot script is querying is populated via udev,
particularly the blkid bits of [1]. This creates a race-condition
where sometimes udev has been triggered and the rules have applied and
sometimes not. Obviously in the first case, the root disk is not
grown correctly.
systemd-udev-settle is mostly disabled on distros because it can cause
an increase in boot-time for systems with lots of disks; this is not
our situation so it makes basically no difference.
That said, I will investigate if some systemd people know even better
ways to do this (possibly the service should depend on block .device
targets in systemd, and then filter out and only apply to the root
disk?)
[1] https://github.com/systemd/systemd/blob/master/rules/60-persistent-storage.rules#L66
Change-Id: I453e3afcd953dfc29ab6c42ddc81e940cfa70ee0
A TODO was placed on the partitioning section of the vm element to
replace sfdisk with a saner (and less arcane) way of partitioning. It
suggested parted for replacement. This changeset should reproduce the
same disk label and partition layout as sfdisk, but with less ioctl
errors and version dependency. It will also ensure partition alignment.
Change-Id: I5d8d75131458b73bfb05f80f1bfa7e2970e004b3
We currently install pip from package in the simple-init element.
We should really allow users to select whether to install pip from
git or package.
Change-Id: Ia5e62b9635af90d81227274a1dd8f20474cdbf73
As described in the comment, there is a dnf equivalent of this command
that doesn't require us installing yum-utils (which drags in yum on
dnf-only systems such as f23)
This is a small consequence to this -- due to us not installing
yum-utils some installs will now be completely yum free. This causes
a breakage in ironic-agent 99-remove-extra-packages where we remove
the yum package. There is a long-standing bug/feature where missing
packages in a group of packages do not cause yum/dnf to exit with
failure, but uninstalling a single package will. Because we have made
the systems yum-free, the uninstall of yum can fail in this corner
case.
It has always been like this, so I'm in favour of the "ain't broke"
approach. To work-around this, I have just put yum into the existing
list of packages to be cleaned up. I have added a note to the yum
installer taking note of this behaviour for future reference.
Change-Id: I8bbdc07ccdb89a105b4fc70d5a215077c42fcd03
InfiniBand interface takes more time to bring up then
Ethernet interface. This patch just increase the retries
to 20 times, to make it work for InfiniBand as well.
Change-Id: I5c4842696207885552413ea2d053f2e90bd6803c
Adds a post-install function that enables installed initscripts,
as that is not done by default in gentoo.
Change-Id: I04e8d506ddcbefa8a983dd31ad16df5e13cb26e7
Closes-Bug: 1539276
This checks the profile, if it has hardened in it's name it needs xattr support
unfortunately xattr support cannot yet be relied on everywhere, so it needs to
be disabled for hardened profile builds to correctly pax-mark.
Change-Id: I7fb855249a9e6c9b6497ab5061b4ea3c014f5081
Closes-Bug: 1537177
Due to upstream bug [1] there are uninstallable packages which mean
our functional tests don't work. We will revert this when things are
working upstream.
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1303660
Change-Id: I93c2990472e88ab3e5ff14db56b4ff1b4dd965ef
subprocess.check_call() returns a byte-string which needs to be turned
into a unicode string for python3 compatability.
Also some minor refactoring while we're here.
Closes-Bug: 1536462
Change-Id: Icd957bc4d93ccad94b1246ad62e6e02ee14d9ca5
Add missing growroot initscript and pkg-map entries for Gentoo.
growpart was added to Gentoo with [1]
Update the readme to reflect reality too (fedora added with
I5630dc638f85b1e80795826ef36a306632075460)
[1] https://packages.gentoo.org/packages/sys-fs/growpart
Closes-Bug: #1539273
Change-Id: I29056c7297489ec04f37757dbe33976901eceb49
As mentioned in package-installs.yaml, git is a transitive dependency
for pbr in this element. Add pkg-map for the Gentoo package.
Change-Id: I7f2fe1663152ea66b941594e86f1da93ddd21677
Closes-Bug: 1539278
Our dib-lint checking is only considering scripts with #!/bin/bash.
While there's nothing really wrong with some other shebang line like
"#!/usr/bin/env bash" let's keep things consistent.
We can use the same regex match to reduce a few forks in the main
checking.
Also a minor cleanup to the file matching
Change-Id: I609721b2671e704ea26075dad7e5b39a8b858f6b
'locales' package gets installed before '12-debian-locale-gen' is executed
and generates effectively empty /etc/locales.gen in debian, which makes
dpkg-reconfigure call to locales ignore the values set by
debconf-set-selections.
* Remove /etc/locale.gen generated by 'locales' installation to ensure
proper locales generation on debian images
* Remove 'locales-all' package installation from debian element since
it's not needed anymore to build the image and cosnumes additional
~120MB of space
* Remove unused 'package-installs' dependency from debian-minimal
element
Change-Id: Ic39ba2b5ceb5018efb75742547b2babf80827e56
Closes-Bug: #1452400
Add systemd/fedora support to growroot element. This involves
installing the correct packages, shipping the systemd service file and
ensuring it is enabled.
Note the required growfs/resize packages for Ubuntu/Debian are
installed in other places. This is probably a bug in that path, but I
have not addressed that here.
I have tested this with a F23 build with all openstack-infra elements,
uploaded to RAX, and it boots and resizes the main file-system.
Change-Id: I5630dc638f85b1e80795826ef36a306632075460
The undercloud actually has dib run twice on it - once to create
the instack image, and again when we run instack itself. The
first run creates the dib-python symlink, and the second blows up
because the link already exists. Force the link creation so the
script is idempotent.
Change-Id: I78f9e6f5afcf8ebe6d7911a7a434525ba7c737cf
This is to aid with Fedora packaging, since rpmlint complains about
including empty files.
Change-Id: I4ad867cd21304880a571e46805ab56044542400c
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
The bootloader element lacks the mapping for grub-pc and extlinux
This adds it.
Change-Id: Ic7b82903f02faaab143d2bd682876bf4853fd90d
Closes-Bug: 1534387
While the patch looks sane and the change worked locally,
it has broke the ironic-inspector gate. As we're close
to deprecating the DIB ramdisk in favor of IPA, I suggest
reverting it.
This reverts commit 802f14862c.
Change-Id: I0525e545cb2fe8ce184312a2f9bbe3763904f61a
Closes-Bug: #1534648
As you can see in the CI failures for
Ib11b9df84b593ab25232729a570c812f1b4b8774, you can not see what file
is causing the problems when the parser fails. Add a print, but raise
the error as it contains the cause.
Change-Id: I127ff7d57d2d898969195464c6e774d496e872e6
Our bootloader install fails on non-gentoo builds due to missing pkg-map
for grub-pc. This map should really live in the bootloader element, so
move it there and fill it out.
Change-Id: Ib11b9df84b593ab25232729a570c812f1b4b8774
uses upstream's stage4 images, includes all the needed bells and
whistles for openstack on kvm.
Change-Id: Ibca43173c30c2a74a73a2e2d9dd6d6d832c62694
Closes-Bug: 1530911
Hardcoding subscription-manager to use rhel-7-server-rpms causes
users building rhel6 with Satellite6 to fail. This setting cannot
be overridden with environment variables, therefore needs to be
smarter. Setting RHEL_MAJ_VER in the rhel/rhel7 environment fixes.
Change-Id: Ifbd88bc76ef8b38a739272ba6e045a12849d68df
Closes-Bug: 1404364
This patch is fixing a syntax error in the 70-ironic-root-device init
script for the deploy-ironic element.
Change-Id: I767486ca5893605720fba41bee3af72725a26377
Closes-Bug: #1531835
This element allows installation of pip and virtualenv from either
distro packages or git.
Change-Id: Id294f0936c8fef8a3b27a415bfcc93b3f327e104
Depends-On: I731cc8a0f5bfeda8f17a78c33b9f44062323a361
Use dib-python to run package-installs using the provided python
version. Automatically detect the python version for our
package-installs-squash since that runs outside the chroot.
Change-Id: I926022bcf8cbcd81b051026ffd5d6477650045ad
Fedora has changed the location of epel, shorting the link
from 'download.fedoraproject.org' to 'dl.fedoraproject.org'.
This change updates the epel mirror to prevent it from timing
out.
Change-Id: I87090282a2f5f757495daec6ad14123b436b1aa0
This fix uses dmidecode and awk to simply multiply by 1024 when
the value is represented in GB, otherwise it returns the given
value. I should note that I've only observered this occurence
on "some" SuperMicro Hardware
Closes-Bug: #1486689
Change-Id: I352b1891326f72af3a56c7bbe8b7f3c422169404
Install selinux policy packages as part of the base-installs. selinux
is part of the base-system and the kernel boots by default in selinux
mode.
Without both of these, we can get in a situation where later scripts
(particuarly, some of the infra scripts) might install systemd-policy
without a base policy (targeted), leading to a messed up situation
where systemd will halt during boot due to missing policy files.
Change-Id: I6bf156304d1134fb328fba9b12dc364701b13696
Add an environment variable to control the creation of eth0/1
interface enablement scripts.
With a tool such as glean, the presence of these scripts will indicate
the interface is configured and configuration-drive settings will not
be applied. This means in a non-dhcp situation like on Rackspace,
network is broken.
On Fedora, where later systemd provides "predictable network interface
names" [1] eth0 & eth1 ironically aren't predictable so this just
confuses things. You really need cloud-init or glean or something to
bring up your interfaces in a sane fashion.
This maintains the status-quo on centos-minimal, but disables creation
for fedora-minimal.
[1] http://www.freedesktop.org/wiki/Software/systemd/PredictableNetworkInterfaceNames/
Change-Id: I3f1ffeb6de3b1f952292a144efab9554f7f99a5f
As described in the comment, systemd will create a broken
/etc/resolv.conf link if there is no file in the base-image (as you
can read in the bug, it is debated if this is a bug or a feature).
The solution is to leave a dummy /etc/resolv.conf file in the image.
Whatever network manager you choose (NetworkManager, glean,
cloud-config, etc) will overwrite this anyway.
It's just that some tools, such as dhclient, get confused with the
broken symlink. This affects you if you're using glean to configure
the network in a DHCP situation, for example -- dhclient won't
configure nameservers and everything goes to heck.
Change-Id: I734834d03e7fdb13f9ab2e86f877b07bf4a84ff9
We are incorrectly detecting major/minor device numbers for the growroot
rootfs. This can also be simplified by querying udev for partition
information.
Change-Id: I68059bf11f2563872f6b4d0e23fa09a15de980a8
The detection logic in pkg-map for DIB_DEBUG_TRACE assumes that this
variable being unset means tracing is on, when in fact this means
tracing is off.
Change-Id: I584a634c57bbe03e26a6ee94cef473e634616885
In order to add more flexibility to the vm and bootloader
elements, split the functionality in two different ones, and
make vm depend on bootloader element.
This will allow to construct more elements that depend on
bootloader, and develop both elements independently.
Change-Id: Iad2503b7b8fe53b768a3bc79e4cb839700fbd747
This element enables creation of Ubuntu deploy ramdisk and
user images which could be used to deploy the HP Proliant
Servers with Dynamic Smart Array Controllers. Without this driver
the disk with the Dynamic Smart Array Controller is
not visible to the ramdisk.
Closes bug: #1492803
Change-Id: Ibb3b298cd379cd7333279484df6ae30e9d7f6aaa
Creating an element which we can use in #! lines to refer to either
python2 or python3 depending on what it available.
Change-Id: Ic47e18ad21c33ab9f0d11c04260a33725aeee814
