This patch add the lshw utils to allow the agent
to retrive the vendor name. It also keeps the
/usr/share/misc/pci.ids file for Ubuntu distro.
Change-Id: I2828a7e585449f63887ad5a2e338b4e043d68e63
I have seem some occasional odd failures coming from the "dnf -y
update" done by elements/base/install.d/00-up-to-date.
dnf seems to sometimes think a package is not installed when it really
is. It then seems to try and re-install them, but notices they are
installed, and then bails with a failure exit [1]. The packages that
seem to cause this vary, but the common thread is that they seem to
have all been installed during the initial phase of installing the
package manager in the chroot.
I suspect that when we are building the chroot, we do our initial
install with the "external" yum & rpm. Then we start using the
dnf/yum in the chroot, but we're actually using meta-data created by
the *external* tools -- which could be vastly different versions or
who-knows-what. While I honestly I don't have an exact root cause,
empirically I've found rebuilding the rpm db always seems to fix
things up.
So this change takes care to rebuild the rpm db with the chroot
version of rpm, and clear out the package metadata for a refresh with
"update". This should hopefully put us in a consistent state.
[1] http://paste.openstack.org/show/487356/
Change-Id: I565df23897ae511356c4861fdbe63823fa6b6ff9
This element is similar to vm, but allows more flexibility
on disk partitioning. It receives a DIB_PARTITIONING_SFDISK_SCHEMA
setting, allowing to partition the disk according to project needs.
Change-Id: I01dca5d5fd670d317f7761911a1549507de6e97d
If we are being run from a python 3 virtualenv where python2 is also
available on the host we will use the hosts' python2 over the virtualenv
python3 (even if the virtualenv is higher precedence in PATH).
Change-Id: I9a25b9e45845a4121aab8250fd53c6f006825742
This patch fixes the profile autodetect. If the profile was passed at install
time, it would overridden by the environment.d script. This was fixed by
accepting the value passed by the user, if one was passed.
The enviroment.d file is run outside of chroot, because of this eselect will
not be available on non-gentoo systems. We change the behavior to work
elsewhere, it was not working on non-gentoo systems and giving bad results on
gentoo systems (returing values from the host, not the guest being built).
This patch also fixes shm detection by checking if it is mounted already.
Change-Id: Ie58d8e19529a731bfbc9eeb4bb246988d1aaa772
Newer distros, such as debian jessie and ubuntu xenial, do not provide
ifupdown by default, but simple-init depends on it. Add it to the pile.
Change-Id: I6f4876863c67c65a82464d4e0593015cdc839c5c
This is breaking deployments that use registration. I
believe os-refresh-config scripts don't source environment.d
files, so the variable is unset there.
Given that this a blocking issue for RHEL deployments, I'm
pushing a quick revert and we can come up with a proper fix
when this isn't stopping other work.
This reverts commit 71bd8b3a33.
Change-Id: I87504660352220d45f5445bb933edc1c28885fcb
This patch update the find_interface to lookup
for InfiniBand interface according to it's BOOTIF
Closes-Bug: #1532534
Change-Id: I21b91cfd10888ac036f6347a0a44cdca422830a6
In certain cases, with packages cached that need an upgrade,
when performing that action the system hangs waiting for
a prompt.
Add force-confdef option, that will let dpkg overwrite
configuration packages that were not modified. In combination
with already existing force-confold flag, it will allow
to avoid any conffile prompt.
Change-Id: Ifb177f9ac2c9ad29f8b92309c5b8cfe8e60a4e14
In order to avoid conflict with installing dependencies for IPA from
pypi and distribution packages this patch propose installing IPA in a
virtual environment.
Closes-Bug: #1506792
Change-Id: I6a4c6403813d86f4110e98449ddd874109101b9e
By default we create a tarball of any debootstrap rootfs we create. For
the majority of use cases this is a large performance hit for no
benefit. Lets make this an opt-in feature.
Change-Id: I58fc485aacacaa17243bf9ce760ed91256d1f182
python3 is a hard requirement of dnf so can't be removed [1]
grubby is also required for kernel installs on Fedora. For too much
detail see I1a6e45d04755515286b3d49f8280c16b527e2f48; but the kernel,
via dracut, now has this as a "recommends" due to people removing it
and making unbootable systems.
[1] http://logs.openstack.org/76/248976/2/check/gate-dib-dsvm-functests-devstack-f21/734c8bd/console.html
Change-Id: I5867ecd57834eece9477aa9ea4b8bdd70e238084
The debian-minimal element creates /etc/apt/sources.list solely with
the 'main' component. I need to add 'non-free' and 'contribs'. I tried
to pass them via DIB_DEBIAN_COMPONENTS but it is not recognized.
Make debian-minimal to honor DIB_DEBIAN_COMPONENTS. Note that it is
comma separated for 'debootstrap', so replace commas with spaces to fit
the needs of sources.list.
Example usage:
DIB_DEBIAN_COMPONENTS='main,non-free,contrib'
Will debootstrap with the three components then when debian-minimal is
realized pre install a sources.list that has:
deb http://example.org/debian jessie main contrib non-free
Change-Id: I1dca2e8ffd31044a6b441ccb277298601e62f67c
Gentoo provides eclean-dist via the gentoolkit package
eclean-dist isn't needed anyway was /usr/portage is removed anyway
Removed redundant package update in cleanup.
Change-Id: Icf4f9ed549b9a6d923448d927d7c42bcf8d6091b
This action has been reordered so cleanup can occur before copy to blockdev
occurs. Documentation has been added about the ordering of this element in
relation to cleanup actions as well.
Change-Id: I3f9334a3669ee588d7fa7129202c97fa22fdb050
While it does save 1MiB of space, it might also pull python packages
depending on it. E.g. it makes impossible to install python-hardware
on the IPA image for advanced introspection.
Change-Id: Iab80dde63e6de62a5e45dcf404b4f9f633e50ac3
We were getting some subtle issues in fedora-minimal builds that
turned out to be because /var/run was not a symlink to /run.
Upon further investigation, it turns out that yum is creating a
/var/run directory for it's pid file when it starts working in the
empty chroot (which I verified by stracing it)
---
5905 stat("/home/ubuntu/tmp/dib-tmp/image.Ac4VZZsl/mnt/var/run", 0x7ffddffa0330) = -1 ENOENT (No such file or directory)
5905 mkdir("/home/ubuntu/tmp/dib-tmp/image.Ac4VZZsl/mnt/var/run", 0755) = 0
5905 open("/home/ubuntu/tmp/dib-tmp/image.Ac4VZZsl/mnt/var/run/yum.pid", O_WRONLY|O_CREAT|O_EXCL, 0644) = 6
---
Because this happens *before* we install "filesystem" (the package),
we mess up it's symlinking.
To work-around this, pre-install the trio of base packages (setup,
basesystem, filesystem) with rpm from outside the chroot.
Change-Id: I411b6ec9d91d95d3a0f98e76853086af3b70abe8
The Ubuntu Xenial cloud server images set the mode of
/var/lib/apt/lists/partial to 700, so when mounted it's inaccessible to
an unprivileged user, resulting in an error:
find: `/tmp/image.aDQKdkRi/mnt/var/lib/apt/lists/partial': Permission denied
There's no reason an image should come with anything already in
/var/lib/apt/lists/partial, so just avoid trying to descend into that
directory when fixing the apt translations packages.
Change-Id: Id27f0166bfb09d67200f337a5ffff2f2037b7c1c
This is a slight refactor I found useful when debugging. The
udevadmin info query will be visbile in the output of "set -x" which
helps, and is the logs/journal.
We can also reduce some calls by keeping the value and just grepping
with a herefile.
This also does some error checking and bails out if it does not see
what it needs to continue.
Change-Id: I39c4d262f9c5ce53f6b83d95b1363a74834cf2c8
Tucked away in systemd-udev-settle.service is the following comment
# This service can dynamically be pulled-in by legacy services which
# cannot reliably cope with dynamic device configurations, and
# wrongfully expect a populated /dev during bootup.
The info that the growroot script is querying is populated via udev,
particularly the blkid bits of [1]. This creates a race-condition
where sometimes udev has been triggered and the rules have applied and
sometimes not. Obviously in the first case, the root disk is not
grown correctly.
systemd-udev-settle is mostly disabled on distros because it can cause
an increase in boot-time for systems with lots of disks; this is not
our situation so it makes basically no difference.
That said, I will investigate if some systemd people know even better
ways to do this (possibly the service should depend on block .device
targets in systemd, and then filter out and only apply to the root
disk?)
[1] https://github.com/systemd/systemd/blob/master/rules/60-persistent-storage.rules#L66
Change-Id: I453e3afcd953dfc29ab6c42ddc81e940cfa70ee0
A TODO was placed on the partitioning section of the vm element to
replace sfdisk with a saner (and less arcane) way of partitioning. It
suggested parted for replacement. This changeset should reproduce the
same disk label and partition layout as sfdisk, but with less ioctl
errors and version dependency. It will also ensure partition alignment.
Change-Id: I5d8d75131458b73bfb05f80f1bfa7e2970e004b3
We currently install pip from package in the simple-init element.
We should really allow users to select whether to install pip from
git or package.
Change-Id: Ia5e62b9635af90d81227274a1dd8f20474cdbf73
As described in the comment, there is a dnf equivalent of this command
that doesn't require us installing yum-utils (which drags in yum on
dnf-only systems such as f23)
This is a small consequence to this -- due to us not installing
yum-utils some installs will now be completely yum free. This causes
a breakage in ironic-agent 99-remove-extra-packages where we remove
the yum package. There is a long-standing bug/feature where missing
packages in a group of packages do not cause yum/dnf to exit with
failure, but uninstalling a single package will. Because we have made
the systems yum-free, the uninstall of yum can fail in this corner
case.
It has always been like this, so I'm in favour of the "ain't broke"
approach. To work-around this, I have just put yum into the existing
list of packages to be cleaned up. I have added a note to the yum
installer taking note of this behaviour for future reference.
Change-Id: I8bbdc07ccdb89a105b4fc70d5a215077c42fcd03
InfiniBand interface takes more time to bring up then
Ethernet interface. This patch just increase the retries
to 20 times, to make it work for InfiniBand as well.
Change-Id: I5c4842696207885552413ea2d053f2e90bd6803c
Adds a post-install function that enables installed initscripts,
as that is not done by default in gentoo.
Change-Id: I04e8d506ddcbefa8a983dd31ad16df5e13cb26e7
Closes-Bug: 1539276
This checks the profile, if it has hardened in it's name it needs xattr support
unfortunately xattr support cannot yet be relied on everywhere, so it needs to
be disabled for hardened profile builds to correctly pax-mark.
Change-Id: I7fb855249a9e6c9b6497ab5061b4ea3c014f5081
Closes-Bug: 1537177
Due to upstream bug [1] there are uninstallable packages which mean
our functional tests don't work. We will revert this when things are
working upstream.
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1303660
Change-Id: I93c2990472e88ab3e5ff14db56b4ff1b4dd965ef
subprocess.check_call() returns a byte-string which needs to be turned
into a unicode string for python3 compatability.
Also some minor refactoring while we're here.
Closes-Bug: 1536462
Change-Id: Icd957bc4d93ccad94b1246ad62e6e02ee14d9ca5
Add missing growroot initscript and pkg-map entries for Gentoo.
growpart was added to Gentoo with [1]
Update the readme to reflect reality too (fedora added with
I5630dc638f85b1e80795826ef36a306632075460)
[1] https://packages.gentoo.org/packages/sys-fs/growpart
Closes-Bug: #1539273
Change-Id: I29056c7297489ec04f37757dbe33976901eceb49
As mentioned in package-installs.yaml, git is a transitive dependency
for pbr in this element. Add pkg-map for the Gentoo package.
Change-Id: I7f2fe1663152ea66b941594e86f1da93ddd21677
Closes-Bug: 1539278
Our dib-lint checking is only considering scripts with #!/bin/bash.
While there's nothing really wrong with some other shebang line like
"#!/usr/bin/env bash" let's keep things consistent.
We can use the same regex match to reduce a few forks in the main
checking.
Also a minor cleanup to the file matching
Change-Id: I609721b2671e704ea26075dad7e5b39a8b858f6b
'locales' package gets installed before '12-debian-locale-gen' is executed
and generates effectively empty /etc/locales.gen in debian, which makes
dpkg-reconfigure call to locales ignore the values set by
debconf-set-selections.
* Remove /etc/locale.gen generated by 'locales' installation to ensure
proper locales generation on debian images
* Remove 'locales-all' package installation from debian element since
it's not needed anymore to build the image and cosnumes additional
~120MB of space
* Remove unused 'package-installs' dependency from debian-minimal
element
Change-Id: Ic39ba2b5ceb5018efb75742547b2babf80827e56
Closes-Bug: #1452400
Add systemd/fedora support to growroot element. This involves
installing the correct packages, shipping the systemd service file and
ensuring it is enabled.
Note the required growfs/resize packages for Ubuntu/Debian are
installed in other places. This is probably a bug in that path, but I
have not addressed that here.
I have tested this with a F23 build with all openstack-infra elements,
uploaded to RAX, and it boots and resizes the main file-system.
Change-Id: I5630dc638f85b1e80795826ef36a306632075460
The undercloud actually has dib run twice on it - once to create
the instack image, and again when we run instack itself. The
first run creates the dib-python symlink, and the second blows up
because the link already exists. Force the link creation so the
script is idempotent.
Change-Id: I78f9e6f5afcf8ebe6d7911a7a434525ba7c737cf
This is to aid with Fedora packaging, since rpmlint complains about
including empty files.
Change-Id: I4ad867cd21304880a571e46805ab56044542400c
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
The bootloader element lacks the mapping for grub-pc and extlinux
This adds it.
Change-Id: Ic7b82903f02faaab143d2bd682876bf4853fd90d
Closes-Bug: 1534387
While the patch looks sane and the change worked locally,
it has broke the ironic-inspector gate. As we're close
to deprecating the DIB ramdisk in favor of IPA, I suggest
reverting it.
This reverts commit 802f14862c.
Change-Id: I0525e545cb2fe8ce184312a2f9bbe3763904f61a
Closes-Bug: #1534648
As you can see in the CI failures for
Ib11b9df84b593ab25232729a570c812f1b4b8774, you can not see what file
is causing the problems when the parser fails. Add a print, but raise
the error as it contains the cause.
Change-Id: I127ff7d57d2d898969195464c6e774d496e872e6
Our bootloader install fails on non-gentoo builds due to missing pkg-map
for grub-pc. This map should really live in the bootloader element, so
move it there and fill it out.
Change-Id: Ib11b9df84b593ab25232729a570c812f1b4b8774
uses upstream's stage4 images, includes all the needed bells and
whistles for openstack on kvm.
Change-Id: Ibca43173c30c2a74a73a2e2d9dd6d6d832c62694
Closes-Bug: 1530911
Hardcoding subscription-manager to use rhel-7-server-rpms causes
users building rhel6 with Satellite6 to fail. This setting cannot
be overridden with environment variables, therefore needs to be
smarter. Setting RHEL_MAJ_VER in the rhel/rhel7 environment fixes.
Change-Id: Ifbd88bc76ef8b38a739272ba6e045a12849d68df
Closes-Bug: 1404364
This patch is fixing a syntax error in the 70-ironic-root-device init
script for the deploy-ironic element.
Change-Id: I767486ca5893605720fba41bee3af72725a26377
Closes-Bug: #1531835
This element allows installation of pip and virtualenv from either
distro packages or git.
Change-Id: Id294f0936c8fef8a3b27a415bfcc93b3f327e104
Depends-On: I731cc8a0f5bfeda8f17a78c33b9f44062323a361
Use dib-python to run package-installs using the provided python
version. Automatically detect the python version for our
package-installs-squash since that runs outside the chroot.
Change-Id: I926022bcf8cbcd81b051026ffd5d6477650045ad
Fedora has changed the location of epel, shorting the link
from 'download.fedoraproject.org' to 'dl.fedoraproject.org'.
This change updates the epel mirror to prevent it from timing
out.
Change-Id: I87090282a2f5f757495daec6ad14123b436b1aa0
This fix uses dmidecode and awk to simply multiply by 1024 when
the value is represented in GB, otherwise it returns the given
value. I should note that I've only observered this occurence
on "some" SuperMicro Hardware
Closes-Bug: #1486689
Change-Id: I352b1891326f72af3a56c7bbe8b7f3c422169404
