I recently built a ramdisk for IPA and was confused by
the fact that the source-repositories name did not
match the element name. (this is a convention,
confusing when they don't match but certainly not
required).
This patch makes it so you can use DIB_REPOREF_ironic_agent to
customize the IPA ramdisk sources when building ramdisks.
For backwards compat if DIB_REPOREF_agent is set it automatically
sets the new DIB_REPOREF_ironic_agent to that value as well.
Change-Id: I082d989d0d85601f5984dc7c3767b8d66a3d5438
Troubleshooting an image can be quite hard, specially if you can not get
a prompt you can enter commands to find out what went wrong. By default,
the images (specially ramdisks) doesn't have any SSH key or password for
any user. Of course one could use the ``devuser`` element to generate
an image with SSH keys and user/password in the image but that would be
a massive security hole and very it's discouraged to run in production
with a ramdisk like that.
This commit is adding a new element called dynamic-login, which inserts
a helper script into the image to allow operators to inject a SSH key
and/or change the root password dynamically when it boots via parameters
in the kernel command line.
Those parameters are:
sshkey = If the operator append sshkey="$PUBLIC_SSH_KEY" to the kernel
command line on boot, the helper script will append this key to the root
user authorized_keys.
rootpwd = If the operator append rootpwd="$ENCRYPTED_PASSWORD" to the
kernel command line on boot, the helper script will set the root password
to the one specified by this option. Note that this password should be
an encrypted password.
Change-Id: I6b87a1b90163d79745f30dfacd37516051fa0aea
When the kernel gets installed on Fedora, the rpm post scripts call
"/bin/kernel-install" [1] to install it. This is a script provided by
systemd.
However, in [2], Fedora ships a patch to kernel-install that makes a
call-out to /sbin/new-kernel-pkg -- the install script provided by
grubby [3]
Without grubby installed, systemd's kernel-install script goes off and
runs dracut plugins directly [4], which eventually creates the initrd.
For reasons that are not clearly explained, the initrd will end up in
a a "machine-id" sub-directory of /boot (possibly, so you can symlink
it?). It is also called "initrd", even though it's an initramfs, for
historical reasons in dracut I think.
It is at this point that I think 99-ramdisk has been written to move
the generated initrd file back into /boot. Later on, when we build
the image, we run grub-install and it picks up the kernel and the
initrd and installs everything.
grubby's new-kernel-pkg [6] it's very similar -- it uses dracut to
make the initramfs ... but in this case it is put in /boot and is
actually called initramfs.
The subtle change that led me down this path is that dracut has been
modified to have a "Recommends" for grubby for >F22 [7]. After
discussing this change with the author, it turns out it was *always*
intended to use the grubby-based kernel install scripts for Fedora --
our builds have been incorrect in not including the package. The
author got sick of people removing the package and making unbootable
systems, hence the change.
Thus this removes the workarounds in 99-ramdisk and replace it with an
install of the grubby package. grubby's kernel install script will
put the kernel & generated initramfs in /boot, and it will be
installed correctly via the usual grub install later when we build the
disk image.
I have built F22 & F23 fedora-minimal images with this and they boot.
[1] http://pkgs.fedoraproject.org/cgit/kernel.git/tree/kernel.spec#n1832
[2] http://pkgs.fedoraproject.org/cgit/systemd.git/tree/kernel-install-grubby.patch
[3] http://linux.die.net/man/8/new-kernel-pkg
[4] https://github.com/haraldh/dracut/blob/master/50-dracut.install
[5] 81516adcb7
[6] https://github.com/rhinstaller/grubby/blob/master/new-kernel-pkg
[7] 47ff68e78b
Change-Id: I1a6e45d04755515286b3d49f8280c16b527e2f48
This package seems to be broken and isnt useful for recent releases
(where the rootfs can be resized online). Therefore this should be
optional so people can use things like the growroot element.
Change-Id: I6e3c8d095d9fc188094f3b8811f06be0847ef08c
This patch is extending the root device hints to also look at
ID_WWN_WITH_EXTENSION and ID_WWN_VENDOR_EXTENSION from udev.
Prior to this patch the bash ramdisk only cared about ID_WWN but in some
systems in some platforms with a RAID controller, this ID can be same
even if they are different disks (see bug 1516641).
Related-Bug: #1516641
Change-Id: I45b3910d03d164d880b32169b91e94e88812e183
Install-types are a user facing feature, not just for developers. Lets
move the docs on them in to the user guide.
Change-Id: I6ee8f657c270cf90da9c0729494740bb23aa47c5
On Debian/Ubuntu installs of RPM, /usr/lib/rpm/macros sets
%_dbpath %(echo $HOME/.rpmdb)
which makes quite a bit of sense, because RPM is not the system
packager and thus RPM is setup to install things into a hierarchy in
the users homedir.
However, this messes things up when building a Fedora chroot on an
Ubuntu platform.
We use RPM & yum from the base-system to bootstrap the Fedora chroot.
While both obey --root flags, they still pick up the %_dbpath macro
and so end up creating the RPM database in <chroot>/home/user/.rpmdb
After we have bootstrapped yum/dnf, we execute further installation
commands from inside the chroot -- where we now have the Fedora
version of /usr/lib/rpm/macros and hence have _dbpath set to
/var/lib/rpm -- except there is no rpm database there.
Should anyone be finding this in the future, the actual issue that
appears is
$ sudo chroot /opt/dib_tmp/image.b6B5S3f6/mnt dnf makecache
Error: Failed to synchronize cache for repo 'fedora' from \
'https://mirrors.fedoraproject.org/metalink?repo=fedora-$releasever&arch=x86_64': \
Cannot prepare internal mirrorlist: file "repomd.xml" was not found in metalink
Note the issue there is that $releasever is not expanded, because the
rpmdb where this info is kept is not populated.
The trick is to make sure we override this value when using the host
rpm/yum to setup the chroot. The bare rpm calls, which we use to
install the repos, have a --dbpath argument where we can override
this. yum does not however, so we override this in the global
~/.rpmmacros while we are installing the packaging tools and
dependencies into the chroot.
Copious comments are included, because this is super-confusing.
Change-Id: I20801150ea02d1c64f118eb969fb2aec473476f7
This patch fixes the calculation of the resultant image size
when building an image with diskimage-builder on ext4 a
filesystem.
Prior to this, using the '--image-size 2' (2GB) setting would
generate an image that would not boot under a 2GB nova flavor.
Change-Id: I7a753bdef84c6300ccea73ae4a92bf330dcd77cb
Closes-Bug: #1513622
This new element installs hpssacli utility (for configuring
RAID) and installs proliantutils python module (which has
ironic-python-agent hardware manager for HP ProLiant hardware).
This module also exposes a new environment variable DIB_HPSSACLI_URL
which allows operator to pass a custom HTTP(S) URL for RPM of hpssacli
utility.
NOTE: This module currently supports only installing from source.
Change-Id: I0494e3db623fdd7ea9182ffba21c0652aaad113c
fedora-minimal fails to build on Ubuntu Trusty due do being unable to
find the initrd (see Id4c04d7ae20068643df34d2fa31068e8a917a52d).
This is a rather obscure problem that comes from the intersection of
several things.
The first thing to note is that the post-install scripts of the
kernel-core package use kernel-install [1]. For whatever reason, this
installs the kernel to /boot/MACHINE-ID/KERNEL-VERSION
MACHINE-ID comes from /etc/machine-id; a UUID that should have been
created by the systemd post-inst scripts with systemd-machine-id-setup
[2].
The chroot environment provided for root.d elements has no kernel
file-systems like /proc or /dev mounted. This is where differences in
the base-system come into play -- on more recent systems that
implement getrandom() systemd does not need /dev/urandom to generate
the machine-id [3]; we get a value and /etc/machine-id is populated.
On older platforms (Trusty), systemd-machine-id-setup fails (unable to
access /dev/urandom) and we end up with a blank /etc/machine-id. This
ends up making kernel-install (the script) fail during yum's
installation of kernel-core, which means the initrd is not installed
correctly.
We end up bailing out in fedora-minimal/install.d/99-ramdisk, where we
try to put the installed ramdisk in /boot for the later grub install
scripts to find.
The solution here is to mount the standard kernel file-systems within
the chroot before we try installing.
[1] http://www.freedesktop.org/software/systemd/man/kernel-install.html
[2] http://www.freedesktop.org/software/systemd/man/systemd-machine-id-setup.html
[3] https://github.com/systemd/systemd/blob/master/src/basic/random-util.c
Change-Id: Ibcce35da928f64e6a719b070bcc833346ee7ee92
Clarify what this script is doing. It currently fails on some
platforms due to earlier errors, see
Ibcce35da928f64e6a719b070bcc833346ee7ee92
Change-Id: Id4c04d7ae20068643df34d2fa31068e8a917a52d
The check suffered from various flaws.
First, due to missing quotes around $initrd, 'wc -l' would always see
1 line no matter how many results the find returned.
Second, echo adds a line break making 'wc -l' count 1 even for empty
string. We need to add a check for empty string.
Change-Id: Ib2c67960f566dbdc471d9585a4cef1beb1cc38ab
Closes-Bug: #1506692
5af25b5f fixed the hostname of Debian images to "debian" since a lack of
hostname definition set the hostname to "(None)".
It has been done by introducing /etc/cloud/cloud.cfg.d/01_hostname.cfg
with content:
hostname: debian
Review supposed the hostname would be overriden by cloud meta-data. That
might have stand true for Wheezy but it is not the case for Jessie.
cloud-init 0.7.6 ignores cloud metadata whenever "hostname" or "fqdn"
are set in a config file. Roughly:
# no fqdn set, get fqdn from cloud
# get hostname from cfg if available otherwise cloud
fqdn = cloud.get_hostname(fqdn=True)
if "hostname" in cfg:
# hashar: set from config file NOT cloud
hostname = cfg['hostname']
else:
# fallback to cloud
hostname = cloud.get_hostname()
Relevant code is
https://github.com/number5/cloud-init/blob/0.7.6/cloudinit/util.py#L839-L860
Only inject "hostname: debian" for the Wheezy release.
Bug: https://phabricator.wikimedia.org/T117283
Change-Id: I6e2522bd725cbf9651f11c76ecdc72ecbc92f402
Previously all files in /root were ignored when building the
ironic-agent ramdisk. This prevented for example to use the
local-config element to connect to the ramdisk via ssh as root user.
This commit change the exclude rule on /root to only ignore the
/root/.cache directory.
Change-Id: I18d839e8d97636f5f2164ba407f252407d9bc956
Closes-Bug: #1451668
Now 'tox -efunc' can be invoked to run all functional tests in
the 'venv' tox environment. Also `tox -efunc element-name` can be
used to run function tests for one element (e.g. ironic-agent).
Change-Id: Ia685d1b2a7deef2f8b98876ac09792134dd30f2f
yum-minimal/root.d/08-yum-chroot runs before yum/root.d/50-yum-cache,
and thus if run on a completely fresh system will fail in
08-yum-chroot as the YUM_CACHE directory isn't made.
This is probably hidden by testing & nodepool builds, because it sets
DIB_IMAGE_CACHE. It was hidden from me because locally I have done
builds using the "yum" element previously, which had created the
cache.
Change-Id: I333f5f7e67d198f75a522cc296c118c2e94a5ecb
download.fedoraproject.org uses dns round robin and occasionally
hits a bad server. Using DIB_EPEL_MIRROR when finding the
epel-release package will allow us to avoid it e.g. in ci.
Change-Id: I756223b3e669532476663c05e79c238449b8a0db
Xen paravirtualised disks (supported by most modern kernels) have the
"xvd" prefix (e.g. xvda0). The functions to strip partitions need to
match on Xen PV disks otherwise the device name is discarded.
Change-Id: I5539d2afba3fae30d1ddb49dcbf077113d38bbf7
Closes-Bug: #1498576
Grub fails to install the bootloader due to it being on the root
partition of a block device. This is not actually a problem for us, so
we need to force it to succeed.
Change-Id: I335ef04ca8a8a8a5c242d3444b09bcce0a9f51e7
