fedora-minimal fails to build on Ubuntu Trusty due do being unable to
find the initrd (see Id4c04d7ae20068643df34d2fa31068e8a917a52d).
This is a rather obscure problem that comes from the intersection of
several things.
The first thing to note is that the post-install scripts of the
kernel-core package use kernel-install [1]. For whatever reason, this
installs the kernel to /boot/MACHINE-ID/KERNEL-VERSION
MACHINE-ID comes from /etc/machine-id; a UUID that should have been
created by the systemd post-inst scripts with systemd-machine-id-setup
[2].
The chroot environment provided for root.d elements has no kernel
file-systems like /proc or /dev mounted. This is where differences in
the base-system come into play -- on more recent systems that
implement getrandom() systemd does not need /dev/urandom to generate
the machine-id [3]; we get a value and /etc/machine-id is populated.
On older platforms (Trusty), systemd-machine-id-setup fails (unable to
access /dev/urandom) and we end up with a blank /etc/machine-id. This
ends up making kernel-install (the script) fail during yum's
installation of kernel-core, which means the initrd is not installed
correctly.
We end up bailing out in fedora-minimal/install.d/99-ramdisk, where we
try to put the installed ramdisk in /boot for the later grub install
scripts to find.
The solution here is to mount the standard kernel file-systems within
the chroot before we try installing.
[1] http://www.freedesktop.org/software/systemd/man/kernel-install.html
[2] http://www.freedesktop.org/software/systemd/man/systemd-machine-id-setup.html
[3] https://github.com/systemd/systemd/blob/master/src/basic/random-util.c
Change-Id: Ibcce35da928f64e6a719b070bcc833346ee7ee92
Clarify what this script is doing. It currently fails on some
platforms due to earlier errors, see
Ibcce35da928f64e6a719b070bcc833346ee7ee92
Change-Id: Id4c04d7ae20068643df34d2fa31068e8a917a52d
The check suffered from various flaws.
First, due to missing quotes around $initrd, 'wc -l' would always see
1 line no matter how many results the find returned.
Second, echo adds a line break making 'wc -l' count 1 even for empty
string. We need to add a check for empty string.
Change-Id: Ib2c67960f566dbdc471d9585a4cef1beb1cc38ab
Closes-Bug: #1506692
5af25b5f fixed the hostname of Debian images to "debian" since a lack of
hostname definition set the hostname to "(None)".
It has been done by introducing /etc/cloud/cloud.cfg.d/01_hostname.cfg
with content:
hostname: debian
Review supposed the hostname would be overriden by cloud meta-data. That
might have stand true for Wheezy but it is not the case for Jessie.
cloud-init 0.7.6 ignores cloud metadata whenever "hostname" or "fqdn"
are set in a config file. Roughly:
# no fqdn set, get fqdn from cloud
# get hostname from cfg if available otherwise cloud
fqdn = cloud.get_hostname(fqdn=True)
if "hostname" in cfg:
# hashar: set from config file NOT cloud
hostname = cfg['hostname']
else:
# fallback to cloud
hostname = cloud.get_hostname()
Relevant code is
https://github.com/number5/cloud-init/blob/0.7.6/cloudinit/util.py#L839-L860
Only inject "hostname: debian" for the Wheezy release.
Bug: https://phabricator.wikimedia.org/T117283
Change-Id: I6e2522bd725cbf9651f11c76ecdc72ecbc92f402
Previously all files in /root were ignored when building the
ironic-agent ramdisk. This prevented for example to use the
local-config element to connect to the ramdisk via ssh as root user.
This commit change the exclude rule on /root to only ignore the
/root/.cache directory.
Change-Id: I18d839e8d97636f5f2164ba407f252407d9bc956
Closes-Bug: #1451668
yum-minimal/root.d/08-yum-chroot runs before yum/root.d/50-yum-cache,
and thus if run on a completely fresh system will fail in
08-yum-chroot as the YUM_CACHE directory isn't made.
This is probably hidden by testing & nodepool builds, because it sets
DIB_IMAGE_CACHE. It was hidden from me because locally I have done
builds using the "yum" element previously, which had created the
cache.
Change-Id: I333f5f7e67d198f75a522cc296c118c2e94a5ecb
download.fedoraproject.org uses dns round robin and occasionally
hits a bad server. Using DIB_EPEL_MIRROR when finding the
epel-release package will allow us to avoid it e.g. in ci.
Change-Id: I756223b3e669532476663c05e79c238449b8a0db
Xen paravirtualised disks (supported by most modern kernels) have the
"xvd" prefix (e.g. xvda0). The functions to strip partitions need to
match on Xen PV disks otherwise the device name is discarded.
Change-Id: I5539d2afba3fae30d1ddb49dcbf077113d38bbf7
Closes-Bug: #1498576
Grub fails to install the bootloader due to it being on the root
partition of a block device. This is not actually a problem for us, so
we need to force it to succeed.
Change-Id: I335ef04ca8a8a8a5c242d3444b09bcce0a9f51e7
Without this patch, the devuser element attempts to find public keys by
iterating over the string "rsa dsa". When two keys are grouped together
in quotes, a bash for loop treats it as a single key. You can see the
issue this causes when debug output is turned on:
+ for fmt in '"rsa dsa"'
+ '[' -f '/home/krinkle/.ssh/id_rsa dsa.pub' ']'
This is not a reasonably named key to look for, so this patch removes
the quotes so that the loop will look for id_rsa.pub and id_dsa.pub
separately.
Change-Id: I0b5b1abd14013de85d90e76a95918a8071a5e013
Make sure we reset the yum/dnf cache to /var/cache/${YUM}, not just
/var/cache/yum
This was resulting in the F22 fedora-minimal image being larger than
the base-image. Because F22 fedora-minimal does some installs with
dnf when bootstrapping the chroot before we set "cachedir=" to the
bind-mounted external cache, we have "/var/cache/dnf" created and and
populated with the package meta-data, etc.
When we globally point dnf to /var/cache/yum here, we effectively
orphan the /var/cache/dnf created in those first steps. dnf doesn't
care, but we end up with two copies of all the package metadata, etc
in "/var/cache/dnf" & "/var/cache/yum".
This also cleans up the sed a bit, by just replacing the lines.
Change-Id: Icc98fe30c34cb941aed4b987647ab67ac34af15a
I'm not sure why we try to do an extra install of these, it is done
inside the chroot in _install_repos. Currently it just gets skipped
saying the packages are already installed.
Change-Id: Ic7aa8cbe13e4347b447e84bb9c12483a4e125228
Add basic F22/dnf support to yum-minimal path. We extract common
code, add some comments and reduce duplication.
Change-Id: If4bd5f88e26bd6f2168958f1ec1efff1072de7ba
Evidently the readme file hasn't been updated since rhel7 finished
beta, so this is long overdue.
In addition, since it's not possible to download the base image
file directly, let's stop pretending we can and bail out if the user
didn't set the necessary env vars.
Also updated the README to use the new table format instead of free text
Co-Authored-By: Augustina Ragwitz <aragwitz+lp@pobox.com>
Change-Id: Ie8343ee2ce1715583c28de7f59daed7e58c8ca0f
Move yum-based install into a function, to make way for a second
related function where use dnf later
Change-Id: Iad09f3753ecdfa0c10cb8a0970a3c8e5a2dccab1
Find doesn't like listings disappearing while its trying to find them,
in this case if a PID directory disappears while find is running. Using
-xdev prevents find from going into ./proc and as a side effect /dev
will also be avoided which is mounted on boot so not needed either.
Change-Id: Iaa282e58d81d533ad4445da0a44200dd14bf0850
Closes-bug: #1502142
Reorder the script number of 'elements/dkms/post-install.d/99-dkms'
to 'elements/dkms/post-install.d/97-dkms' to ensure that
it will always get executed before the
'elements/ramdisk/post-install.d/99-build-ramdisk'. This
would make sure that the DKMS module is there in the ramdisk.
Closes bug: #1492904
Change-Id: I2145d0ac29646335f76745a7678d169a62f13d44
Traversing the /proc filesystem causes find to error if it changes
while its being searched.
We have had a lot of ci failures on this find command since it was
added in Ibe40e6b8b884f37e3b5aeab6e7654593bcd63123
Change-Id: Ia8cfc923cce749a69d5108e588db2360238d866c
Closes-Bug: #1501949
Otherwise on rebuilding the agent the following error is produced:
failed to create hard link /home/stack/ironic-agent.vmlinuz
and the vmlinuz file is not updated.
Change-Id: I2015da889c932a854727235b1e34256a28e9eac6
Always no quotes for $() statement.
We don't need quotes to hold blanks in result:
# i=$(echo 1 2 3)
# echo $i
1 2 3
#
These quotes can make something wrong in some case:
# i=$(echo '!')
#
# i="$(echo '!')"
-bash: !: event not found
#
No real problem for current code, only to use a better code style.
Change-Id: I5909636bdc8de3d44a305d033c8c892af446acf3
Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com>
When a ubuntu/IPA ramdisk is used to boot a baremetal machine with
ironic agent-ilo driver, it fails at the point of mounting /proc
and /sys. After the vmlinuz(kernel) is started and it tries to
load the partitions on ramdisk. It need the directory of "/sys"
and "/proc" to mount the corresponding filesystems.
In order to fix this issue, the directories of "sys" and "proc"
are retained but the subdirectories or files under them are empty.
With this change, the directories of "/sys" and "/proc" shows
up in the ramdisk and kernel will mount sys and proc filesystems
on them respectively.
Closes-Bug: #1488445
Change-Id: Iad5d62f373b73789118f23db4c932ea6e9a784c3
Signed-off-by: Gary Duan <duanlg@live.cn>
ironic-agent requires expect to be installed for config drive
creation and hence this commit adds it.
Change-Id: Ie1c0f488f416b4c373aa7f38dfd8df1917cd6be2
Depends-On: Ib4dd8c082a50e1dbaf0df91477b062716cb780ff
Closes-Bug: #1486967
fedora-release >= 22 has acquired a dependency on /bin/sh. This comes
from a %posttrans section of the spec file, which is symlinking the
os-release file.
As discussed in [1], the links are setup correctly in the rpm, so the
post-install script isn't doing anything. Thus we can safely ignore
the dependency with --nodeps
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1265873
Change-Id: Icf17c84580a75d42d8e90d5d6e81ae7f5f576c32
Adds support for debian to iso.
Updated README to further clarify what the element does.
Change-Id: I17ac89cfbc84365860c591fab0e4c78123035983
Co-Authored-By: zhangjian <jian.zhang8@hpe.com>
The ironic-agent element is created using the disk-image-create utility
(even being a ramdisk) and outputs a .vmlinuz file for the kernel
(different than the ramdisk-image-create which outputs a .kernel file
for the kernel). This is inconsistent and make scripting against the
diskimage-builder more complicated if one wants to support different
types of ramdisk.
This patch creates a hard link for the .vmlinuz file to a .kernel file
at the end of the process (to keep backward compatibility) and print a
deprecation message.
Depends-On: I81400305f166d62aa4612aab54602abb8178b64c
Change-Id: I476f9ec9ec4206ece0261eaaf2b4182c6bcbd802
Closes-Bug: #1482606
Added support for ramdisk-type elements in tests/test_functions.bash
Elements are distinguished by element-type file in a test element.
Note that ironic-agent ramdisk is built with disk-image-create.
Change-Id: I4759859e7f3c004c2d00e7318729602e6c3c4d95
Largely enhance the documentation so it renders nicely when generated
with Sphinx.
Culpirt: the 'package' type is documented but unhandled in the shell
script.
Change-Id: I9f4f46e770077c147c0a5b1245b779bc3afa4e98
The centos cloud images are both arround a GB in size, and
downloading them causes lot of CI timeouts, downloading the
compressed version saves 700MB of bandwith and should save
a lot of time.
Change-Id: I8dcd1db81fe5c4661945638ef3e6344fdf651243
The ability to specify a distro mirror is part of other
distro elements, centos should have one too.
Change-Id: I4cc9062ff92fbe301f414820798e08c66e9793f7
This exact repository along with others are part of the
centos cloud image. From the looks of it this was required for
a very early cloud image.
Change-Id: Ib928e4ea739bc48f196f81c96ed4fba3177471f0
Completed in Kilo the blueprint ipa-as-default-ramdisk [1] ported all
the Ironic drivers in tree to be able to use the IPA ramdisk for the
deployment.
Now in Liberty the blueprint deprecate-bash-ramdisk is deprecating the
bash ramdisk created using the "deploy-ironic" element in DIB.
This patch is printing a deprecation message when the user uses the
"deploy-ironic" element and as well updating the README file to indicate
that it has been deprecated.
[1] https://blueprints.launchpad.net/ironic/+spec/ipa-as-default-ramdisk
[2] https://blueprints.launchpad.net/ironic/+spec/deprecate-bash-ramdisk
Related-Blueprint: deprecate-bash-ramdisk
Change-Id: I8057f52104225326f45eb3ae6065cd02a27f5ef2
In Id1e430e7d050a0b99ac449e2ea435e06cda1c4e6 I made the mistake of not
actually removing grub in 15-remove-grub.
This restores the removal phase and adds a bunch of comments. It
seems the centos7 and centos (6) images have grub2 installed, but F22
does not; hence the check.
For anyone interested in the history; it seems the whole idea of
removing grub and re-installing it in the finalise stage is to do with
Ubuntu grub scripts failing in the chroot. It is not clear this does,
or has ever, affected rpm based systems; but that's how it is, so
leave well enough alone.
The whole reasoning behind the rpm download & re-install is actually
explained in If095adc4abb52a19a3aa0b1caebfb3e4d8f605ef, but over time
the comments got lost as code moved around. I've restored in here
some detailed explaination of why we don't just re-install the package
"normally". I've also added a note to the pre-install of various
things that are related to this step. Again I think there are some
questions around this that we can investigate in another change.
Change-Id: I1acd19da8567ab93b5003caf67673cc70efea5fa
Currently they are used for inspection, but may be also used for
other purposes, as they're accessed from IPA generic hardware layer.
Change-Id: I32c6a711d466131b9445023812a2a260ed2e01f3
Switch to using svc-map element for systemd based agent.
This allows both .deb and .rpm installs to share the
element for systemd based installs. There are not any
plans to package a .rpm package for upstart or sysv, so
these are left as is.
Change-Id: Idca7ad97355cae785162989774a7e6dea6fdc5b5
Closes-Bug: #1490584
Fixing the ironic-agent pkg-map by adding missing commas. Validated
updated form passes json linting. Also includes a listing for curl.
Change-Id: I1983f7a581be3a5aaa771b19c6609cf12b61a7bb
Closes-Bug: #1488969
Appears that growroot was running before /dev is mounted so the script
is unable to introspect the filesystem partition info. Run this after
all local filesystems are mounted to fix this issue.
Change-Id: Ia7c41ba6ef79788fdbf198998622eeaa20dd4245
We can resize the rootfs without the initrd based approach. Create a
growroot element which performs rootfs resizing as part of system init.
Change-Id: Ibeb846b0170d141fb72323a441d14b65b93ae0a1
There is a bug where the init scripts element incorrectly munges the
install path making it useless. Also removing the dep on rsync since
this occurs from inside the chroot.
Change-Id: I8f2717d36d7d2ff4b195ec21e91afeaf30a1d803
This patch is reducing the size of the ramdisk image generated by the
ironic-agent element. It does remove extra packages (graphical stuff,
dev stuff, miscs, docs, etc...) and purges directories that are not
needed for a ramdisk (like /boot since it boots using an external
kernel)
Currently it was tested generating a Fedora 22 image and reduced the
size of the final image from 464 MB to 211MB compacted (54% decrease).
I was able to boot a VM with 1.3 GiB of ram instead of the previous 3 GiB
needed.
Change-Id: Id6333ca5d99716ccad75ea1964896acf371fa72a
The default value was set in the centos7 element, but not
exported, which caused issues in rpm-distro. Also changed
a test in rpm-distro to only check for DIB_RELEASE > 22
if it's fedora.
Closes-Bug: #1477172
Change-Id: Ib6f4227411c2e8f1965c3b78bc318512c59a7876
The script for ironic-agent utilizes curl, however an extremely
minimal system may not have it, and as such we should list it as
a package that must be installed to support the element.
Change-Id: Id118f84e2d5e6adf0ae3d653864565368b0d76bf
As described in the comments, sfdisk was rewritten for util-linux 2.26
(as shipped in F22) and now interprets arguments a sectors, rather
than cylinders.
The current partitioning line is "1 - - *" (start/size/type/bootable)
which means you start getting:
---
/usr/sbin/grub2-install: warning: this msdos-style partition label has
no post-MBR gap; embedding won't be possible.
/usr/sbin/grub2-install: warning: Embedding is not possible. GRUB can
only be installed in this setup by using blocklists. However,
blocklists are UNRELIABLE and their use is discoura ged..
/usr/sbin/grub2-install: error: will not proceed with blocklists.
---
when building images, because the start is interpreted by the new
sfdisk as sector 1 and it crams the partition right next to the MBR.
Specifying "-" for the size is undefined in the man page; even reading
the source it's not totally clear what "-" for the size does [2]. In
any case, the alignment is wrong in sectors or cylinders; we want to
be a multiple of 4KiB for best performance.
The intent here is to create one single, Linux, bootable, partition
taking up the whole disk starting at 1MiB, so "2048 + L *" makes this
clear.
We use the -uS argument to ensure both versions treat this start-value
as a sector offset (newer sfdisk essentially ignores the argument).
As described in the comments, bugs in the older sfdisk necessitate
usage of "--force".
Although we could choose more or less, it seems most common to align
to a 1MiB boundary (i.e. starting at sector 2048). libguestfs has
some disucssion around --alignment and where it sets it's default to
this [3]. The 2.26-era sfdisk also defaults to putting partitions
here. 1MiB should be enough for GPT schemes in the future as well.
[1] https://github.com/karelzak/util-linux/blob/master/libfdisk/src/script.c#L1050
[2] https://bugzilla.redhat.com/show_bug.cgi?id=1249893
[3] http://libguestfs.org/virt-resize.1.html
Change-Id: I2c2966f98d1d5ad4ebb433ea148b3b26c65dc1b5
Using the default release for the Debian test-element leaves
us at the mercy of packaging problems in Debian unstable. Since
we only care if the image itself builds, use the safe choice of
Debian stable.
Change-Id: I4dd58171489aa07d60db7e4e353ccde6aa534da7
Splits the install.d into source-install and package-install in
order to allow building from a distro packaged version of
ironic-python-agent.
Change-Id: I17513c29efd8c199e07ac1ef20ea5d7456585413
Adds support for Ubuntu and Debian to ironic-python-agent. This will
enable building ramdisk with Ubuntu and Debian as base OS.
Updated README to further clarify what the element does.
Change-Id: I194f85b051974d8ccb197a0993a67761046cfe98
interfaces are not configured at all because specific interface files
are created but not sourced. This will result in the VMs running with
the loopback interface only
This is a regression, the problem was already fixed in the past, see commit
b822581d88
Change-Id: I846642772ab582d7567e5182c860cfa0fe882a15
The resize module is problematic for using grub on very large root
partitions. It is also extremely slow on large partitions, and users are
likely better off creating new partitions in the empty space.
Change-Id: Ic050c74aa71165b43c8908c4d6c7c0ea99ddafa3
Story: 2000175
Using "yum --downloadonly" breaks the abstraction of
"install-packages" because it downloads to the yum cache. It also
acts funny if the package is already there.
Add an argument to "-d" which is the directory to download to. dnf
has "download" built in, and for the old case use yumdownloader which
acts about the same. Ensure it is installed, since it comes in
yum-utils.
Also a slight cleanup of the getopt parsing so it's easier to have the
required argument for -d
Thus we can remove most of the stuff in 15-remove-grub. The check for
centos6 and it's lack of grub2 is clarified. All the stuff about
having to remove the package, purging the cache etc so yum gets the
right thing is no longer relevant. The long section of commented out
code at the end is also removed for clarity.
I tested this with an F21, F22 & centos (6) build
Change-Id: Id1e430e7d050a0b99ac449e2ea435e06cda1c4e6
Docker can export root filesystems, which is what we want out of root
elements. Make a very simple passthrough element that will grab a docker
image and export it into a root filesystem.
Change-Id: Ie1e2d5dd5a61558f100e02c953b18d697a8fe8a2
There is a common pattern of if init_system == foo then install init
script foo-service-init into /etc/init... Lets encode this pattern by
allowing elements to put files into init-scripts/init-system directories
and then copying the appropriate files for them.
Change-Id: I541db18a0a8c5e0755a0af5732f4e15a5e5cf984
When building an image having the elements debian and vm, cloud-init
fails with:
ci-info: !!!!!!!!Net device info failed!!!!!!!!
Looking at the source code, it executes ifconfig to gather informations
but the `debian` elements does not provide it. There is also no DHCP
client available which is rather painful.
Install isc-dhcp-client to provide a DHCP client.
Install net-tools to provide ifconfig, required by cloud-init.
Ref: https://phabricator.wikimedia.org/T105152
Change-Id: I76dfd4f87a5c9f08e7c572fb4f5ebeeb34f5f66a
Remove hardcoded refrences for ethernet interfaces from ironic-agent
and sets a dependency on dhcp-all-interfaces to ensure it works for
all interfaces for all other operating systems.
Change-Id: I7ae6d1c5bd9911ef3db45187c0010cf0973badf1
Closes-Bug: #1471802
The fix for static links to the latest image has been rolled out.
Update documentation accordingly.
Change-Id: Ic92d0e1d584ca2bf1d82f411102079cb4455bddb
The init scripts have now been moved in to glean itself, so just consume
them directly.
Change-Id: Ib85128579c62020df23d73404c0563894038d2dd
Depends-On: I2ed25ce434023bfc8b6a88a08c0c06c1cef63982
Glean now supports setting a hostname, lets ask it to do this.
Change-Id: Iea8d210b4b5add8fed4038cf81ce28d1d7c7c1c4
Depends-on: Ia9155bc565ad79af44d88acc06759be2bf4e5f20
This element installs oat-client on the image, that's necessary for
trusted boot feature in Ironic to work. This element only works on Fedora.
Intel TXT will measure BIOS, Option Rom and Kernel/Ramdisk during trusted
boot, the oat-client will securely fetch the hash values from TPM.
Change-Id: I0f1221b5708e9a5792df62ee6e73034f8bf1577c
Passing a source-repositories ref of "*" should signal fetching all
heads similar to when a non-cached ref is requested. Reuse the same
fallback logic, but skip unnecessary checks since "*" is not a real
refname. Also expand the fallback to update tags, and to --purge
local refs that no longer exist on the remote for additional safety.
Change-Id: I4562c9689a8d235ebe09b2f7178aa5890dbc85f1
