We can resize the rootfs without the initrd based approach. Create a
growroot element which performs rootfs resizing as part of system init.
Change-Id: Ibeb846b0170d141fb72323a441d14b65b93ae0a1
There is a bug where the init scripts element incorrectly munges the
install path making it useless. Also removing the dep on rsync since
this occurs from inside the chroot.
Change-Id: I8f2717d36d7d2ff4b195ec21e91afeaf30a1d803
This patch is reducing the size of the ramdisk image generated by the
ironic-agent element. It does remove extra packages (graphical stuff,
dev stuff, miscs, docs, etc...) and purges directories that are not
needed for a ramdisk (like /boot since it boots using an external
kernel)
Currently it was tested generating a Fedora 22 image and reduced the
size of the final image from 464 MB to 211MB compacted (54% decrease).
I was able to boot a VM with 1.3 GiB of ram instead of the previous 3 GiB
needed.
Change-Id: Id6333ca5d99716ccad75ea1964896acf371fa72a
The default value was set in the centos7 element, but not
exported, which caused issues in rpm-distro. Also changed
a test in rpm-distro to only check for DIB_RELEASE > 22
if it's fedora.
Closes-Bug: #1477172
Change-Id: Ib6f4227411c2e8f1965c3b78bc318512c59a7876
The script for ironic-agent utilizes curl, however an extremely
minimal system may not have it, and as such we should list it as
a package that must be installed to support the element.
Change-Id: Id118f84e2d5e6adf0ae3d653864565368b0d76bf
As described in the comments, sfdisk was rewritten for util-linux 2.26
(as shipped in F22) and now interprets arguments a sectors, rather
than cylinders.
The current partitioning line is "1 - - *" (start/size/type/bootable)
which means you start getting:
---
/usr/sbin/grub2-install: warning: this msdos-style partition label has
no post-MBR gap; embedding won't be possible.
/usr/sbin/grub2-install: warning: Embedding is not possible. GRUB can
only be installed in this setup by using blocklists. However,
blocklists are UNRELIABLE and their use is discoura ged..
/usr/sbin/grub2-install: error: will not proceed with blocklists.
---
when building images, because the start is interpreted by the new
sfdisk as sector 1 and it crams the partition right next to the MBR.
Specifying "-" for the size is undefined in the man page; even reading
the source it's not totally clear what "-" for the size does [2]. In
any case, the alignment is wrong in sectors or cylinders; we want to
be a multiple of 4KiB for best performance.
The intent here is to create one single, Linux, bootable, partition
taking up the whole disk starting at 1MiB, so "2048 + L *" makes this
clear.
We use the -uS argument to ensure both versions treat this start-value
as a sector offset (newer sfdisk essentially ignores the argument).
As described in the comments, bugs in the older sfdisk necessitate
usage of "--force".
Although we could choose more or less, it seems most common to align
to a 1MiB boundary (i.e. starting at sector 2048). libguestfs has
some disucssion around --alignment and where it sets it's default to
this [3]. The 2.26-era sfdisk also defaults to putting partitions
here. 1MiB should be enough for GPT schemes in the future as well.
[1] https://github.com/karelzak/util-linux/blob/master/libfdisk/src/script.c#L1050
[2] https://bugzilla.redhat.com/show_bug.cgi?id=1249893
[3] http://libguestfs.org/virt-resize.1.html
Change-Id: I2c2966f98d1d5ad4ebb433ea148b3b26c65dc1b5
Using the default release for the Debian test-element leaves
us at the mercy of packaging problems in Debian unstable. Since
we only care if the image itself builds, use the safe choice of
Debian stable.
Change-Id: I4dd58171489aa07d60db7e4e353ccde6aa534da7
Splits the install.d into source-install and package-install in
order to allow building from a distro packaged version of
ironic-python-agent.
Change-Id: I17513c29efd8c199e07ac1ef20ea5d7456585413
Adds support for Ubuntu and Debian to ironic-python-agent. This will
enable building ramdisk with Ubuntu and Debian as base OS.
Updated README to further clarify what the element does.
Change-Id: I194f85b051974d8ccb197a0993a67761046cfe98
interfaces are not configured at all because specific interface files
are created but not sourced. This will result in the VMs running with
the loopback interface only
This is a regression, the problem was already fixed in the past, see commit
b822581d88
Change-Id: I846642772ab582d7567e5182c860cfa0fe882a15
The resize module is problematic for using grub on very large root
partitions. It is also extremely slow on large partitions, and users are
likely better off creating new partitions in the empty space.
Change-Id: Ic050c74aa71165b43c8908c4d6c7c0ea99ddafa3
Story: 2000175
Using "yum --downloadonly" breaks the abstraction of
"install-packages" because it downloads to the yum cache. It also
acts funny if the package is already there.
Add an argument to "-d" which is the directory to download to. dnf
has "download" built in, and for the old case use yumdownloader which
acts about the same. Ensure it is installed, since it comes in
yum-utils.
Also a slight cleanup of the getopt parsing so it's easier to have the
required argument for -d
Thus we can remove most of the stuff in 15-remove-grub. The check for
centos6 and it's lack of grub2 is clarified. All the stuff about
having to remove the package, purging the cache etc so yum gets the
right thing is no longer relevant. The long section of commented out
code at the end is also removed for clarity.
I tested this with an F21, F22 & centos (6) build
Change-Id: Id1e430e7d050a0b99ac449e2ea435e06cda1c4e6
Docker can export root filesystems, which is what we want out of root
elements. Make a very simple passthrough element that will grab a docker
image and export it into a root filesystem.
Change-Id: Ie1e2d5dd5a61558f100e02c953b18d697a8fe8a2
There is a common pattern of if init_system == foo then install init
script foo-service-init into /etc/init... Lets encode this pattern by
allowing elements to put files into init-scripts/init-system directories
and then copying the appropriate files for them.
Change-Id: I541db18a0a8c5e0755a0af5732f4e15a5e5cf984
When building an image having the elements debian and vm, cloud-init
fails with:
ci-info: !!!!!!!!Net device info failed!!!!!!!!
Looking at the source code, it executes ifconfig to gather informations
but the `debian` elements does not provide it. There is also no DHCP
client available which is rather painful.
Install isc-dhcp-client to provide a DHCP client.
Install net-tools to provide ifconfig, required by cloud-init.
Ref: https://phabricator.wikimedia.org/T105152
Change-Id: I76dfd4f87a5c9f08e7c572fb4f5ebeeb34f5f66a
Remove hardcoded refrences for ethernet interfaces from ironic-agent
and sets a dependency on dhcp-all-interfaces to ensure it works for
all interfaces for all other operating systems.
Change-Id: I7ae6d1c5bd9911ef3db45187c0010cf0973badf1
Closes-Bug: #1471802
The fix for static links to the latest image has been rolled out.
Update documentation accordingly.
Change-Id: Ic92d0e1d584ca2bf1d82f411102079cb4455bddb
The init scripts have now been moved in to glean itself, so just consume
them directly.
Change-Id: Ib85128579c62020df23d73404c0563894038d2dd
Depends-On: I2ed25ce434023bfc8b6a88a08c0c06c1cef63982
Glean now supports setting a hostname, lets ask it to do this.
Change-Id: Iea8d210b4b5add8fed4038cf81ce28d1d7c7c1c4
Depends-on: Ia9155bc565ad79af44d88acc06759be2bf4e5f20
This element installs oat-client on the image, that's necessary for
trusted boot feature in Ironic to work. This element only works on Fedora.
Intel TXT will measure BIOS, Option Rom and Kernel/Ramdisk during trusted
boot, the oat-client will securely fetch the hash values from TPM.
Change-Id: I0f1221b5708e9a5792df62ee6e73034f8bf1577c
Passing a source-repositories ref of "*" should signal fetching all
heads similar to when a non-cached ref is requested. Reuse the same
fallback logic, but skip unnecessary checks since "*" is not a real
refname. Also expand the fallback to update tags, and to --purge
local refs that no longer exist on the remote for additional safety.
Change-Id: I4562c9689a8d235ebe09b2f7178aa5890dbc85f1
Some minor workarounds for Fedora >= 22 where dnf is the default
package manager. The changes are documented on the Fedora release
notes https://fedoraproject.org/wiki/Changes/ReplaceYumWithDNF
Change-Id: I7d7d6f5d294980dcb217d6190a1efd9e0bbea9a6
Add a YUM variable that defaults to dnf for Fedora 22 and greater. At
this stage the yum element can do double-duty with dnf -- it's mostly
the same. If we find it starts getting too unwieldy we can separate
this later.
Modify the install-packages for yum to use this variable when set, but
default back to "yum" to retain the status-quo.
Change-Id: Ibff71465b392d9f66b6f93955ff9223575d6165c
Installing Python to a ramdisk takes quite a long time because of
the way dracut checks for dependencies of every single file
installed. We could avoid that, but then we might miss a required
library file.
This change alters the installation method to speed up
the process. First, it creates a list of files that are needed and
then installs them all at once using inst_multiple instead of calling
inst on each file separately. This doesn't make a huge difference,
but in my testing it is marginally faster.
Second, and more significantly, we don't need the *.pyo and *.pyc
files as those are simply an optimization to speed up module
loading. Because the deploy ramdisk is a short-lived operation,
we probably lose more time transferring those extra files to the
target system than we save in improved load times.
In my testing, these two changes netted about a 20% improvement
in build times, and about 13% decrease in image size.
Change-Id: Ibc2b778c28fc9fb7177380dffe8dbce5722d0733
On some systems, it can take longer than 10 seconds for the root
disk to be detected. Because enterprise hardware. Increase the
wait time to 60 seconds so we don't incorrectly fail due to a
missing root device.
Change-Id: I4f67ef0295af8f2ae783fe3aea347b41987c6a66
This reverts commit 7a4c396948.
Note this time it sets it to F21, not F22, and gets rid of the
duplicate definition that doesn't get exported.
Change-Id: I240ad25d7a73c379559517a2a8399ae8c098314b
Changed simple-init to utilize a PATH variable in order to allow
for glean to be executed in the event the operating system places
glean in /usr/bin, such as what occurs on CentOS 7.
Change-Id: Ibf95fcd7ca368595e8fb3473f25eb0a919726e39
Now that we are changing fedora to default to f22, lets gate on f21
still working.
Depends-On: Icbd08fb5aa69446ad65ff72af631902c4e1fa12b
Change-Id: I8b7b957fafc028aa2970803bd23ea644114f9b7f
We are using which to find the location of setfiles. Our script is set
-e though, and we need to also be able to handle the case where setfiles
does not exist (like on centos-minimal).
Change-Id: If53c7a80efc081b95b143c28be64d39b12bfb469
The default release for fedora is actually 21 for us, and we are unable
to build 22 right now.
This reverts commit 379d6a2650.
Change-Id: Iffcc505f1e115cb6bc662b57a78878e498ce338e
Wrong package in the list, dracut-modules-growpart is wrong,
needed dracut-modules-growroot for proper resize to work
Change-Id: Iea8789ea3d44d182197a4713244b551f2cd4dd55
Closes-Bug: 1461601
This patch is adds a new grub2 element that installs the
grub2 bootloader on the image so it can work with Ironic's
local boot support.
This patch also modifies the iso element which was installing
same grub2. It removes the grub2 installation from iso element
and makes it dependent on grub2 element.
Co-Authored-By: Ramakrishnan G <rameshg87@gmail.com>
Change-Id: I37bcf2c525708d1e2e0f95cf5874a279f76861f7
This commit adds support for providing custom kernel
cmdline args while building deploy ISO. This is useful
for adding kernel cmdline like 'console=ttyS1' in
environment (assuming all bare metals have been configured
to output to COM2 in BIOS).
Closes-Bug: 1451634
Change-Id: I20b04d9d104cfe46df0439c3f567a721a27e186a
This commit addresses follow-up comments on
I1ffb832ebab009b2d77a46e6c8fc758dd9632359. The change
is to delete get-pip.py immediately after installing pip.
Change-Id: I2768da2365b08304b8e7fcf55c91101b05ec33ea
Add current version of Ironic API for consistensy with IPA and Ironic
API settings, and for compatibility in the future.
Change-Id: I13c7a26b6cfb47a14aa49ee78441a1d97d7b42d0
In the case of using portal registration with an activation key, the
rhel-common element is still executing a `subscription-manager attach`
command. This should not happen if an activation key is provided. This
is because an activation key already provides the subscriptions to
attach.
This patch fixes this behavior.
Change-Id: I5a8425d1778362bb7a0dadc91a46308f16b2a526
Closes-Bug: #1456648
It's useful to be able to pass in multiple yum repo configuration files
via $DIB_YUM_REPO_CONF, not just a single one.
Change-Id: I43722229a2df58be55bdb2b50c253e957b18e6fe
When something goes wrong, you usually can't boot the image. nova
console-log is usually available though.
Change-Id: Ie4525d0c3ee8b59f035544592b30f0635aba1811
Hard coded path fails on Distros (such as el6) with setfiles bin
in different places, for example, rhel6 has this in /sbin/setfiles
Change-Id: I7aff9cdadd9aed9cfc806a1010acbf36b7b6d0e7
dib-run-parts filters the acceptable characters in script names,
and "." is not allowed (see $allowed_regex there), so
01-clean-old-kernels.sh is never executed.
Rename it to drop its .sh extension, so it is executed for real.
Change-Id: Ieb633b31214f1accf03b92a2b06590fdf2127b6b
Weve had some regressions recently with the changes in the debian
element. Lets tests that we can build debian images.
Change-Id: I048e7a32ecb4088ec1b1e3b1efdf146187b093db
Adding a test function which allows us to use elements to perform
element-specific tests. In order for this to work sanely, also adding
some configuration to our break system so we can assert on negative
tests.
Also adding a test for apt-sources to verify this code actually works.
Change-Id: I378a74255010eca192f5766b653f8a42404be5ea
We should make use of the CentOS-7-x86_64-GenericCloud.qcow2 symlink from
http://cloud.centos.org/centos/7/images/ instead of having a hard coded cloud
image. Specific cloud images can still be downloaded by overriding
$DIB_RELEASE.
More importantly, using the symlink will keep us automatically up to date with
the latest CentOS 7 cloud image. The image in use by the hard coded value
occassionally exhibits "No space left on device" errors after the cloud-init
filesystem resize. More info about this issue is at:
http://xfs.org/index.php/XFS_FAQ#Q:_Why_do_I_receive_No_space_left_on_device_after_xfs_growfs.3F
The newer cloud image (with a newer kernel) does not exhibit this issue.
Change-Id: I3e19f6269ceba937fcd630bab265d132bd525519
Check for the current distribution using $DISTRO_NAME instead of
`lsb_release`.
Also, remove the existency check, as $DISTRO_NAME is supposed to be
provided by distribution elements.
Change-Id: I2276c63e9ac43576da528a70235129800c093b3e
Cloud-init and simple-init are not meant to play together. Lets disable
cloud-init if simple-init is installed.
Also guarding cloud-init-datasources against running in an environment
where cloud-init is not installed.
Change-Id: I5bfa9a3e83d3259db2436404034ad58c780de1c9
Diskimage-builder currently writes cloud-init config file which adds a
host entry mapping the hostname and FQDN to 127.0.0.1 into every image
built. This is probably useful for some use cases but not for all, so we
now allow customizing the manage_etc_hosts value via
DIB_CLOUD_INIT_ETC_HOSTS variable and also not writing the config at all
if that variable is explicitly set to an empty string (currently the
default is 'localhost' but in the future the default will be empty
string).
Particular description of the problem this causes in TripleO follows:
We get hosts files like this:
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
127.0.0.1 ov-rl5i5saoc6h-1-hj5tzsbrdv4c-controller-dy6nuyarqy5z.novalocal ov-rl5i5saoc6h-1-hj5tzsbrdv4c-controller-dy6nuyarqy5z
# HEAT_HOSTS_START - Do not edit manually within this section!
192.0.2.17 ov-rl5i5saoc6h-0-wfzcsrqo34p6-controller-m3hy26lhxavl ov-rl5i5saoc6h-0-wfzcsrqo34p6-controller-m3hy26lhxavl.novalocal
192.0.2.15 ov-rl5i5saoc6h-1-hj5tzsbrdv4c-controller-dy6nuyarqy5z ov-rl5i5saoc6h-1-hj5tzsbrdv4c-controller-dy6nuyarqy5z.novalocal
192.0.2.16 ov-rl5i5saoc6h-2-a6v7saxnivm5-controller-7jboskte34r7 ov-rl5i5saoc6h-2-a6v7saxnivm5-controller-7jboskte34r7.novalocal
# HEAT_HOSTS_END
The duplicate hostname/FQDN entry for 127.0.0.1 and 192.0.2.15 confuses
Corosync, which then fails to start a cluster when using hostnames in
the config file instead of IPs.
Change-Id: Ia8582883f737548e2911d3f36a1943e5b236281b
Partial-Bug: #1447497
vconfig is actually deprecated in favor of the ip command, and is not
available on some newer distros (RHEL 7 at least). I'm not honestly sure
why it needs to be installed in all images anyway. I traced the origins
of installing the vlan package here all the way back to the first dib
git import from some other repo...so, I don't see any obvious reason why
it needs to be installed.
Change-Id: I272667cf29f5e41c217a26f70937b2842a04f748
subprocess.CalledProcessError in Python 2.6 does not have the 'out'
parameter for __init__, so pass only two of them and manually set
'output' in that case.
Fixes/improves commit 7f410aaff2.
Change-Id: I279bdf433b1272a9c3af4d66a2a52c78a7ac5de2
Instead of manually creating epel.repo files, make use of the epel
element, which will properly install epel-release.
Change-Id: Iea7b389bc1ade716c622fd39d5e7dcf119dcb447
This commit address last comments on
I5e8a706989bad13051eb47db0b1e762e6c672318. It adds
the date for a comment was added and removes redundant
wait period for initialization.
Change-Id: Idff38835969c094175f68be78c407ae975473b57
This commit fixes errors while trying to create a
DIB ramdisk with ironic-agent element when built
behind proxy. It fixes the issue by making it install
latest versions of pip and setuptools which has the
fixes for them.
Change-Id: I1ffb832ebab009b2d77a46e6c8fc758dd9632359
Closes-Bug: 1449852
rhel7 and centos7 images are both only available on x86_64 arch.
if $ARCH is misconfigured, some strange error will happen during the build.
For example, DIB will try to install EPEL i386 on the 64bit root system.
For the record, tripleo-incubator $NODE_ARCH default value is i386. The
problem will happend as soon as the default value is used with one of
these root elements.
This commit ensure the $ARCH is set to amd64 as soon as the centos7 or
rhel7 root element are used.
Change-Id: Ie41fa2da48eac6bf89b96cfa137c0f572dae6734
In Kilo, we added added an iscsi extension in
ironic-python-agent which requires tgtd and tgtadm.
This commit adds scsi-target-utils to the ramdisk for
this.
Also the git protocol for retrieving the source code is
changed to http. Git protocol doesn't go through a proxy,
but http can.
Closes-Bug: 1449854
Change-Id: I8cf274913a16404941770d0c6115bd6feec1ccb8
Starting from syslinux 5.00, isolinux.bin is dependent on
ldlinux.c32 to boot for BIOS machine. syslinux > 5.00 is
delivered with Fedora 21 cloud image which breaks the boot
from ISO if ldlinux.c32 doesn't exist.
Change-Id: If722f36aeaabc759d93ef6ae3f49b21bb840a92d
Closes-Bug: 1449882
These are ubuntu-isms that do not exist on debian mirrors and cause
builds to break if they're based on a stable debian release.
Change-Id: I08c2826eba4aabd0be69955220624b2f179a15ee
Closes-bug: #1450198
Set the pbr option 'warnerrors' to make build_sphinx turns warnings into
error. Fix all warnings.
`tox -edocs` will thus abort whenever someone introduce a new error.
Change-Id: Id6d09768a241866e1fdc1a1e2bf90336f5c5087d
debootstrap is not debian or ubuntu specific. We can make a debootstrap
element that knows how to do all of the things, and then a
debian-minimal and ubuntu-minimal image that use it. Finally, make
the debian element simply be a collection of the extra things we do to
make it look like a cloud-init based cloud image.
Change-Id: Iaf46c8e61bf1cac9a096cbfd75d6d6a9111b701e
glean is now moved into the openstack-infra repos, so the reference to
the originally temporary home can be discarded.
Change-Id: Ie89fff85e264a36d9bab15801314d5195b45031c
In some cases, like linux-image-* on debian, we need to only install
packages for a specific target architecture.
Change-Id: Ic0009d0c1e121d6f3f1f21345c544e2d98f080f9
This change uses blkid to identify the fs type during redhat-common
extract-image. The image is mounted with -o nouuid for redhat/rhel
images that have XFS filesystems.
This is required when building images from the same base image
as the host VM to avoid "Filesystem has duplicate UUID" mount
failures.
Change-Id: I066289fbb27733a5a555242a0e2c363d58dd27d0
Closes-Bug: 1443706
Now that we have a generic yum-minimal element, just use it in centos
instead of rinse. Adding base as an element-provides of yum-minimal
because this element conflicts with the base element.
Co-Authored-By: Gregory Haynes <greg@greghaynes.net>
Change-Id: I15275d821781171c118f21aa0c0bca55f65a65b3
The loopback handling in the Linux kernel limits the filenames of
files associated to loopback devices, see also linux/loop.h.
This is reflected also on userspace, as kpartx will silently do nothing
(exiting with 0) when requesting to remove a filename longer than 64
characters, as that name will obviously not match the truncated
filename. The result of this is that, when extracting qcow2 images for
the first time, if the qcow2 filename is long enough then the loopback
device will not be removed, remaining as stale in the host.
As a workaround, use a temporary file name when convering a qcow2 image
to raw, instead of using the base name of the qcow2 file.
While this still will not fix the issue when manually using a long
temporary directory (e.g. TMP_DIR=/very/long/etc...), at least should
avoid it in other cases.
Change-Id: Ibf46cd313a9d89412c0e1068fa0993be6c5a29db
This commit changes Ironic deploy ramdisk to find out
the virtual media device by using labels instead of
looking at the model of block device. This helps in
finding out the device irrespective of the hardware.
Corresponding Ironic change is
If5b78d9af7048f2631d050ee5ce01ab7a67e2354.
Closes-Bug: #1429340
Change-Id: I5e8a706989bad13051eb47db0b1e762e6c672318
Commit b4a1f1c190 wrongly changed the format of the available
images. There was a missconfiguration on the buildservice which produces
the images so no static links were available. That's fixed now so use
the correct names again.
Change-Id: Iac4cbc8672da67f5a89ac2f1be8bb9530215ea19
The centos-minimal approach of using rinse does not, it turns out, work
on centos. That's a bummer. It's also rather heavyweight. Instead, with
minor machinations, we can just use yum itself pointed at a chroot.
Also adding fedora-minimal element which creates a fedora image using
the new yum-minimal approach.
Co-Authored-By: Gregory Haynes <greg@greghaynes.net>
Change-Id: I026fd9d323e786dae5bb67824c6501067e1ceaa3
If you don't want cloud-init, you may need to get a few things
from config-drive because you may be operating on a cloud with no DHCP.
In that case, simply reading some values from config-drive and writing
out either DHCP or static network info, in addition to grabbing ssh keys
is helpful. Both Infra and bifrost want this for their images.
Co-Authored-By: Gregory Haynes <greg@greghaynes.net>
Change-Id: I2746ed256b9783eab058b803130d3ccac484eaeb
We support building elements without depending on the base element.
Breaking install-types out into its own element while making base depend
on it so elements can depend in it without base.
Change-Id: I104543d5482c76f60902e9fc32d91e196eeab51a
Turns out that updating packages last causes some pretty
non-intuitive behaviour if you are trying to pin a package
to a specific version. Lets just update the base RPMs first...
subsequent installations should install the most updated version
anyways (unless they are pinned).
Also moves the package-installs script from the 00 step to 01 so
we can do the update first.
Co-Authored-By: Ben Nemec <bnemec@redhat.com>
Change-Id: I962046cc6048e852e6582fbc579f88bb73e23fdd
This fix prevents loading of unsigned ubuntu kernel in UEFI secure
boot environment when image is created using 'iso' element.
'iso' element uses 'linux' and 'initrd' modules of grub2 to load
kernel and initrd respectively. The grub2 implementation of Ubuntu
can load unsigned kernel when these modules are used.
Ubuntu has Grub2 modules 'linuxefi' and 'initrdefi' which exits
boot process if unsigned kernel is used in UEFI secure boot mode.
The 'iso' element should use these modules in grub.cfg to prevent
loading of unsigned kernel when node is booted in the UEFI secure
boot environment.
'linuxefi' and 'initrdefi' works seamlessly when node is booted in
normal UEFI boot mode (non-secure).
Fedora do not have this issue. This fix has been tested in Fedora
environment. It works fine.
Closes-Bug: 1443114
Change-Id: If256ba1f7d7c149482d0f37fabcdfa8ed22e3f91
ubuntu-signed element would install 'linux-signed-image-generic' that
provides signed kernel that can be used for deploy in UEFI secure boot mode.
Package 'linux-signed-image-generic' ships signed kernel with extension
'.efi.signed' (Ex. '/boot/vmlinuz-3.13.0-49-generic.efi.signed').
The kernel modules directory for signed kernel and unsigned kernel is same.
It is without 'efi.signed' extension to its name. This is different from normal
practice of directory naming in '/lib/modules' (Ex. For signed kernel
'vmlinuz-3.13.0-49-generic.efi.signed', modules directory is
'/lib/modules/3.13.0-49-generic').
This needed some changes in '/lib/ramdisk-functions' and 'ramdisk' element to
copy kernel modules.
The signed kernel package contains both signed and unsigned kernel. The
unsiged kernel is without extension '.efi.signed' (Ex.
'/boot/vmlinuz-3.13.0-49-generic'). This required change into
'/lib/img-functions' and 'baremetal' element to pick up signed kernel version
when this element is used.
Closes-Bug: 1443076
Change-Id: I60061cbea847b47fa752b9463cfd387e8e7f0635
The targetcli element was triggering a bunch of errors from dracut
when we installed all of Python. It turns out this is because there
were filenames with spaces in the find output and the loop didn't
handle that properly. This switches to a while loop that can
handle odd filenames.
Change-Id: Iacbf16f26f2bc9991840250dc8ae7990db54d811
Currently, calling the troubleshoot function in a ramdisk script
doesn't work as expected on dracut ramdisks. This adds an alternate
troubleshoot implementation that will behave as intended.
I did not make it conditional on a kernel param as was done in the
original because dracut can behave strangely if you allow it to
continue after an error. Always dropping to a shell immediately
should be less confusing.
Change-Id: I98000f4ac6d7890b1f44fe4d10394ac0ea332fcb
Do not rely on environment changes (like exporting REG_HALT_UNREGISTER)
to persist between different hooks run. This helps when the hooks are
run in different new environments every time.
Instead, in 99-unregister redo the same checks on REG_METHOD as done in
00-rhel-registration, still respecting REG_HALT_UNREGISTER in case the
user does not want to unregister the image generated.
Change-Id: Id594dcd72334f38a2fa96da21206da77a83d7a1a
Closes-Bug: #1434431
Cleaning up the apt-sources README to be easier to consume. Also
removing some tripleo references from the README.
Change-Id: I6937fd5cd51288b36890dde214701bcef1d61381
We don't want to trace the RHEL registration scripts because that
is likely to log things like passwords and activation keys. To
still allow for debugging failed runs, add sanitized logging of
the arguments passed to the registration commands, since that is
the part of the process where problems are most likely to manifest.
Change-Id: I0f661e9c152f43b814fda61211bd56ba93e3b9dc
The default locale set by cloud-init is now generated to prevent the
warning printed when the user is logged in.
Closes-Bug: 1440728
Change-Id: I2faff6c9d3ab8bb5f66d58e77bcf37f186bf501d
Make sure that the target directory for 50targetcli exists already, in
case there is no dracut installed at extra-data.d run time.
Change-Id: I85ade9e85e823b7564a5839c8b6181548a15ad41
This commit changes the 80-deploy-ironic script of
deploy-ironic element to report back the status of
boot loader install (when boot_option == "local")
using a newly introduced vendorpassthru.
Closes-Bug: 1422723
Change-Id: I9c1d8643be7cb9e273d65ddd791715a5c271fd93
Copy all of the necessary parts for a Fedora based dhclient to work. This
includes a number of network scripts. Also grab the ip command supplied by
the iproute package, the busybox "ip addr" command was missing the valid_lft
and preferred_lft options.
This will allow the dhcp to work in the ramdisk instead of getting passed the
PXE net config.
Related-Bug: #1417026
Change-Id: I8feee9a740855dab7b47162c5727bf91db77fcc6
The listing of *-$INSTALL_TYPE-install files currently uses ls, which
errors out when the glob matches no files, thus using true to not fail
it.
Instead, use find to collect the file list, so there is no need to
ignore the command errors.
Change-Id: Ic6888106858df320a1c90a84f1b9ec74d436b9e6
The wrong APT config name is used to disable download of translations.
It's Acquire::Languages, not APT::Acquire::Languages.
Change-Id: Ie0c12d444bab19b4486845944ef51031e9133470
Closes-bug: #1436523
Not all operating-system elements install cloud-init, but the base
element assumes its existence. Create the directory if it does not
exist.
Change-Id: I4bda8dc5d200825ea0c8163a4e5c44050a45083f
it may happen that if the system where disk-image-create runs is busy,
then the kpartx -l run may leave a stale autodelete loop device.
This is because kpartx -l first adds a new loop device, then does the
listing and removes the loop device. The latter may not end before the
end of the kpartx run, leaving a loop device marked as autodelete.
Such kind of loop device will automatically delete itself, so the
rm -r $WORKING
after
sudo umount -f $WORKING/mnt
in the EXIT trap will fail because $WORKING does not exist anymore.
To prevent this situation, just ask udev to finish its operations,
properly removing the (temporary) loop device.
Change-Id: I12246f3dbe6b5669e698767682a5a142f803823b
RHEL 7 does not ship tgtadm or tgtd so they cannot be used in the
deploy ramdisk. This change separates the tgt-specific parts of
the ramdisk into their own element, and adds a new one that supports
targetcli instead.
For now, the tgt implementation can only be used with traditional
busybox ramdisks and the targetcli one can only be used with dracut.
This is because dracut is primarily used for RHEL right now so it
makes sense to keep the dependencies simple. If there is a future
desire to mix and match the implementations that could be done, but
it would require users to explicitly select between tgt and
targetcli.
Change-Id: I4f99c91016287e08d836095c2f2261de8b45abdc
Co-Authored-By: James Slagle <jslagle@redhat.com>
It is reasonable that elements may need to include additional
kernel modules in a dracut ramdisk. This is done with the
--add-drivers option to dracut, but previously the value passed
was hard-coded.
This change allows an element to put a file containing its desired
drivers in a dracut-drivers.d directory, and the list there will
be added to the list of drivers added. This functions in
essentially the same way as the binary-deps.d directory that
already exists for including additional executables in a ramdisk.
Change-Id: Ie892b908d36c175a469f7cde7dd803ad4b1942b6
This is required on Fedora 21 in order to build some
packages via source. Includes files like:
/usr/lib/rpm/redhat/redhat-hardened-cc1
Specifically this fixed MySQL driver compilation issues on Fedora 21
for source builds.
Change-Id: I459f2203fa145049dda185da952813118193d573
Official MariaDB repositories offer the package : MariaDB-Galera-server.
This package has been now ported within Fedora (and also RDO), the
package is now called mariadb-galera-server. Yum install being case
sensitive hence this change.
Change-Id: Icd03877f17d01708b3916578991e42eef30a69e4
As part of the blueprint root-device-hints Ironic will pass some to the
deploy ramdisk some hints about which disk device it should pick to be
root device (the one where the image will be deployed on).
Before the deploy ramdisk would pick the first device it finds, but as the
machine could have more than one SATA, SCSI or IDE disk controllers the
order in which their corresponding device nodes are added is arbitrary
causing devices like /dev/sda and /dev/sdb switching around on each
boot time.
Plus, as people are adding support to build RAID arrays in Ironic we need
a way to tell it to use the just created device to be the root device.
The list of hints that could be passed to the deploy ramdisk so it finds
the right disk is:
* wwn (STRING): unique storage identifier
* serial (STRING): disk serial number
* model (STRING): device identifier
* vendor (STRING): device vendor
* size (INT): The size of the disk in GB
If not hints are passed, the deploy ramdisk will continue to do what it
did before to find the disk.
Change-Id: I8425f593e1a610af5a3697988702603ff218f2de
This commit adds support for uefi localboot in
deploy-ironic element. The change is to mount the efi
system partition (created by Ironic) in /boot/efi.
The corresponding Ironic change is
I00ac31da325676ea4ea1ac4185f5ac3a52c5809a
Implements: blueprint local-boot-support-with-partition-images
Change-Id: Idf7ac5987e14e1d31311834196ca7283deec15c6
Commit 36b59c001c introduces
DIB_DEBUG_TRACE, to be checked in element scripts for enabling tracing.
In the aforementioned conversion, few scripts were left with
unconditional "set -x" calls: remove them, changing the default value
for unset DIB_DEBUG_TRACE from 0 to 1, to retain their older behaviour
(as it was done in 36b59c001c too).
Change-Id: I3d1a9290021bf63de7d4e7752e809852e784ac8b
Previously, this code was not checking for the proper environment
variable for an element's installtype. There was a line replacing '-'
with '_' as is required, but that value was not actually used when
searching for the environment variable.
Change-Id: I0bbd56969188389db81844d9276269464870f776
/tmp does not contain anything useful anyway, and excluding its content
makes the initramfs smaller too.
Change-Id: Ia72867e0cdebacf668ac1a1f551a965da0d69694
This adds support to UEFI secure boot by copying signed shim and
grub bootloaders into ramdisk image.
Closes-Bug: 1419707
Change-Id: I1193cd3a9011855a6804966a31c7c0e28da90ada
The newest stable Fedora splits out kernel modules into
a separate package. By default this is not installed in
the Fedora cloud image... and it contains some things we
need for Ironic (iscsi_tcp module) among other things that
might be very useful.
Change-Id: I3374ea278fecfeb6552e4664717ef3646d382c17
Closes-bug: #1429504
The other distro elements set DIB_RELEASE which allows the other
elements to know what distro release is being built during the
extra-data or environment.d phases.
Change-Id: I00bf13410ded5b678ebc66ff191891ed3cc80f4f
This is the centos element, which builds CentOS 6.
There are a couple of modifications to redhat-common because
the version of tar in el6 doesn't support --xattrs-include.
There is a change to both pkg-map and svc-map to add 'centos'
to the 'redhat' family.
Also explicitly have to install cloud-utils growroot and
dracut growpart for proper fsresize at instance launch.
Also sets the DIB_EXTLINUX variable because there is no grub2
for this distro.
Change-Id: Iffd57bce1484c43c2cffcbdb37b602185216e63a
The construction of the combined svc-map-services file uses the host's
/tmp directory to store a work-in-progress file. That file's left behind
after it's done with.
There's a potential race here; together with problems that may arise
if two users attempt to run devtest on the same system.
This patch fixes this by creating the temporary file specifically under
$TMP_MOUNT_PATH.
Change-Id: Iecbdc583e37bed542249c316919c3712c28c7440
This allows them to install packages already specified in .yaml files
for them, which are not installed otherwise since the migration to .yaml
files.
Change-Id: Iac8c7eb55116938616e5299b84487cd52e0cbf6f
openSUSE appears to have redone their fs layout on the repo site
and also changed the naming scheme of their image file.
Change-Id: I75afe54b88ea531d3c1f3b7d85e5435cdacabed2
subprocess.check_output() has been introduced in Python 2.7, so the
script will fail when trying to install stuff in guests with Python 2.6
and older (like RHEL 6 / CentOS 6, for example).
Thus gracefully fallback to subprocess.Popen() when
subprocess.check_output() is not available.
Change-Id: I335148397932177810f095a942b993b249991107
Closes-Bug: #1415240
A recent commit (I29c73355a85112840b57f93b39a9eeda421875e6) updated the
epel element to rely on wget, but wget is not always installed. Since
wget is used in pre-install.d, we must install it in that phase.
Change-Id: Iad2a9c4f0081835a1aa78e6fd2f721f6afd44dc2
Also make the apt-sources element exit with a non-error status code
when the DIB_APT_SOURCES is not set.
This allows including the apt-preferences and apt-sources elements
even if DIB_DPKG_MANIFEST and DIB_APT_SOURCES are not exported.
Change-Id: I8507dd9c69d1371eab38f720dcfdd89aa5345e8b
When there is no pkg-map file for the specified element, list the passed
packages if --missing-ok is specified.
This mirrors what is being done already later, when a pkg-map file
exists but it does not have the mapping for the requested package.
Change-Id: I50287f8e6a5e4335deb3de2252075b8bfdd53ce5
epel-release 7.2 has been superseded by 7.5, so update accordingly.
this time, detect the available epel-release package from the repo
instead of hardcoding version number which will break build each
time the version number gets bumped up.
Change-Id: I29c73355a85112840b57f93b39a9eeda421875e6
Pip now supports retries on failure. If setting multiple pypi mirrors a
fallback will not occure until the failures have finished for the first
mirror. This can cause a substantial delay if mirror fallback occurs a
lot.
Change-Id: Iad37a9015a2d5c861a345a111bd1725b965a42d3
We don't export DIB_RELEASE in the base distro scripts, so many
times it will be an unset variable in 51-bootloader from the vm
element. Currently that script doesn't handle the situation
properly and we get errors like:
line 160: [: =: unary operator expected
This change quotes the references to DIB_RELEASE so that won't
happen anymore.
Change-Id: Ic8f91804334f8862c217f465be80e96d3116f5c8
We have a good pattern of namespacing env vars with DIB_. Add support
for DIB_PYPI_MIRROR_URL* and maintain backwards compat support.
Change-Id: I434c9d1b14cd571b20754c9ad7cd64c936f8399a
As we started using jq it makes sento to use it everywhere when
building a resulting JSON.
Also removes some unneeded code.
Change-Id: Ib1391dc9f4e1463a9a3d0c13909ff60e3c993e82
When setting IPMI credentials is requested, ironic-discoverd
will return ipmi_setup_credentials=true, ipmi_username and ipmi_password
in response. Use ipmitool to set credentials in this case.
Change-Id: I8ace9e817e5ce9dbfb8564feec4a3ce6c0e1b233
There is a wide variety of tracing options through the various shell
scripts. Some use "set -eux", others explicity set xtrace and others
do nothing. There is a "-x" option to bin/disk-image-create but it
doesn't flow down to the many scripts it calls.
This adds a global integer variable set by disk-image-create
DIB_DEBUG_TRACE. All scripts have a stanza added to detect this and
turn on tracing. Any other tracing methods are rolled into this. So
the standard header is
---
if [ "${DIB_DEBUG_TRACE:-0}" -gt 0 ]; then
set -x
fi
set -eu
set -o pipefail
---
Multiple -x options can be specified to dib-create-image, which
increases the value of DIB_DEBUG_TRACE. If script authors feel their
script should only trace at higher levels, they should modify the
"-gt" value. If they feel it should trace by default, they can modify
the default value also.
Changes to pachset 16 : scripts which currently trace themselves by
default have retained this behaviour with DIB_DEBUG_TRACE defaulting
to "1". This was done by running [1] on patch set 15. See the thread
beginning at [2]
dib-lint is also updated to look for the variable being matched.
[1] https://gist.github.com/ianw/71bbda9e6acc74ccd0fd
[2] http://lists.openstack.org/pipermail/openstack-dev/2014-November/051575.html
Change-Id: I6c5a962260741dcf6f89da9a33b96372a719b7b0
Both ccache and linux-image-generic were being installed by
install-packages rather than using a package-installs.yaml.
Change-Id: Ib54c5b51d605278bb9d046a897be989c17c6fb0f
Patch adds support to create PowerPC image with vm element.
It creates 2 partitions, one for PReP boot and other for root and
installs grub-ieee1275.
Change-Id: I4675ef2b82aa69b63e63a1cc7db01b0c0e6f9fff
Closes-Bug: 1418739
If there are no old-style package-installs and the package-installs
element is included we fail because we call install-packages with no
arguments.
Change-Id: I3c78b51d0a2673ecc581f9faded078974dbc27b7
Since the package-installs-squash script relies on the defined
environment variables for figuring out what the values are for an
element's install type and the default install types, sudo -E must be
used to preserve the environment.
Change-Id: Ice5cf0bca65d8078f84daedbef123fc73d5a7b63
Working on host systems without selinux, where the guest image
does have selinux, creates a situation where the instance will
have about a 1 minute delay on first boot because it must relabel.
The previous check for sysfs assumes that the host system has
selinux, which is not needed for the guest setfiles to work.
Change-Id: Ic186a45991b6d80880ad635e9c80985612f53a05
Closes-bug: 1414200
This patch extends the deploy-ironic element to make it install a
bootloader on the disk in case the boot_option parameter passed in the
kernel command line is set to "local".
This parameter is set by Ironic as part of the
blueprint local-boot-support-with-partition-images
Change-Id: I5ebe6f364ae0ac408939399e5f28728b41c4766e
Patch adds support for PowerPC Big-endian and Little-endian local
fedora VM images and using VM images with two partitions, PReP
boot and root partition. Since PowerPC requires PReP boot
partition to be present.
Change-Id: Iba03226e187609df898732c13b1aa5f895b156e3
Closes-Bug: 1413487
ramdisk-image-create assumes the presence of vmlinuz file in OS image.
But ubuntu PPC LE images have only vmlinux file. This patch adds
support for using vmlinux file.
Change-Id: I5dd0b8ceb46b73be57d4c15b39b96f99b524fa3f
Closes-Bug: 1413362
When set to true this cloud-init setting overwrites the /etc/hosts
file at reboot, breaking stuff in TripleO where controllers rely
on resolving each other by name. We still want cloud-init to add in
there a mapping for the local fqdn, so we switched to 'localhost' [1]
1. http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/doc/examples/cloud-config.txt#L470
Change-Id: I85468dece3b67f39841aa826f9b7ca4bf113027a
Closes-Bug: 1411594
This commit adds http_proxy=$http_proxy to use apt-get update through a
http-proxy.
Change-Id: Iee589f6d11b84991512493d22f0e3dbfe60736ec
Closes-Bug: #1412314
The rhel-common element has a script that uses subscription-manager
to register an instance of RHEL with the customer portal or Satellite.
If os-refresh-config runs this script more than once,
subscription-manager returns a non-zero value causing os-refresh-config
to fail. Adding --force to the subscription-manager calls doesn't
properly unregister the image from Satellite and the result is multiple
content host registrations. This patch changes the registration to only
run once.
Change-Id: If372e7f59bbdc77e2b233024c1342f005004f2e5
Previous to the Sattelite 6 release, the katello-agent and it's
dependencies needed the rhel-7-server-rh-common-beta-rpms repo
enabled. Since the the release of Satellite 6, the "beta" can
be safely dropped. This fixes an error where registration to a
satellite instance fails due to a 404 for the beta repo.
Change-Id: Ia9879498c7db0c912bb58ea084d448b8b3915569
The dpkg element was hardcoding ~/.cache/image-create as the cache path,
but elements should use DIB_IMAGE_CACHE as the root for the cache to
allow users to override the path. Replace the hardcoded path with one
that is rooted at $DIB_IMAGE_CACHE.
Change-Id: I76030654e7dfc7b7d4dfa87e082c14e7cee479b9
