During internal testing we are getting the following error from
merge-svc-map-files
File "/tmp/image.K2MYCphY/hooks/extra-data.d/10-merge-svc-map-files", line 54, in main
with open(data_path, 'r') as dataFile:
IOError: [Errno 21] Is a directory: '...diskimage-builder/elements/svc-map'
Somehow IMAGE_ELEMENT contains a extra white space that manifests itself
as an element without a name. It is very hard to find where this is coming from so
instead this patch makes merge-svc-map-files more robost to this situation
Change-Id: Id1500ead8a77d691408617dcdc4e095bc5775be8
The element builds dracut from source on Ubuntu because the
Ubuntu dracut package is broken and very old, so it can't be
installed properly and causes a number of other issues that
are fixed by using a newer version of Dracut.
This initial version should work in virtualized environments.
Further validation of its suitability for real baremetal
deployments will need to be done in the future, but this should
be sufficient to enable that work.
Regarding Dracut specifically, in order to limit the changes
needed in the existing scripts this element continues to use a
cut down version of the /init script that we were building for the
existing ramdisk. However, instead of running it as pid 0 it is
run as a Dracut pre-mount hook. This allows Dracut to set up all
of the hardware and system bits, while falling early enough in the
Dracut sequence to complete the deployment before Dracut would try
to boot off the hard disk.
bp tripleo-juno-dracut-ramdisks
Change-Id: I144c8993fe040169f440bd4f7a428fdbe3d745cf
This reverts commit 4bf38a829a.
This variable is pointless since we can always replace it with
/lib/systemd.
Commit I459f7514ab35082d31607968252a9005fa25de2a will fix the issue
in tripleo-image-elements.
Change-Id: I524badb6836a0d04a5f6e6a5b5d95920fd923ef6
We are autodetecting and configuring devices in dhcp-all-interfaces
so having one configured in /etc/network/interfaces by default
is redundant and slows boot down.
Change-Id: Ic4e8a0668c793d21ed2dd96908649c9a77264f67
Closes-Bug: #1239480
After being deprecated two releases ago, finally remove any reference
for the support of first-boot.d
Change-Id: I08d67404ef48cad61db3b18fb86e970abfa5d2b6
In our official image builds we are only allowed to use resources
that are "blessed" by the build system. This means that external
things like git repos and tar files are not allowed. Currently,
even in offline mode source-repositories expects those things to
be available in the cache, so we need a way to disable it entirely.
This change adds an environment variable NO_SOURCE_REPOSITORIES
that does so. It can be set in an environment.d script so elements
that might rely on a source repository will know it's not available.
The 02-lsb script in redhat-common is one such example and is
updated to handle this case.
Change-Id: I0de63bee6ad79733d6711478c707a9b41593e85f
This is the final released image; it is the same one deployed in many
cloud provers such as HP and AWS. It is also safe from all known
vulnerabilities up to its release
Change-Id: Ib8ba561662d2a98da55eb292b850dc019226dd59
Each distro had its own copy of the file, but they were all the
same and it's actually installed in redhat-common so it belongs
there.
Change-Id: Iad15f39fd28d1c4e20d30dcb2eb0ae6f2fa2b015
When uploading images to multiple clouds it is possible that the same
image will be needed in multiple formats to accomodate hypervisors
across clouds. Update disk-image-create's -t flag to take a list of
desired output image formats so that a single disk-image-create can
output all of the desired image formats.
Change-Id: If121b2342ae888855ba435aa3189f039e985b812
The centos7 images come with the base version of cloud-init already
installed.
---
$ virt-inspector CentOS-7-x86_64-GenericCloud-GA-7.0.1406_01.qcow2
...
<application>
<name>cloud-init</name>
<version>0.7.5</version>
<release>10.el7.centos.1</release>
<arch>x86_64</arch>
</application>
...
---
I think this is a hang-over from early rhel7 images that didn't have
it.
Change-Id: Idf488942222d9e6bca4d66f04f5af42e6707944d
instack runs this script on a system where epel has already been
installed, which causes the yum command to fail. Only installing
the epel-release rpm when it isn't already present addresses the
issue.
Change-Id: Ia179d7c7ed69ea5f785b5dcd16394c7663a145cc
rpm-distro element was trying to install bin programs into the
image but it doesn't provide any. Move that logic into the yum
element which provides the install-packages bin.
Change-Id: Id5339fc7ffbef6327fe41f50a3e39ab30b6d0320
disk-image-create processes a DIB_IMAGE_CACHE variable and exports it,
but there are several elements that ignore the value and wrote out
the base location themselves. Use the variable everywhere so that it
will get overridden everywhere.
Change-Id: I00fff354d6c931ad67cf3052d055f0e4604dfdc8
The way redhat-common's extract-image script was creating the base
tarball caused file capabilities to get dropped, which meant that
things like ping in RHEL 7 images was unusable for regular users.
This change adds the necessary options to the tar call to maintain
as many extended attributes as possible.
--acls and --selinux are intentionally omitted, and the selinux
xattrs are filtered out because all of those items cause issues
in our chroot environment. We restore selinux attributes at the
end of the build anyway so that shouldn't be a problem.
bz reference: https://bugzilla.redhat.com/show_bug.cgi?id=1144149
Change-Id: Ibff99ce9bde01bc5ecf95dc3a5d3e2cebe5015b9
EPEL 7 was released September 30. The beta release rpm was removed
from the repo and was replaced with the final release rpm.
Change-Id: I103a2f7abd7563869b9e39ccc1fbb3cfbd9ff6ae
In both of the changed files in this patch, the $DISTRO_NAME usage
is checking against "centos". The valid value for the centos7
element for $DISTRO_NAME is "centos7"[1].
[1] http://git.io/8IvuTw
Change-Id: I0c4a4b2ecf87c7a9283d10071f6feb494a6c716a
A sanity check that cache-url does the right thing when presented
with the appropriate curl return codes. This change brings in the
test code for scripts from tripleo-image-elements, which needs to
be factored out into a place both projects can use it.
I'm stubbing out curl so we don't have a dep on curl in our unit
tests, and because I've seen some strange behavior out of curl in
the gate that caused random failures in this test.
Change-Id: I31e6b06b45415feec7285511d07e65eb78d0d045
The openstack-selinux package contains policy updates for OpenStack.
Updates are accrued in openstack-selinux before they are merged into
the upstream selinux-policy package.
Change-Id: I7c00e5cae17489202fe7aee3e656fca27ab630ba
As some finalise steps can install packages we need to generate the
dpkg manifest after that has occurred.
Change-Id: I2177db2e64d4d9c21deeac7cf017919888a2d524
Per [1], our current root label of cloudimg-root does not work with
XFS because XFS only allows 12 character labels. This change
addresses that by allowing a custom rootfs label to be specified
in the call to disk-image-create. There should be no backwards
compatibility concerns as the default label is unchanged. Any
external elements dealing with the label would need to be updated
to support this new feature, but should continue to work as before
as long as a custom label is not specified.
[1]: https://bugzilla.redhat.com/show_bug.cgi?id=1139584
Change-Id: I596104d1a63b5dc6549e8460a1ae3da00165ef04
Adds a new svc-map element which can be used to install
services based on an in element 'svc-map' YAML config
file format.
This change is intended to decouple elements from DIB
and allow new elements to support multiple distributions
(with various service naming schemes) without having to
constantly maintain DIB's various bin/map-services files.
This patch ensures all of the YAML files are compiled into
a single file without namespaces. The YAML compilation
process occurs during image creation time and errors if
conflicting mappings are found.
Change-Id: Id11433ea342aace71a358936a7ca3151ec11d506
In preparation for enabling Dracut-based ramdisks, this change
factors out functionality that is common to both busybox and Dracut
ramdisks. Said functionality is moved to a ramdisk-base element
which is added as a dependency of the ramdisk element. ramdisk now
only contains the functionality specific to building busybox-based
ramdisks.
bp tripleo-juno-dracut-ramdisks
Change-Id: Iad2907c8be491c88727d87ed5e5a720e5beb66c3
This patch adds a new EPEL element for use with EL-based distros
in accordance with the official docs[1]. This patch also
modifies the rhel7 element to not use the hard-coded repo file.
[1] https://fedoraproject.org/wiki/EPEL
Change-Id: Ib2fa24b2d519837271a0e213a2dc5a012cc31f3c
Update to newer Centos7 images as announced at [1].
Changes included:
* update to GA image
* respect TMP_DIR for image working directory
* use suffix-match rather than prefix-match to build RAW_IMAGE name
to be more robust against weird names
* partition layout work-arounds not needed any-more, all one
partition.
Resulting image booted locally with kvm and virtualbox (after
conversion to vdi)
[1] https://www.redhat.com/archives/rdo-list/2014-August/msg00045.html
Closes-Bug: #1363146
Change-Id: Ie24033468b78587ea87188ee1b843b26895798ff
Adds binaries specified by user to chroot environment
allowing to build images on amd64,i386,armhf,arm64
architectures.
Closes-Bug: #1332458
Change-Id: If6e63a472ee85559b93b5e6b96d9c3ddcf7bcc98
Due to changes in centos7 element to use package-installs the order
of yum-repos in pre-install.d needs to be changed, to make yum repos
available before package-installs is called to install packages.
Change-Id: I47b584506465d90247f30bad515a32323e52180e
When running inside a Docker container, we cannot rely on devices in
/dev/mapper to be automagically created by udev, because we probably
don't have a udev at all. To work around this, run dmsetup mknodes
after every kpartx run.
Change-Id: If7e30579224ce54c5ed26d08974d8293c144719a
15-pip-manifests depends on the variables defined by the manifests
element, so we need to run this one earlier. I would have thought things
were alpha sorted after the numerical sort...but I just ran into this,
so apparently not.
Change-Id: Ifedf544222b5a8eb7630efda609eb6a5b9629de3
When using the yum element, we should reset the changes we've made to
/etc/yum.conf during post-install.d. Otherwise, this build time
configuration is propagated into booted instances.
Change-Id: I1eea586ca0fefe9bc0cf91fedefcbd141a536fa2
deploy-ironic element currently retrieves token over tftp and
expects BOOT_IP_ADDRESS to be set. This commit adds support for
retrieving token file from virtual media if node booted from
virtual media. Also corrects the issue for BOOT_IP_ADDRESS not set
for virtual media boot.
Change-Id: I3d5f1779b9b17842360860c7778baa01db1e1a52
Closes-Bug: #1356339
Add "-e" option to remove packages. This is needed for
elements/package-installs/bin/package-uninstalls which calls
"install-packages -e".
Change-Id: I673942f505d5278e6015324950f6e1455ba50805
Updates additional elements in diskimage-builder to use the declarative
package install support provided by the package-installs element.
package-installs does not yet support the pkg-map functionality, so
elements using pkg-map are not migrated. That support will come in the
near future.
Change-Id: I3d36adad317ba44326eabd95243d45807e2a8a16
While building images in a dual boot host,
/dev/sda is leaking into build's grub config.
It is visible in the logs that the commands
GRUB-INSTALL and GRUB-MKCONFIG probes Host OS environment.
This is avoided by setting grub_disable_os_prober
to true in /default/grub config file.
Change-Id: I7408de7b3bc8a62f8bf103f00a88a596f988353c
Closes-Bug: 1166491
Use the latest available version of the rootfs (instead of a qcow2
image) to simplify rootfs extraction.
Change-Id: Ic3a646b923738464b5217d799758b6980b2deaf2
The previous method only configured serial console for Ubuntu or
Fedora, I noticed this wasn't working when I built RedHat 7.
After reading about how grub-mkconfig works, I realized that we
shouldn't be using sed to modify grub.cfg, and set up the proper
variables in /etc/default/grub
Change-Id: I02b73dc5074cb48a716849cb8a8a9b2f054bea31
For package installs we need to be able to fix up permissions in
/var/log at install time, and if we delete all of the files then
those permission fixes get lost. To deal with that, we can just
empty the files instead of deleting them.
Change-Id: I70abb354e4dc726ec7a176694c7c6e9455d89d08
This is more than a bit hard to test, requiring as it does an actual
iLO BMC, so sadly I have no tests :(.
HP Hardware really wants to be running latest firmware, and its
commonly agreed that one place where it can be sanely applied is in
the deploy environment, so this is my long threatened implementation
of that.
Change-Id: I3117a250d0d0eb8ee33eb4b15c837485a5cbf476
The pypi-mirror tool is no longer supported by the infra team, or
reccomended for installs, and is known to have various bugs that will
not be fixed.
Remove mentions of it from the docs and point people at devpi-server
instead.
Change-Id: Ieedb8bf9bee2f5932edac6b9747e645d0f4eb08c
Now that dib-run-parts has been moved to the dib-utils project, we
need to update diskimage-builder to use it instead of the version
directly in diskimage-builder.
This change removes the old copy of the dib-run-parts script in
the element, adds dib-utils as a dependency of diskimage-builder,
and updates the uses of dib-run-parts to correctly handle the fact
that it is now external to the project.
Requires I0be1f876d0e4a7d38e0d5c6010a552a8ebb158a4
Change-Id: Ia0a0df7784a14c49b5c47ac0b03e6c2602c84b3b
There are a bunch of optional env vars references in the rhsm
registration script, but they weren't wrapped to be set -u safe so
they cause build failures if any aren't set.
Change-Id: I18326b618f0920efa4dbd805611c6ee66d33ebb0
This reverts commit ac73660c40.
This was a backwards incompatible change - it broke the stable tripleo-heat-templates job.
Change-Id: Ideffca203db7b860b9f9d96c1d9f73e046519ec7
The existing support for tar files in source-repositories saves
only the contents within subdirectories contained in the archive.
This change makes the selection of contents within the tarball user-
definable based on the REPOREF specified (or overridden in the env).
A REPOREF of '*' (the current default, but deprecated) will select
the contents of subdirectories within the archive, while '.' will
select the entire contents of the archive.
For reference:
http://lists.openstack.org/pipermail/openstack-dev/2014-August/043197.htmlhttp://lists.openstack.org/pipermail/openstack-dev/2014-August/043249.html
Change-Id: I45db42ce66bf1d63d6ab5730090bf458b1b37ce9
An empty package list can be provided in a pkg_map file for those
cases were there is no need to install the package.
Change-Id: I0df8c6848ec4ac9e0f90790f14e8eb5399650339
Updates the fedora element to use the declarative package install
support provided by the package-installs element.
Change-Id: Ib3eb1936aa5c4efc350828e004794b0a8bd6c6c4
Packages are often also installed in both the pre-install.d and
post-install.d phases. This patch expands the package-installs element
to support declarative package support for these phases in addition to
the existing support for install.d. The actual install/uninstall logic
is moved to common scripts under bin/ so that it can be reused across
the different phases.
Change-Id: Id51d0bbad232737fc8b5ffaf016dec50cd5b66c9
I don't actually think there is a way that disable_interface
would get called on Fedora... because if config_exists
it takes a different path. Furthermore we really don't
want to mess w/ configured interfaces in dhcp-all-interfaces
because that could be side effecty (and is arguably the
job os os-net-config anyways).
This function already did nothing on Debian so removing
shouldn't hurt anything.
Change-Id: Ic53bbab98aa03d487efd290ae8e076045781b5b1
Relabel the filesystem during image builds if SELinux is supported
in the kernel of the build machine and userspace tools are available.
Otherwise touch /.autorelabel to schedule a relabel the first time
the image boots. We relabel when possible because it decreases first
boot time.
Change-Id: I0bec885d6e5d4f4e1106f3bd2a90ba5f86395b07
Partial-Bug: 1347845
Skip the contents of /tmp/ when creating tarballs with the results of
debootstrap, as that directory contains just ccache cache (and in
general temporary files).
Small (positive) side effect is that deboostrap tarballs gets slightly
smaller, something like 2-3 MB less.
Change-Id: Ib4adf3dc908f02deed408bf9188f032767eeb2bb
Just remove globally /lost+found after the root and install phases, so
distribution elements don't need to do that by themselves.
Change-Id: Ic783e613bd8794aefd3f40c9a7c308d14cd04b8d
Grub installation happens in finalize.d so cleanup should be done after this.
This reapplies I6322b8b529e31fd1b7251dd9e07cc85f442f7e81 after the revert,
but working.
Change-Id: Ie1873e64d407552da37a8dbedc13c3adbf79c085
Some extlinux packages install mbr.bin under /usr/lib/extlinux or
/usr/lib/EXTLINUX so tese directories need to be checked as well.
Also, there are packages that create a /boot/extlinux directory
which needs to be taken into account. Finally, commit
bccffc8bfd dropped the /boot prefix
from the kernel and initrd references in the ext/syslinux config
file which is simply a bug that needs fixing.
Change-Id: Idb071c9b18ff557b0f0f76d0d754536f2deca904
Closes-Bug: 1335042
This mapping is not needed anymore because mariadb-rpm element
(renamed from mariadb-rdo) uses pkg-map.
Relies on: I90e7ca4803516b86179b0f6639e307a60297960b
Change-Id: I19fb44ee6008c6c57dd331a780eec3234d46b15c
UBoot needs the kernel and ramdisk to be post processed
using mkimage utility. This element helps user do it as
part of the DIB process.
This element needs u-boot-tools to be installed.
The load address and entry point for UBoot kernel can
be specified as shown in the example below.
Example:
export UBOOT_KERNEL_ADDR=0x80000
export UBOOT_KERNEL_EP=0x80000
Change-Id: I0e1039bf57d1ef11a3f9831d82e24b1ec2136c13
This calls chroot as a non-root user, which will always fail with
EPERM.
This reverts commit ab2d1a31f2.
Change-Id: Ie674fef694ad66e1ebc22083dec2a0bc34371e7b
Includes "Don't match editor backup files in environment" and "Order
execution of environment.d scripts". These had to be squashed because
the first change alone changes the order env.d scripts are sourced in.
The current glob match for environment files can source editor backup
files (foo.bash~) which will override the real changes you have made.
Other parts use the regex to avoid matching such files, so do
the same for environment file matching. Note this has to match "."
unlike the other regex, as most env files are "foo.bash"
Also sort the files before sourcing them to ensure a reliable order.
This change should be in line with current expectations, given the
convention is to numerically prefix scripts in this directory.
Update of: I934486b3ff5884063d29c6d9b66fd9b11140464c
Subsumes: Icc509f695d7a15a8026d8c7e463f06acf65499d7
Change-Id: Ibfb562c5970b40598fc95da1e8d4beb9d51d7612
This change adds support in ramdisk element for recognising
boot from virtual media cdrom and read the parameters from
the configuration file in virtual media floppy.
Change-Id: I28ca888b4ead4905f7141b38ed4b9a3614d00d26
Closes-Bug: #1321564
This patch updates the RHEL7 element to work
properly now that it has changed from beta to
release status. It includes a repo update to
locate os-config rpms not in epel yet and an
update to the latest cloud image name.
Change-Id: I05607d62193c8f8fb1c1bece9872d456b19c5a08
There are some situations where the list of packages to install
in a image is blank (packages blacklisted).
Change-Id: I315f97fe619c25b36cf47ed1b7a65936b753312e
After running through pkg-map we could have no packages to install,
if so, don't attempt to run a malformed command.
Change-Id: Ia8e0aed62bcf814bf85c86b54ff0837da49ae7dd
We need to be able to do install.d like things for ramdisks
themselves, but install.d runs outside the ramdisk context - and its
likely to break peoples brains if we mangle the two together - so this
adds a new hook point, ramdisk-install, specifically for installing
things into the ramdisk.
Change-Id: I37d1660309cda6e28bd0b316b08f61db4e080613
Initial support for a centos7 image.
This is separate to rhel7 because the major differences are things
like repo and image locations, which are always going to be different.
We should merge any real changes into the redhat-common layers.
Apart from the added support files in centos7/*, the other change is
mostly modifications to redhat-common's extract-image to handle
different partition layouts of the centos7 image.
Change-Id: I943abe5ff0a803f36eda266a79af0d9220edcae7
The default value points to old image name
which results in 404 error. So updating the
value to point to correct name.
Change-Id: Ia66f0cf4f4167926892a0786467412d277ebffc3
This change adds support for retrieving the IP
through DHCP if 'ip' was not provided as a kernel
command-line argument by pxelinux. This is used
when the kernel/ramdisk is booted by virtual media.
Change-Id: I1097ce5b56ad40f2d6dc3181681d54f924ec7145
Closes-Bug: #1321563
Problems were seen when building images in parallel with
git clones and fetches interfering with each other.
Stop it.
Co-Authored-By: Michael Kerrin <michael.kerrin@hp.com>
Change-Id: Iaa48f259755d8b22327459a5c6d224a342909aee
