To create an auditable log of packages installed in an image use
dpkg-query to create a json data file with the package names, version
and install status.
Change-Id: I59018221edcb944a876ad47b402538c63c426bc7
The cloud-init package on Debian Linux overwrites the system
hostname to "(None)" unless a hostname is specified in one
of the /etc/cloud/cloud.cfg.d/* files. This change applies
a default hostname of "debian" using cloud-init.
Change-Id: I9b411df44cfa05f899e708900f95719c0bcd6267
Corrects an issue in the config_exists function that could
cause duplicate interfaces to be configured on Ubuntu.
Previously we used 'return ifquery' directly which caused
a silent 'numberic argument required' error to go unnoticed.
This would also return 1 meaning a new interface would get
configured.
The new logic should handle things properly.
Change-Id: I625225e15113d7e184e3bcb5054df1616dec008a
Closes-bug: #1298430
Virtual baremetal instances have their NICs show up as ens4
which doesn't play nicely as a default in devtest at
the moment.
Ideally we'd want it to be em1 like real baremetal
(which is why we put this in the baremetal element
to begin with). Turns out the baremetal element is
required in devtest so we can extract the ramdisk and kernel
so lets leave it alone for now and move this elsewhere
until we get a better solution for fake 'baremetal'
testing.
Closes-bug: #1298152
Change-Id: Ia71e1d32b93db0c4c844a6dc1ebcd04ab0c13f05
Currently the dpkg element fakes start-stop-daemon, initctl and
invoke-rc.d to stop dpkg from starting a bunch of daemons in the
chroot. This is problematic when packages use service, start, stop or
restart commands.
This patch uses a policy-rc.d instead of faking invoke-rc.d to achieve
the same thing. This approach now aligns exactly with debootstrap.
Without this patch DIB runs on some debian distributions fail
when trying to umount the chroot loop device as there are daemons
running.
The log will now show "invoke-rc.d: policy-rc.d denied execution of
start." instead of "Warning: Fake invoke-rc.s called, doing nothing."
Change-Id: I6db192127aca19b5b7915179b781f5192078bfc7
Related-Bug: #1211564
cloud-init packages are not available in the Debian Wheezy repository.
However, they are available in wheezy-backports.
With this change, the cloud-init packages are installed from
wheezy-backports if DIB_RELEASE is wheezy.
Change-Id: I3daa3b0ba185ba3f7c1bf6b8f9b1f8c167fcb340
Those parameters allow to access the console for VMs using
extlinux (CentOS 6 for example). They are already added to
Fedora images.
Change-Id: I035170bd101062e004fb3782b5f8e5a7944a4f03
Closes-Bug: 1296899
pypi-mirror creates a separate mirror index for wheels (one per OS
that mirrors are built on). To be able to use it one then needs to be
able to export multiple mirrors for inclusion in pip.conf. As a drive
by I made it possible to disable the use of the pypi.python.org index
without using --offline (as --offline has larger impact).
Change-Id: I3e85a8069b18cafd7eae4cd0591821acc3b5a739
Remove the correct ifcfg-eth0 file on Fedora.
This will require an associated incubator change as well
to ensure the undercloud/overcloud now use em1.
Change-Id: I4b4bb7b6a25b058eebe889fd8ce1a403aaf193fd
libmariadb-dev is named mariadb-devel on Fedora. mariadb-devel
package is needed for upcoming mariadb element.
Change-Id: Ib62071a964a4449c4e32ef1646ab7a0fb0f5f7b7
Update the systemd serial port element implementation
so that it uses udev rules. Much nicer than
hard coding ttyS0 and ttyS1.
The udev rule only adds real serial device types
(i.e. == 4). See:
define TTY_DRIVER_TYPE_SERIAL 0x0003
from include/linux/tty_driver.h.
Change-Id: I5d7de4815fd09c01693b7da4b8949f22fee20745
Updates the dhcp-all-interfaces element to fix a race
with the recent udev rules implementation on Fedora.
With the new approach we make the udev rule want (require
to startup) a generic dhcp-interface@.service template which
can be started individually for each interface that is
discovered.
The dhcp-interface@.service is setup such that it:
1) It calls dhcp-all-interfaces <iface> directly with
a pre-exec script. This creates the ifcfg file right
before we need it but avoids the case where network.service
might get greedy and try to start it itself.
2) Only runs if the ifcfg script doesn't already exist. This
is important because we only need to bootstrap the DHCP configs...
Once they exist the network.service will take care of starting them
on reboots, upgrades, etc.
3) On initial boot ensure that the initial DHCP interfaces come
up after network.service. Since we really only want
dhcp-all-interfaces to help bootstrap that haven't already
been configured this seems reasonable.
4) We also try to ensure that cloud-init
comes up after the DHCP interfaces. Cloud init has a decently
long timeout that this wasn't a functional problem but it keeps
log file spew down.
Change-Id: I71b026f027182aad49c3435bb903e5e38e524685
Closes-bug: #1294803
This is being added as a separate element because RHEL 7 is too
different from either RHEL 6 or Fedora to share all of the
configuration they do. It uses the redhat-common element for
pieces that are duplicated.
Some current limitations of the element:
-RHEL 7 is currently still in beta, so final image locations are
not available. This currently defaults to the RHEL 6.5 image,
but that will need to be overridden using the DIB_* env vars.
-There are additional tripleo-image-elements changes that will be
needed to be able to use this for building tripleo images.
-Open vSwitch packages are not currently available in the official
RHEL 7 repos.
-RHEL does not support qemu without kvm acceleration, so at this
time it cannot be used as a compute node for virtualized testing.
Change-Id: I705fd475303576525a27ce6801c673b5721792c9
Add an element intended for use in both Fedora and RHEL. This
allows them to share install steps that are common to both.
Change-Id: Ie4e820a7b777b8701514351b1f802cfe57c3812e
Refactors dhcp-all-interfaces.sh so that if an optional
INTERFACE argument (the first argument) is passed to the script
it only inspects that single interface. If no argument is
passed then the previous default behaviour is used which
causes all interfaces to be inspected.
To avoid a collision with the previous $1 we move to using
$FLOCKED for the exec flock command which runs on distributions
using ENI.
Also sets PATH so that the commands within the script
can all be found if it isn't set properly (/sbin/ip, /bin/cat, etc.)
This is a move towards using udev rules to add these types
of interfaces automatically.
Change-Id: I3ec8fd2cc2071bfc6943c744ca619e31b71146fc
Move whether the package or source install type is enabled out of the
source-repositories element and into the base element.
source-repositories wasn't a great fit for this functionality to begin
with.
Specify the install type for an element via the
DIB_INSTALLTYPE_<install_dir_prefix> environment variable.
Additionally with this approach, any new install type can be added
in an element, without having to update source-repositories or anything
in dib.
You could just write your install type implementation under
nova-foo-install, then define DIB_INSTALLTYPE_nova=foo in your
environment, and the scripts under nova-foo-install would get run during
the image build.
Source installs (any scripts under <install_dir_prefix>-source-install)
is the default install type for all elements.
Change-Id: I9414aca360c41e030e27d3d0c0a52d9d8e13d8b1
Since file to be fetched from the root tftp directory it shouldn't
contain any hardcoded path. Additionaly it eliminates bearing with
map-file containing regular expressions for tftp path replacement.
Change-Id: Iee1672834f735d7dfefce43000540522e1d053b9
Closes-Bug: #1291533
Some of the tripleo-image-elements were failing for RHEL due to lack of
build essential, python-dev and libz-dev.
This commit should fix those failure.
Change-Id: I080f9601faedb3259deada5cfbda7d00d3f13eb9
By default most cloud images have disabled the
udev rules which give us stable network interface names. While
this is a reasonable default for something like EC2 (the target
for these images) this is very bad behavour on bare metal
and can cause a "musical NICs" situation to occur where
on reboot the NIC names get swapped.
In this commit we re-enable stable interface naming by
removing the dev nulled rules softlink and installing
the biosdevname package. Once this is done interface
names should again show up as em1, em2 instead of
the old eth0, eth1.
We also remove some the common, statically configured
eth0 config files on some distributions. Equivalent
files (if needed) may be auto-generated by using
the dhcp-all-interfaces element.
Change-Id: Idd67977342719e2f295e2fa2c0ed6cfa8602171a
As git repositories are copied into the image make a note of their
details and add them to a manifest on the build system.
This allows the code used in building an image to be found after
the image build is completed without needing to open the image up
for inspection.
Change-Id: I7dbe9e163ad38a418cf2869a81e720de2c27dfb1
In 8b2325118f we added a line
so that 98-source-repositories tries to return from the top
level script (not from within a function).
Also, remove the return 1 within the function as well
because we just want to continue in these cases.
Change-Id: I66eddc12208b278594a0a8d8676c38d72045ca75
NFS client utilities for Debian are found in nfs-common, and the
Fedora equivalent is found in nfs-utils. The NFS server components
are already accounted for in the nfs-kernel-server mapping.
Change-Id: I2a43208f54b7fb625c239ce22231c43e194a97ed
Looks like shells only support alphanumeric characters and "_"'s in
environment variables. We were substituting "-" characters but in
order to be able to set overrides in projects with names containing
other characters we need to substitute more, I'm looking at you
oslo.config.
Change-Id: I3e2b1b0bc5871e4ec4ffd8117906cd077aa2cb0d
Co-authored-by: James Polley <jp@jamezpolley.com>
This will allow us to add environment variables to source-repository
scripts. Useful for things like arch-specific download urls.
Change-Id: I4d046825a317dc88db3d7673ad261cedacd1a104
This got removed accidentally in a recent consolidation of
install-packages. Adding it back so that -d support works again.
Closes-bug: #1287142
Change-Id: If17e613bc9bc0f808517edfa1506858d3b9dce54
The RHEL 6 Guest Image has been updated with a new release that
addresses a number of bugs. The diskimage-builder element that
references this file needed to be updated to reference the correct
image name. In addition, some notes to help users download the
image from RHN have been added to the README.
Change-Id: I3c9e64a1887afd9c0b370d70eff4b7f9598cebd1
Closes-bug: launchpad-bug-1286568
This patch alphabetizes the openSUSE packages dictionary in
map-packages. The openstack mappings were left
at the bottom as it looked deliberate.
Change-Id: I3661fba4d4ed68c836893e21fb4a8728c691af20
This patch alphabetizes the Fedora packages dictionary in
map-packages. The openstack mappings were left
at the bottom as it looked deliberate.
Change-Id: I4874c19f2a6826062874698e8f61bc6e2685ab19
Extlinux bootloader configuration in "vm" element requires vmlinuz and
initrd.img files in /boot/ with "generic" suffix. When these don't exist,
bootloader configuration should fall back to using whatever does exist.
Change-Id: I79d426e8f008e9f23db899f0b8f8cf2f23643e28
Testing showed that show-ref did not accurately determine whether a
git repository contained the given reference. Testing with name-rev
showed that it did.
Change the HAS_REF determination appropriately to avoid unnecessary
cache updates when using fixed references.
Change-Id: Iece213348516358af84417807f8dd36e959139eb
Reference to ${GRUB_OPTS:""} in 'vm' element causes syntax error
if GRUB_OPTS is set. Fix by changing to ${GRUB_OPTS:-""}.
Change-Id: Iae3ee8b7c4064c92d0cfc1b5a8caa22e53d6d112
We were applying proxy settings after apt-get install runs are done
in the Ubuntu element, which made it ineffective. Secondly, we were
not applying no_proxy settings to apt, which caused it to fail in
corporate settings. More explicit control may be facilitated in future
but local-config is a DTRT tool, and so we should help it DTRT.
I think I've correctly setup Zypper for this, and I have no idea
how to make the same improvement for Yum (and also Yum currently
doesn't set HTTPS proxies, which may lead to similar reports there).
Change-Id: Idbb9781d64d44b717ea22a25429383fa45d84148
Debian builds fail due to missing rsync in the baseline set
of packages that debootstrap defaults to. Rsync is used by a
variety of tripleo-image-elements and should be included in
disk images by default.
Change-Id: I8d83a53dfab279075e11f9825f15ee960395dd17
Add the DIB_GITREPOBASE variable to the source-repositories element
to enable global changing of git repository location.
Change-Id: Icfac5e103fce6e5a3f1a2db094144a0ed89fcc33
* Rename the cache directory for sources
Make the cache directory name match that of the element that creates
it. This makes it easier to find and more logically implies what
element created the cache directory.
* Add REPONAME to repository cache directory
For better usability add the reponame to the directory created
to cache it.
* Add migration code
Move pre-existing cache directories of the old name to the new name.
Change-Id: Ic62f19acb1746888ccbdf95eebf6e653f3e66357
The current method to rewrite the mirror URL in the ubuntu element will
only rewrite archive.ubuntu.com, which leaves the security URL alone.
This is probably not intended for people who are creating images behind
firewalls.
Change-Id: I54a867e0e9d5a7c94324a5c00ff1a83b4e6e4524
All of the yum-based install-packages scripts are doing essentially
the same thing, so let's use one script for all of the elements
that depend on the yum element.
Change-Id: I49e8c9b44e41bcf4cb9fa820e8a9179754694a97
We need to update vim-minimal, otherwise when rabbitmq-server gets
installed and pulls in vim-common, we run into:
https://bugzilla.redhat.com/show_bug.cgi?id=1066983
Change-Id: I910cd6c5c5fcfc6f6780b86029b1b4e660dd8d05
The pypi element is cool, but some folk have local network mirrors
which we should permit them to use.
Change-Id: Ie840ad1184e72b0e01966eee0298cfd6511b6c19
The ubuntu element supports simple replacement of sources.list by using
the DIB_DISTRIBUTION_MIRROR environment variable. Drag the debian
element into line by supporting both DIB_DEBIAN_MIRROR and the new shiny
DIB_DISTRIBUTION_MIRROR.
Change-Id: I9d5f8d1e3251965b34e55929182aa601a524fe8f
We can now support file:// URIs in source-repositories files and other
places, by ignoring curl's inability to determine an HTTP status code
for them.
Change-Id: Ie226ce05ecb912b23cedc73fd7ba53c0b9c0fe35
Updates the dhcp-all-interfaces element so that
we add new DHCP interfaces using a udev rule
instead of a systemd service.
The previous approach (using the systemd service)
was proving to be problematic when used with
the new Fedora constistent network naming
convention rules (em1, em2, etc.). We aren't using
these device names as a default yet... but we will
need to soon and when we do udev rules seem to be the
best way to avoid racy behaviour when calling
dhcp-all-interfaces.sh from the systemd service.
Change-Id: I87e0aed1f34718e896bd414388886a1f1073b0c0
Refactors dhcp-all-interfaces.sh so that if an optional
INTERFACE argument (the first argument) is passed to the script
it only inspects that single interface. If no argument is
passed then the previous default behaviour use used which
causes all interfaces to be inspected.
Also sets PATH so that the commands within the script
can all be found if it isn't set properly (/sbin/ip, /bin/cat, etc.)
This is a move towards using udev rules to add these types
of interfaces automatically.
Change-Id: Ia482c1d3ddce0f0d8d77f9bc3ac76d6924640715
Don't assume environment.d is always under /tmp/in_target.d. This makes
dib-run-parts more flexible, and less specific to diskimage-builder. For
instance, dib-run-parts is already used by os-refresh-config, but
/tmp/in_target.d/environment.d is never going to exist when it's called
by os-refresh-config.
This is useful b/c elements can now install os-refresh-config scripts
under /opt/stack/os-config-refresh/environment.d to influence how
further scripts are run. Specifically, the OpenSuse packages prepend the
OpenStack user accounts with "openstack-". Instead of checking for which
users exists with an if/else in all the os-refresh-config scripts that
need to do user related things, we can just update the scripts to use
$NOVA_USER, etc. The OpenSuse element can then install a file under
/opt/stack/os-config-refresh/environment.d to set NOVA_USER to
openstack-nova. We could have files that declare defaults as well within
each specific element, or just set a default variable at the top of each
os-refresh-config script.
Change-Id: Iadbfad995da657e2965fd55fc4ba3a88138b4cfc
Updates dhcp-all-interfaces so that we avoid using
ifquery on Fedora to determine if an interface exists.
Adds a new config_exists function which simply looks for
the ifcfg config file on distro's that use netscripts.
Change-Id: Ie55524b83820fe4fab28cc9d3f7e08ca2e42c182
With https://review.openstack.org/#/c/70721 installing
the dhcp-agent subpackage was introduced, which is needed
for openSUSE. It seems on Fedora this is part of the main
package, so we need to map it back to that one.
Change-Id: I80f8412d291b3571d220fd26cfece86a03479440
Just adding an element shouldn't make a disk image build crash, even if
we're not using the element for its intended purpose.
Change-Id: I2ec91ce4975b3b0deee7c85bb223f854c694553f
Adds the ability to set $DIB_LOCAL_IMAGE to use as the base cloud image
for a Fedora image build. There are many repetitve tasks that are done
every image build. With this change you can build an image with the
fedora and vm element and then reuse the resulting image as input to
future image builds. This greatly reduces future image build times.
For instance 99-up-to-date is already taking almost 2 minutes (even with
rpm downloads already cached) for Fedora 20, and is only going to keep
taking longer. By having a local up to date Fedora cloud image, this
time can be saved on each image build.
There is one minor change to support this in that /tmp/grub needs to get
cleaned up at the end of image builds so that the image can be reused.
Plus, there is no reason for it to stick around anyway. (didn't think
this was worth a seperate commit).
Change-Id: Ic74d138da922ecc99c38c27f105170d90009a84a
Otherwise some dependend libraries might be missing in the ramdisk. E.g.
on SUSE bash is dynamically linked against libreadline.
Change-Id: I8b36bee6aa5a1e1da419e748e8bba046c29f3ae2
/usr/bin/env is more portable (/bin/env does not exist on SUSE).
Also address minor nit on review about python 3.x compatibility.
Change-Id: I6bc1a579bb43664f4d2be35a96f45189a6df0e27
Rename generate-interfaces-file.sh to dhcp-all-interfaces.sh
so that it is easier to find/remember in the $PATH on a deployed
image.
Also, on some distros the script actually generates many files
so the previous name was a bit confusing.
Change-Id: I7152fa7c28e8ade251311da2cd5f75972423b66c
Updates the generate-interfaces-file.sh script in
the dhcp-all-interfaces element so that we only
add interfaces that have real MAC addresses.
The generate-interfaces-file.sh script is run early enough
in the boot process (before OVS is initialized) that this
isn't usually a problem unless you execute it manually by hand
after booting. Then you'll end up with network/DHCP
configs for all of your OVS bridges, etc. This
patch avoids configuring all of the virtual interfaces
which have generated MAC addresses.
Change-Id: I7a705084aa5b11305ac0ec5ca37fd2e87a2ae8b7
Closes-bug: 1239479
map-services is used by os-svc-restart from Change-Id:
I15eb2a5db4b0a08e1fb40bda640cd8f224939a92
map-services is not only distro specific, it also needs to be able to
handle different service names for a given distro based on source vs
package installs, since the service names may be different. This is done
via a simple check if the requested service is defined, if it is, we
print the requested name, otherwise, we print the mapped name.
Currently only implemented for Fedora. Support for other distros is
easily implemented just like map-packages.
Change-Id: Ie7b2dcfa7cabd887d5c212df110d90f9d00a7f65
The SELinux relabel of the filesystem is taking almost 2 minutes and
isn't needed unless you actually plan to run with SELinux enforcing.
Plus, it appears to "leak" out of the chroot, referencing filesystems on
partitions that aren't even mounted in the chroot.
Note you just can't use getenforce or selinuxenabled here to get the
state of SELinux because those commands are not accurate inside a
chroot.
TBH, a downside of this is that if someone goes to try to enable SELinux
in an image where it was built with it not enabled, the file contexts
are going to be wrong. So they'd need to relabel themselves at that
point. However, this saves me quite a bit of time during image builds,
so I thought I'd submit to get other folks opinion on it.
Change-Id: I2132060d573fc93cf974f3560fdc651ff8ba38b4
If creating an image of a disto without grub2 extlinux is instead used,
extlinux was being configured to mount the root filesystem as
read/write which in turn causes e2fsck to fail to test the filesystem,
on RHEL 6.5 this causes the boot process to fail.
Change-Id: I07859e0df9a2bc1ddd8c5f2ed4509ba14312e140
Delete stale symlinks created by current openSUSE 13.1
for kpartx loopback partitions. Those break bootloader
installation otherwise.
Change-Id: I13a7387e341b4426b459610fd1d3c49ce5730f46
openSUSE's ccache package doesn't include symlinks to enable ccache
for gcc by default, so we set them up manually.
Change-Id: I79dbe814ff2dfad8158fb1f06d68cc07eae19a41
A Red Hat Satellite URL may be provided as the source for
Red Hat packages. A Satellite key may be provided for a pre-
defined set of RHN channels to be automatically applied. Support
for installation of a Satellite certificate also provided.
Change-Id: Iae5b97d6f4732a28efa08ab778a56c2f9b6b6bd9
When registering to RHN or RHSM a list of space-separated
channels or repositories may be provided. This allows control
over which packages are available to the system during build.
Note, the optional channel or repository is added by default
since diskimage-builder requires access to its packages. This
list is for additional channels or repos.
Change-Id: I9cf90d47ab1d09f5491f574132410438f9e294e3
This provides basic support to register to the Red Hat Network
(RHN Classic) for packages. The image is unregistered from RHN
before completing image build. The rhnsd service is prevented
from starting since it is not needed for packagage install.
Change-Id: I2dc124cdc6ae59a63fd94d82d77108acccc3f668
Red Hat periodically updates the qcow2 guest image available
for download. This sets the default image name to the latest.
Change-Id: Iba3075bbee3b41918d5cd3da9721fcbf98ff3bcd
Rather than using a script to mount the image using nbd to extract the
kernel and ramdisk, make a new element called baremetal, which contains
a cleanup.d script that will copy them out to <image name>.{vmlinuz,initrd}.
Closes-Bug: 1224669
Change-Id: I8f3569aa12148d18b1c8242b6fbbd8857894b26f
Add a script to ubuntu's pre-install.d that will change
archive.ubuntu.com to a mirror specified in $DIB_DISTRIBUTION_MIRROR.
Change-Id: Idfd5a40ec8a1912a5de5dcdcaf9795946b0b53b8
Update the default version of Fedora to 20 for the cloud image. A few of
us have run through devtest using Fedora 20 and it's working.
To continue to use 19, just specify DIB_RELEASE=19 for your image
builds.
Change-Id: I3d48272dedcd424747db5485ae0d8e9953f04191
We need to use the eval command together with curl so that the
$TOKEN_HEADER variable is seems by curl as two arguments: "-H" and
"'X-Auth-Token: ...". Without the eval bash would interpret the variable
as only one argument "-H 'X-Auth-Token: ...'" making the curl command
to not understand that parameter and fail to pass the auth_token to the
Ironic API.
Change-Id: I4dcfc323d6ab9b7fa207328386ef65a146a93617
The source-repositories element will now symlink the install scripts
corresponding to the set install type for an element into the install.d
directory.
Different install types are implemented by elements by writing scripts to do
the install type under install.d/<element-name>-<install-type>-install
directories.
For example, the nova element would provide:
nova/install.d/nova-package-install/74-nova
nova/install.d/nova-source-install/74-nova
source-repositories will create the following symlink for the package
install type:
$TMP_HOOKS_PATH/install.d/74-nova -> nova-package-install/74-nova
Or, for the source install type:
$TMP_HOOKS_PATH/install.d/74-nova -> nova-source-install/74-nova
Change-Id: I1bfaf39e5a98b2af904fbc6d674dbada30b27ed9
dib-run-parts should dereference symlinks, and if they point to an
executable file, execute that file as a script. This can be accomplished
by using the xtype predicate in the find command instead of the type
predicate.
This change is needed so that we can dynamically symlink hook scripts
into hook directories such as install.d at runtime to support different
install types.
Change-Id: I933e7f4b4dcf16956841d8c14aa63b0f9a18fc5d
Upstart support in Debian is somewhat experimental, but having it would
be especially useful for TripleO which already generates upstart scripts
which will work fairly well on Debian based systems.
Change-Id: I267da5ae7bc4a484fac152a14963d96ca2ad2693
