If we're installing pip/virtualenv from source, we need to make sure
we pre-install the packaged versions before the upstream
versions. Otherwise, CI jobs later on that depend on packaged versions
of pip/virtualenv can bring them in and overwrite the upstream
versions we have installed, which leads to a heck of a mess and
usually very confusing failures.
I have also moved in a small hack from system-config:install_puppet.sh
that we found was necessary when using pip versions from upstream.
Note this is not as much of an issue on Debian/Ubuntu, as they keep
their pip packages in a separate place to the system packages, so you
don't have these overwite conflicts as much.
Change-Id: Ib40708c07b939b84661c44df88a5a308fd0c7216
The initramfs file created by the ironic-agent element is
owned by the user running disk-image-create; ensure that
the other files created by the element are also owned by
the user.
Change-Id: I829db5b8e8bf1fc68face9cd2bda52d2a5ccdd4f
Closes-Bug: 1593010
This adds some fixtures to ensure the log output is what we expect,
along with ensuring we test the operation of the debug environment
variables.
Change-Id: I01c36299539b15a633b9307da5a348a5ae4e2563
This adds a common usable logging configuration for DIB, that
can be used by different python tools or elements.
This change is a factored out patch from the block device
refactoring.
Change-Id: I34d44610f74030936e7c0f602340b183baec3f1b
Signed-off-by: Andreas Florath <andreas@florath.net>
The latest Fedora/Ubuntu images don't ship python2 by default, so we
need to use our dib-python wrapper for this so we work in python3 only
environments.
Closes-bug: 1577105
Change-Id: I1048ceef35f269960216066924986eec6117ca00
This element takes up a *huge* part of the logs of openstack-infra
builds as we go and cache every git tree.
This silences most of the noise which will reduce the mess
in the logs considerably.
Note that we've had logging on this turned down since
I91c5e55814ba9840769357261d203f4850e2eba6 but it has been
ineffective in stopping the log-spew output, see the
dependency change.
Change-Id: I60f06f84d57087c82b3907575bff125015d35171
Depends-On: I1e39822f218dc0322e2490a770f3dc867a55802c
The temporary directories for image creation and building the OS both
started with 'image' as their names followed by some random
characters. During debugging this is annoying, because on first sight
it is not clear, where which files are stored.
This patch renames them to dib_build.XXXXXXXX and dib_image.XXXXXXXX.
This patch introduces no user-visible change:
the temporary directories are only used during the
run of disk-image-builder.
Change-Id: I249cdb7750fe9a746b375b462789cd9b82681a2e
Signed-off-by: Andreas Florath <andreas@florath.net>
There were a couple of functions which were unused:
ensure_nbd, map_nbd, unmount_qcow_image, mount_qcow_image, ensure_sudo
Because some of them use 'trap ... EXIT' this hinders introducing a
separate exit phase - therefore they are removed.
(It would also be impossible to use them in the current setup, because
they overwrite the 'trap ... EXIT' of the disk-image-creates 'main'.)
Change-Id: If932a557dca9aea4864154ad6c4f286373d6dd7c
Signed-off-by: Andreas Florath <andreas@florath.net>
It turns out our manual locale cleanup is causing issues (see
I54490b17a7f8b2f977369044fcc6bb49cc13768e). Upon further
investigation, I think this is a better approach than manually
deleting repos.
glibc on Fedora obeys the %_install_langs macro for reducing the
installed locales (as mentioned in the comments, F24 has moved to
having different packages, but worry about that later).
So our existing clear-out is really only required for CentOS, whose
glibc does not have any way to indicate to build less locales.
However, %_install_langs is still correct there, as it restricts some
of the translation files and other things installed with the %lang
macro in spec files.
This is complicated by us having to set this at glibc-common install
time, which happens with the "yum" from outside the chroot (i.e. on
trusty). Since this is too old to have flags to pass this, we need to
fiddle with rpmmacros.
I've tested this with fedora-minimal builds and the locales file is
about 2MiB, which is what it was after the cleanups, and the listed
locales are only those we expect (i.e. it appears to be working).
Change-Id: I528a68beeb7b2ceec25ccbec1900670501608158
Because Fedora 21 is EOL, the appropriate cloud images were
removed from the mirrors during the last days.
Because of the removal, currently all CI tests are failing.
This patch removes the Fedora 21 CI tests.
(Adding and supporting Fedora 23 is done with another patch -
because some additional changes are needed here.)
Change-Id: Ib85bb6fafd4f56ecc55dd420048f4d9e6e6969f3
Signed-off-by: Andreas Florath <andreas@florath.net>
I realised I'd been using die() in a few places assuming it was
available, but it wasn't exported. I guess it didn't matter because
whatever was wrong, we were failing anyway :)
This exports the function to make it available to sub-processes, which
should remove the need to source it as done in several places.
Change-Id: I7b9a5a6db406e160099b6ed9fde80455ae227327
If REPOREF="*", HAS_REF will be used without initialization. As -u is set
the script will terminate with error.
Change-Id: Ic1d88415adfef66dfc6c1d92610a45a9eb6359f3
Signed-off-by: Liu Qing <liuqing@chinac.com>
Allow file test to return all possible mime-types and ensure anything
that matches the python file type is parsed by flake8 instead of
relying on the first match returned.
Closes-Bug: #1585688
Change-Id: Iba31f1853537fe9234ab6f83d66f13dc1c578abb
The "set to localhost by default" behavior for manage_etc_hosts has been
deprecated for more than a year now by change
Ia8582883f737548e2911d3f36a1943e5b236281b.
Setting that value to "localhost" is still possible, but it won't be the
default anymore. If the previous behavior of assigning the hostname and
FQDN to 127.0.0.1 is still desired in some environments, it can be
achieved by setting the DIB_CLOUD_INIT_ETC_HOSTS environment variable,
as the deprecation warning message suggested.
Change-Id: I5a19d46e2f305769a0c89c9d25d2e6be02910221
Add a pkg-map to install cloud-initramfs-growroot on Precise, as it is
required there because the kernel is too old to use growroot on a live
file-system. This was generically removed in
e23b087505.
Change-Id: I5e6cbc4b74dc72a6f23f73a018c028f3ff7c0157
Add ability to filter in pkg-map on release, which defaults to
DIB_RELEASE. As per the examples, release is a more specific target
than distro, but distro still has to match.
Added a debug flag and ability to use a pkg-map directly for
debugging/development.
Change-Id: Ie282f96966e46236b06bc276de0168fc7a66c5da
Export FS_TYPE from img-defaults and use it to remove hard-coded
defaults in the debootstrap mounting. Also, cleanup the suse element
as it should have access to the exported variable.
Change-Id: Ie9b671ca9336060a5ad294be48aa7eff442bf066
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
For something fairly simple, I went back-and-forward with this a bit.
Firstly, I realise calling readlink constantly sucks. Due to the way
we call dib and source various files, you end up with the source-file
from "caller" being usually a very ugly path including levels of "../"
indirection. Cleaning this up to something canonical is the only sane
way to present it.
Because we evaluate _ps4() from a sub-shell in the PS4 string, there's
no way for it to do something like build a global in-memory cache in
an associative array or similar. It could write out a temp file or
some other side-band method, but the overheads of managing this don't
seem any different to just calling readlink. If anyone can think of a
bash-hack around this that doesn't involve a fork() I'm interested.
We could potentially strip some of the leading paths in the assumption
you know what they are; but it gets complex when things are split
across /usr/bin & /usr/lib and external elements, etc. I thought
about arbitrarily shortening it (e.g. just take last 20 characters)
which gives you enough of an idea of the file, but looks a bit ugly.
Or we could just leave the file-name out all together and assume the
function name is unique enough; this also seemed a bit ugly.
Obviously it's a matter of taste in the output. It is certainly
wider, but it also adds a lot of information. It also makes it fairly
clear where there are things we can make less verbose,
e.g. I1e39822f218dc0322e2490a770f3dc867a55802c disables tracing in
run-parts which is just noise. There's a few other frequently used
loops that we could disable tracing for by default to benefit
signal:noise.
tl;dr : take a look at the logs. I think it is a step in the right
direction of making the logs more usable for debugging.
Change-Id: I8054a3050415fcb527baeb7012bf133e5c864bf3
This reverts commit f07e33a2e9.
This change reverts the revert while fixing the underlying issue --
$PIPESTATUS needed to be encapsulated in ${}s
Change-Id: I1df06ffa7aecf4ea4b8e187dc756e9fc779786bc
This reverts commit 0d1d6bec7c.
This patch breaks tripleo-ci (the instack.qcow2 images is failing
to build) and was merged without passing CI.
Closes-Bug: #1582115
Change-Id: Ic4725ad0689c937fb4c8c792e1eaff5f4ea9ada9
Currently there is no way for a service to become aware that
dhcp-all-interfaces is finished configuring all the interfaces at
boot time. This causes problems for applications like the
ironic-python-agent which scans the interfaces when it first starts as
part of the inspection stage and can race against dhcp-all-interfaces
bringing up the interfaces, leading to inconsistent results.
This patch ensures that the dhcp-all-interfaces script runs before any
network interface is configured and brought up by the rest of the
system, and also ensures that the ironic-agent element also waits for
the network to be online before starting. This is done by using the
network targets provided by systemd.
Change-Id: Id9583b7f54361aa603a6229da598ad6a0f0f7938
I noticed we have no way easy way to get to the release notes. I
updated the front-page text to be a simpler introduction/overview and
provided links to the relnotes and source directly.
Change-Id: I5e339baf2921752ca3d409d82e0cbfc856ead1f8
I'm not sure why this comment is here. base works with centos-minimal
(we changed it in I2956aaa49ba3137a799f97e0983ab4b7c93a0a0c) and we've
got images deployed with such a configuration.
Update the comment
Change-Id: I3207f87dc29280183c0960ea863533a4d441081c
In phases which are called from eval_run_d (block-device.d) we do not
listen to exit 1's nor do we allow break=after-error. This is because
the run_d function is called in a subshell in order to grab its output.
This also turns on pipefail in the main disk-image-create script.
Change-Id: I88ab2e7104148437eabfe6880e3a1e5ebbb2c15d
