After the introduction of 'Add output for mis-configured element
scripts' we started seeing CI failures in tripleo where
instack-undercloud is being used (rocky/queens):
/usr/lib/python2.7/site-packages/diskimage_builder/lib/dib-run-parts: line 108: DIB_DEBUG_TRACE: unbound variable
INFO: 2019-12-02 16:24:33,423 -- ############### End stdout/stderr logging ###############
ERROR: 2019-12-02 16:24:33,423 -- Hook FAILED.
Let's make sure that by default the env variable is set
to 0.
Change-Id: I38c76c0edee436f1e7dd0c9a868cea1e6ee3271d
Closes-Bug: #1854904
I commonly get asked for help when people are attempting to create
local image elements and they cannot get them to work.
diskimage-builder silently ignores element scripts that it doesn't
find to it's liking, such as non-executable or files with extensions
(.sh is a common mistake).
This patch extends the '-x' tracing flag down to dib-run-parts and
will cause it to print out helpful messages when these files would
otherwise be silently ignored.
Examples:
Ignoring non-executable files: 10-do-not-run-me
Ignoring non-conforming filenames: 10-I-can-run.sh
I am not enabling these by default as they can create extra noise
and require additional filesystem IO to produce.
Change-Id: Ic804efca3015c199440b4b10da951d71a815c64f
This adds a devstack-inspired output filter to standardise
timestamping.
Currently, python tools timestamp always (timestamp setup in
logging_config.py) but all the surrounding bash does not.
We have extra timestamps added in run_functests.sh for our own
purposes to get the bash timestamps; but this ends up giving us
double-timestamps for the python bits. Additionally, callers such as
nodepool capture our output and put their own timestamps on it, and
again have the double-timestamps.
This uses a lightly modified outfilter.py from devstack to standardise
this.
All output is run through this filter, which will timestamp it. I
have removed the places where we double-timestamp -- logging_config.py
and the prefix in dib-run-parts.
An env option is added to turn timestamps off completely (does not
seem worth taking up a command-line option for). For callers like
nodepool, they can set this and will just have their own timestamps as
they collect the lines.
Since all logging is going through outfilter, it's easy to add a
--logfile option. I think this will be quite handy; personally I'm
always redirecting dib runs to files for debugging.
I've also added a "quiet" option. I think this could be useful in
run_tests.sh if we were to start logging the output of each test to
individual files. This would be much easier to deal with than the
very large log files we get (especially if we wanted to turn on
parallel running...)
Change-Id: I202e1cb200bde17f6d7770cf1e2710bbf4cca64c
It was an oversight during v2 development for dib to start providing
dib-run-parts. The intention was for dib to use a vendored
dib-run-parts directly from $_LIB and have no dependencies on
dib-utils at all. By exporting dib-run-parts, we created an
unintentional conflict with the dib-utils package which provides the
same script.
Tools that depend on dib-utils are unaffected by this
(os-refresh-config).
The only tool that installs diskimage-builder and then assumes
dib-run-parts is available in the path is instack. I have proposed
Ibfe972208df40fa092b11b5419043524c903f1b4 to modify that to use our
internal version.
Change-Id: I149c345d38d761a49b3a6ccc4833482f09f1cd05
Move dib-run-parts from dib-utils into diskimage-builder directly.
For calling outside the chroot, we provide a standard entry-point
script. However, as noted in the warning comment, the underlying
script is still copied directly into the chroot by the dib-run-parts
element. I believe this to be the KISS approach.
This removes the dependency on dib-utils. We have discussed this
previously and nobody seemed to think retiring dib-utils was going to
be an issue.
This also updates the documentation to not mention dib-utils, or using
disk-image-create via $PATH setup, but rather gives instructions on
installing from pip with a virtualenv.
Change-Id: Ic1e22ba498d2c368da7d72e2e2b70ff34324feb8
