Commit Graph

5 Commits

Author SHA1 Message Date
Ian Wienand
f068e6aa6e Fix package-installs-v2 output
The current output for package-installs-v2 is inscrutable [1]

The problem starts with process_output() which is not capturing
stderr.  This means that any stderr output is dislocated from any
stdout output around it.  This is *really* confusing as you get a
bunch of seemingly meaningless stderr output from any calls before you
see any stdout (e.g. in [1] you can see random yum error output that
should have been with the yum call)).  The simplest thing to do is to
redirect stderr to stdout which keeps everything in sync.

This causes a slight problem, however, because pkg-map outputs both
status information and errors on stderr.  To work around this but
maintain compatibility, we add a "--prefix" argument that prepends
mapped packages from pkg-map with a value we can match on.  The
existing status/debug output from pkg-map is low-value; modify the
call so that it will be traced only at higher debug levels (e.g. -x
-x).

The current loop is also calling pkg-map for every package in every
element (this is why in [1] the same message is repeated over and
over).  This is unnecessary; it only needs to pkg-map once for each
element, giving the package list as the arguments.  Create package
lists by element and pass those to pkg-map.

As a cleanup, there is no point in printing e.output if the
process_output fails for the install because we are already tracing
it; i.e. the output, even for failures, is already in the logs.
Printing it again just duplicates the output.

[2] is an extract showing what I feel is a much more understandable
log output for a fairly complex install.

[1] http://paste.openstack.org/show/595118/
[2] http://paste.openstack.org/show/595303/

Change-Id: Ia74602a5d2db032a476481caec0e45dab013d54f
2017-04-07 13:48:53 +10:00
Ian Wienand
7a155e08bf Merge branch 'master' into merge-branch
Change-Id: I28e4c7837d84e8b66eff3d182666c5a87a9e3c9b
2017-02-09 13:35:53 +11:00
Ian Wienand
b6e631360f Release 1.24.0
-----BEGIN PGP SIGNATURE-----
 
 iQEcBAABAgAGBQJYP+r7AAoJEBty/58O8cX8ZcMIAI7Yy2BfGqY/jfyLYNryATSu
 g78McMxAeOiPYII+KKyVkeWsChKJUafLZr/ppO95pm7ZLXeHQeC4MHDI68RmghIX
 HVXn4hPYzwlZ+Yr4hqaEf11H+vw0TDxo2a5bxlQZw+FCx9+3lkR60QxWf8a6QCxv
 nLqKIhtpwGi9mIyvAya+MaOmHMHqbNYgIBHKP0yXhTdxoHZoJWZeXN1rkelwAF9N
 TS1+KwIk+uemn1ZlqzYDWj8euyFyAv8L0WlF0ztCKe7TAFvNeMo26fLykeKHVDEo
 GC5R8MshiOtihPC972rNIponIzd62AetTlDxfK+gcUZO21pNTrlgdLX5Jn04fo0=
 =BpQc
 -----END PGP SIGNATURE-----

Merge tag '1.24.0' into merge-branch

Release 1.24.0

Change-Id: I23cfa6f8dc66bb6a1d957a49a5b468cfd47c005b
2017-01-31 14:14:19 +11:00
Ian Wienand
448a2602fe Merge remote-tracking branch 'origin/master' into merge-branch
Change-Id: I05cc51c699008018d557ed3874d71af26fd60240
2016-11-29 07:43:46 +11:00
Ian Wienand
97c01e48ed Move elements & lib relative to diskimage_builder package
Currently we have all our elements and library files in a top-level
directory and install them into
<root>/share/diskimage-builder/[elements|lib] (where root is either /
or the root of a virtualenv).

The problem with this is that editable/development installs (pip -e)
do *not* install data_files.  Thus we have no canonical location to
look for elements -- leading to the various odd things we do such as a
whole bunch of guessing at the top of disk-image-create and having a
special test-loader in tests/test_elements.py so we can run python
unit tests on those elements that have it.

data_files is really the wrong thing to use for what are essentially
assets of the program.  data_files install works well for things like
config-files, init.d files or dropping documentation files.

By moving the elements under the diskimage_builder package, we always
know where they are relative to where we import from.  In fact,
pkg_resources has an api for this which we wrap in the new
diskimage_builder/paths.py helper [1].

We use this helper to find the correct path in the couple of places we
need to find the base-elements dir, and for the paths to import the
library shell functions.

Elements such as svc-map and pkg-map include python unit-tests, which
we do not need tests/test_elements.py to special-case load any more.
They just get found automatically by the normal subunit loader.

I have a follow-on change (I69ca3d26fede0506a6353c077c69f735c8d84d28)
to move disk-image-create to a regular python entry-point.

Unfortunately, this has to move to work with setuptools.  You'd think
a symlink under diskimage_builder/[elements|lib] would work, but it
doesn't.

[1] this API handles stuff like getting files out of .zip archive
modules, which we don't do.  Essentially for us it's returning
__file__.

Change-Id: I5e3e3c97f385b1a4ff2031a161a55b231895df5b
2016-11-01 17:27:41 -07:00