Currently we have all our elements and library files in a top-level
directory and install them into
<root>/share/diskimage-builder/[elements|lib] (where root is either /
or the root of a virtualenv).
The problem with this is that editable/development installs (pip -e)
do *not* install data_files. Thus we have no canonical location to
look for elements -- leading to the various odd things we do such as a
whole bunch of guessing at the top of disk-image-create and having a
special test-loader in tests/test_elements.py so we can run python
unit tests on those elements that have it.
data_files is really the wrong thing to use for what are essentially
assets of the program. data_files install works well for things like
config-files, init.d files or dropping documentation files.
By moving the elements under the diskimage_builder package, we always
know where they are relative to where we import from. In fact,
pkg_resources has an api for this which we wrap in the new
diskimage_builder/paths.py helper [1].
We use this helper to find the correct path in the couple of places we
need to find the base-elements dir, and for the paths to import the
library shell functions.
Elements such as svc-map and pkg-map include python unit-tests, which
we do not need tests/test_elements.py to special-case load any more.
They just get found automatically by the normal subunit loader.
I have a follow-on change (I69ca3d26fede0506a6353c077c69f735c8d84d28)
to move disk-image-create to a regular python entry-point.
Unfortunately, this has to move to work with setuptools. You'd think
a symlink under diskimage_builder/[elements|lib] would work, but it
doesn't.
[1] this API handles stuff like getting files out of .zip archive
modules, which we don't do. Essentially for us it's returning
__file__.
Change-Id: I5e3e3c97f385b1a4ff2031a161a55b231895df5b
Installing Python to a ramdisk takes quite a long time because of
the way dracut checks for dependencies of every single file
installed. We could avoid that, but then we might miss a required
library file.
This change alters the installation method to speed up
the process. First, it creates a list of files that are needed and
then installs them all at once using inst_multiple instead of calling
inst on each file separately. This doesn't make a huge difference,
but in my testing it is marginally faster.
Second, and more significantly, we don't need the *.pyo and *.pyc
files as those are simply an optimization to speed up module
loading. Because the deploy ramdisk is a short-lived operation,
we probably lose more time transferring those extra files to the
target system than we save in improved load times.
In my testing, these two changes netted about a 20% improvement
in build times, and about 13% decrease in image size.
Change-Id: Ibc2b778c28fc9fb7177380dffe8dbce5722d0733
The targetcli element was triggering a bunch of errors from dracut
when we installed all of Python. It turns out this is because there
were filenames with spaces in the find output and the loop didn't
handle that properly. This switches to a while loop that can
handle odd filenames.
Change-Id: Iacbf16f26f2bc9991840250dc8ae7990db54d811
RHEL 7 does not ship tgtadm or tgtd so they cannot be used in the
deploy ramdisk. This change separates the tgt-specific parts of
the ramdisk into their own element, and adds a new one that supports
targetcli instead.
For now, the tgt implementation can only be used with traditional
busybox ramdisks and the targetcli one can only be used with dracut.
This is because dracut is primarily used for RHEL right now so it
makes sense to keep the dependencies simple. If there is a future
desire to mix and match the implementations that could be done, but
it would require users to explicitly select between tgt and
targetcli.
Change-Id: I4f99c91016287e08d836095c2f2261de8b45abdc
Co-Authored-By: James Slagle <jslagle@redhat.com>
