Move element-info from a wrapper script to a standard entry-point
console_script.
Update the documentation to explain how to run it for development. I
don't think we should support the idea that you can check-out the code
and run ./bin/disk-image-create -- it has dependencies (dib-utils,
etc) and needs to be run from a virtualenv (this is what CI in the
gate does). A follow-up can clean-up some of the path munging stuff
we have for this in disk-image-create.
Change-Id: Ic0c03995667f320a27ac30441279f3e6abb6bca8
Block device handling can be somewhat complex - especially
when taking things like md, lvm or encryption into account.
This patch factors out the creation and deletion of the local
loop image device handling into a python library.
The main propose of this patch is to implement the needed
infrastructure. Based on this, more advanced functions can be added.
Example: (advanced) partitioning, LVM, handling different boot
scenarios (BIOS, UEFI, ...), possibility of handling multiple images
(local loop image, iSCSI, physical hard disk, ...), handling of
different filesystems for different partitions / LVs.
Change-Id: Ib626b36a00f8a5dc3dbde8df3e2619a2438eaaf1
Signed-off-by: Andreas Florath <andreas@florath.net>
Using the explicit variables, these elements don't have to walk the
path to find the files in other elements; they can extract it directly
Change-Id: I0a64b45e9f2cfa28e84b2859d76b065a6c4590f0
These new variables are a list of elements chosen for the build along
with their full paths. For Python elements, IMAGE_ELEMENT_YAML is a
YAML formatted list that can be easily parsed. For bash elements,
"get_image_element_array" will produce an associative-array of the
same (working around lack of array export in Bash).
This list is intended for consumption of elements who need to copy
files from other elements, such as pkg-map and svc-map. As discussed
in I2a29861c67de2d25c595cb35d850e92807d26ac6, this list has already
been pruned and had overrides processed, so it is safe to simply walk
over this list with no further processing.
Since we're presenting the element list in a couple of different ways,
we combine it all into the element-info script. It will output an
eval-able string that declares the appropriate variables.
I've added some inline documentation so they still appear in grep.
The documentation is updated with examples, and moved to a more
appropriate location as a sub-section of the element sytle guide.
To test this out, use the associative-array in generate_hooks, where
we can now find the element's directory without searching.
Change-Id: Ibbd07d082ec827441def2d3f6240df3efdc6eae3
This is a re-factor of element_dependencies to achieve two things --
centralising override policy and storing path names.
Firstly we want to make the override policy for elements completely
explicit. Currently, elements that wish to copy parts of other
elements walk ELEMENTS_PATH themselves and look for elements in
IMAGE_ELEMENT. How they handle duplicate elements can differ, leading
to inconsistent behaviour.
We introduce logic in element-info to find elements in each of the
directories in ELEMENT_PATHS in *reverse* order -- that is to say,
earlier entries in the paths will overwrite later ones.
For example
ELEMENT_PATHS=foo:bar:baz
will mean that "foo/element" will override "baz/element", since "foo"
is first. This should be sane to anyone familiar with $PATH.
Documentation is clarified around this point and a test-case is added.
The second thing is that we want to keep the complete path of the
elements we have chosen. We want the aforementioned elements that
walk the element list to use these canonical paths to pickup files;
this way they don't need to make local decisions about element
overrides, but can simply iterate a list and copy/merge files if they
exist.
A follow-on change (I7092e1845942f249175933d67ab121188f3511fd) will
expose this data in a separate variable that can be parsed by elements
(a further follow-on I0a64b45e9f2cfa28e84b2859d76b065a6c4590f0
modifies the elements to use this information). Thus this does not
change the status-quo -- elements that are walking ELEMENTS_PATH
themselves and can/will continue doing that.
Change-Id: I2a29861c67de2d25c595cb35d850e92807d26ac6
Dependency to start network-pre (which
depends on network.target) before
dhcp-interface@.service collides with
Ubuntu's own network.target that suupose
to start after network-pre.
Change-Id: I9e59c970bfb1ebdaa15b4ec6b545761ede3ca056
Closes-bug: #1619816
IPv6 privacy extensions can cause issues by preferring a temporary
network over a public one. This preference may limit connectivity
in certain situations. An example of a connectivity issue can be
seen where the command ``traceroute6`` fails or misses all hops
while other traffic to a given domain with a "AAAA" record may
succeed. To resolve this issue the IPv6 privacy extensions have
been disabled.
Change-Id: I62b9d6301b9e8b8e93b49cecbc96334ceea92fa5
Related-Bug: #1068756
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
Currently, ironic-python-agent is installed without using an
upper-constraints.txt file.
This commits ensures ironic-python-agent is installed using
upper-constraints.
Change-Id: I6be6cfc012941e2cc9996717cba39b5415b85e14
Closes-Bug: #1616554
Explain difference between 'DIB_OFFLINE' and
'DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE'
Those variables are not redundant,they hava different effective ranges.
However,some people may be confused about this and reported a bug.
So,this difference should be writen in the README file.
Closes-Bug: #1506275
Change-Id: Ie5316de41d129bf98781708954f09ef0b2592b53
Currently we update portage whenever we could need it. Instead we
should update portage only if we actually need to. This update adds a
check to do so.
Change-Id: Ifdb27fd844b0b3a169ced945ac7ee0ddc235e9ec
Gentoo has updated it's grub ebuild to default to the upstream
recommended installation parameter of grub-mkconfig instead of our
default multislot installation of grub2-mkconfig. Update the command
line parameter so that it works with both.
Change-Id: I359b44338a4f76af7c026f5cad212e6dc3dbf2b3
There has been some confusion about what this environment variable
controls, and it isnt very clear in the docs.
Change-Id: Id21b3c5ce361c4d1121eb7015020235b4c0a2f36
Even though this file ends up in the /tmp directory, for readability
it's good to point out that server.pem is not necessarily the
certificate for a server, but can be a CA certificate which is
trusted if this option is used.
Change-Id: Iea27a702a844456e4472957438f75ed3819d62ca
For some use cases, it can be useful to keep all the kernels
and not just keep the latest one. Add a parameter that allows
it, and continue cleaning up kernels by default.
Change-Id: Ia6e6c1fa18e3724c1eb89226151d81e9e748b793
Kernels are built with auditing support, and without the audit deamon
logs bubble up to spam the console and /var/log/messages. This
package contains the audit daemon that catches these messages.
Change-Id: Ie3e216bab33b27f2d67a9379ddc3e89d66449251
Sometimes the secure_path option value in /etc/sudoers is surrounded by
quotes, in this case the current command creates an invalid entry and
it's not possible to sudo anymore.
This fix adjust the sed command to deal with possible quotes
Change-Id: Ifd6f9e29b3c0d04d6f65d3f55524ad202fb3294e
Optionally remove portage files, so that we can cache package and
keep the portage directory around, specifically for nodepool.
This also adds a section to the Gentoo readme about the variable
and renames the 00-gentoo-distro-name environment file to a more
appropriate name of 00-gentoo-envars.
Also brought up was the location of the gentoo-releng.gpg file,
this has been moved and the refrencing paths updated.
Change-Id: I20c91b36082828faa1ca481585acc5f9933211e1
Since the ironic-agent element builds the ramdisk and extracts the
kernel itself, there's no need to actually generate an image at the
end of the process. Previously the unnecessary image was being
deleted, but this wastes a bunch of time compressing and converting
the image. It's better to just not create the image at all.
This change adds a noop element called no-final-image that
disk-image-create looks for in the element list and, if found, will
cause it to skip the final image generation. This is more flexible
than the previous ironic-agent-specific method that would have
required changes to disk-image-create for every element that wanted
to behave similarly.
Note that this cannot be done using an environment variable, because
element environments.d entries do not propagate out to
disk-image-create. It also doesn't make sense as a user option
because it should be set by the element author, not the user.
Change-Id: I168feb18f0d578b3babbe4784d3ef75e755e1ebd
Most of the time,no useing no_proxy is ok,but sometime this will cause problem.
Add no_proxy here will increase the robustness of the program .
Change-Id: I976e689760d2e6de9e2081fcdee4f71299e8470e
Under some systems this leads to an error if the oder of parameters
does not comply exactly with the way it is specified.
Change-Id: I9561b85985e3f0917f1b4c7801c9048b4e73ae3b
Under some systems this leads to an error if the oder of parameters
does not comply exactly with the way it is specified.
Change-Id: Ie1ff871dfffecaf95e7ac467b18543561aaa0ceb
The proliant-tools element was missing a few
dependencies which were stopping it from
building correctly.
Change-Id: Ib7159a0baa7932d1571272cefffaf01d60e9debc
Closes-Bug: #1590176
This patch solves three issues with Debian packaging / apt:
o When building 'testing' only default apt sources is
included - backports, updates and security are skipped because they
do not exists.
o The default release for Debian was `unstable`: this is now fixed to
`stable`.
o Starting a Debian Stretch VM that was build with diskimage-builder
does not work, because some mandatory packages are missing.
This patch fixes this problem: it adds the mandatory packages and
the test case.
Change-Id: If49b5b162c4da1e074e9b19324839bc59d87dc57
Signed-off-by: Andreas Florath <andreas@florath.net>
We should be doing more to ensure initial configuration during
configuration. Taken from the steps done by [1], here we set
locale.conf and a general timezone.
The only reliable UTF8 locale is en_US.UTF-8; we don't want to use C
locale as it causes havoc with things like python3 and unicode. We
set locale.conf to this.
For Fedora 24 ensure we install the en_* locales too (this is really a
bug separate to this -- when you log in, by default ssh tries to copy
over your locale env variables, so logging into a F24 system would
result in using invalid locales for the most common en_* cases).
While we are here, setup a timezone link. It turns out infra puppet
overwrites this later, but at least we have a sane default.
[1] https://www.freedesktop.org/software/systemd/man/systemd-firstboot.html#
Change-Id: Ib8951a97f1772bc5228c682e88628ff53400a923
This reverts commit a645fa4ffb.
It is really devstack causing problems here; it was removing the
python-virtualenv package & re-installing using pip (see depends-on).
This failed because the pip-install we did here removed the egg-file
that rpm expected to be there, so rpm bailed out on the removal.
But even if it worked, this just leads you back down the path of the
original problem; that the system packaged version can be re-installed
and overwrites the pip installed version. Thus I still believe this
is the correct thing to do in the dib element.
Note it is not a common problem (devstack aside); most jobs don't
touch python-virtualenv & related packages (the one we did notice this
on was being brought over from travisci where it was required for some
reason).
Change-Id: I82acb865378a0fa5903a6267bfcee0e2962eced0
Depends-On: Ib0edf6c4ee8a510e9d671213de35d787f56acfed
