Avoid dangerous file parsing and object serialization libraries.
yaml.load is the obvious function to use but it is dangerous[1]
Because yaml.load return Python object may be dangerous if you
receive a YAML document from an untrusted source such as the
Internet. The function yaml.safe_load limits this ability to
simple Python objects like integers or lists.
In addition, Bandit flags yaml.load() as security risk so replace
all occurrences with yaml.safe_load(). Thus I replace yaml.load()
with yaml.safe_load()
[1]https://security.openstack.org/guidelines/dg_avoid-dangerous-input-parsing-libraries.html
Change-Id: I84640973fd9f45a69d2b21f6d594cd5bf10660a6
Closes-Bug: #1634265
As described in the bug, there are conditions with certain switches
in which the interface is 'admin down'ed during initialization.
Doing a 'cat' on /sys/class/net/<interface>/carrier when it is
'admin down'ed produces an 'Invalid Argument' error and the script
terminates. What this fix does is ignore failures of the 'cat'
operation (by '|| echo 0') and place the link up inside the retry
loop.
Change-Id: I4f098aa5078b8482681394a3e9a6b17ed4bd4451
Closes-Bug: 1654046
Xenial's bind of /dev into the chroot includes /dev/shm which is in
use by the host. An alternitive fix for this would be to use rbind
to recursivly bind mount /dev instead of just the base bind of /dev
Change-Id: I2c0f70afd1e82dd52a522f0dd2b3ea618b30b6c6
As noted in the bug, there may be circumstances where a longer
timeout than the current default is needed. This patch allows users
to tune this timeout for their environment if need be.
Change-Id: I173f3dad684894fbc3c27dece5ae15b5f63bae5a
Closes-Bug: 1654027
When we configure dhcp interfaces before network.target has run,
network.target will try to bring up those interfaces a second time
after our service does so. This causes two issues - first, the
network target will always fail because it can't bring up an
interface that is already up, and second, when configuring interfaces
that don't actually have an available DHCP server it will result in
a five minute delay waiting for DHCP on those interfaces. This will
also cause the network target to fail and is an unnecessary delay.
By moving the dhcp-interface service to run after the network
target we avoid both of these problems. network.target will still
bring up the interfaces on subsequent boots. This could result in
the five minute delay happening on reboots, but the expected use
case for interfaces without DHCP is that they would be configured
statically on initial deployment so this should be a minor issue.
The dhcp-interface service is also configured to run before the
network-online target so that services which depend on the network
actually being available will not race the DHCP process.
A snippet from /var/log/messages on a node with this patch applied
is included in the bug to demonstrate the behavior described above.
Change-Id: I5cfabf20f920beea52abf4c42362b6f6ac0b37c4
Closes-Bug: 1653812
The components documentation was previously referring to the
ramdisk image for deployment, which was previously deprecated.
Corrected to point to the ironic-agent element.
Change-Id: I770460041eb13523896aaadb7705bdc3db1a54ca
We landed the fix for this in
Icdb769541eee9793f261b4b8ec563be76ee13fe2.
This reverts commit 2978ff885b.
Change-Id: Iecfc41ab2aad57bc4f6f86a13810b534d19a8fd5
debian ships a modified site.py which has some interesting behavior when
VIRTUAL_ENV is set. In this case it will add
/usr/lib/pythonx.x/site-packages to the start of sys.path. This causes
pip to install packages to this location (rather than /usr/local). As a
result, later on when booting where VIRTUAL_ENV is not set this branch
is not hit and the path where python packages were installed is not part
of sys.path.
Change-Id: Icdb769541eee9793f261b4b8ec563be76ee13fe2
There are issues with pip packages and a python3 only Xenial systems.
This is occuring after Ie609de51cc5fcde701296c9474e315981d9778a2.
We believe the issue is with VIRTUAL_ENV being set within the chroot
and messing up pip installs
(Icdb769541eee9793f261b4b8ec563be76ee13fe2) but a full solution is not
yet clear.
For now, set Xenial to ensure we use python2. Install the package for
the ubuntu element (75-debian-minimal-baseinstall will install python2
for the minimal elements).
Change-Id: Id403919b0af93b375a900186c01a0d3a3bdfafea
Because we run this image in openstack-infra, we want to increase our
test coverage to help avoid potential breaks to our CI systems.
Change-Id: I26405e3f7465654075278ec35b5e0da1338bb45e
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Since we still run these 3 version of ubuntu-minimal elements in
openstack-infra, also run functional testing for them.
Trusty and xenial will be in voting gate, precise added as skipped for
non-voting.
Add the default skip/run status to the "-l" output just to confirm
this too.
Change-Id: Icfbfd0cb7d9acae824972474b77e2fe0486c4f69
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Set the grub timeout to 5 seconds by default, and add notes on how to
update this. This will stop infra having to carry an element that
goes and rewrites the grub configuration.
Change-Id: I556b3f48eff1b67ee8c4b9b64f749af95100fb99
Our setuptools action classifiers are woefully out of date, notably: we
are no longer alpha and we support python3.
Change-Id: I2425152129406e22073936275761bd5d850903fb
The squashfs format brings a couple of advantages over the other
formats. Image is often an order of magnitude smaller and it can
be used natively, either as an initrd, either with loop mount.
Change-Id: If72940b0c4dafb2504c52dd0429a8eb3f8305751
We now support tgz (tar.gz) as an output format.
Change-Id: Iadec92f2f96c3f904f28bd49f87ffc7d48ef7bd7
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
dracut has a "hostonly" mode where it builds an initramfs that is
suitable for booting the system it is building on. This is on by
default, but obviously in our nested multi-platform chroot situation
this is fraught with danger.
As highlighted by [1] our builds were inadvertently turning off
"hostonly" mode when the mountpoints in the chroot were not found.
The CentOS 7.3 behaviour change broke this and we ended up with an
initramfs with no file-system modules.
Iaf2a1e8470f642bfaaaad3f9b7f26cfc8cc445c9 introduced a regeneration of
the initramfs, which I think does work as described because it runs in
the loopback device.
However, dracut includes a package that installs configuration
overrides to build a generic initramfs. This is really what we want,
and should solve the problem no matter where the initramfs is created.
Add this package into yum-minimal and remove the extra re-create call
which should not be necessary.
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1405238
Change-Id: I5d203f2abe743cb23a44d449850e692a948e7871
openSUSE 13.1 was discontinued on Feb 3rd, 2016, so defaulting
to it doesn't make sense (see https://en.opensuse.org/Lifetime).
Leap 42.2 is the most current release that is supported by
disk-image-builder and being tested in a 3rd party ci.
Enable functests for it to ensure we're not regressing again.
Moved to non-voting gate first.
Depends-On: Iff495b3cd0b6c3558c44cf4883651eca67b572d6
Change-Id: Iae6cd34a5853f1e309861c554d94d8595cbd9993
For some reason [1] introduced -m option without ever checking that the
mapping exists. Because there is no grub-ieee1275 mapping anywhere (not
in base, not in bootloader), pkg-map fails. So stop using the mapping in
package-install of grub-ieee1275 on ppc.
There is another patch that tries to solve the same bug by adding the
mapping [2]. I think it is better to undo the breakage introduced in [1]
first, and then, if various distributions have differing names for the
package, introduce various mappings. My reasoning is that at the moment
this element is broken for all ppc64 distributions. This patch would
fix it for some (namely, Ubuntu). Then we can add mappings as tests
are done for other distributions.
[1] Ibca43173c30c2a74a73a2e2d9dd6d6d832c62694
[2] Id2b0f63a7015f883070fd59b79fd96a1c024858a
Change-Id: I8425876c26e9e416c8ce2f53a4e38d26b4208633
Closes-Bug: #1624021
dracut has a loop [1] where it probes top-level directories, tries to
find what block device they are on, then determines the file-system of
that block device. It then puts those file-system modules into the
initramfs for boot.
Since we install the kernel package during the chroot phase, / there
is not a block device and thus this loop matches nothing and we end up
with no file-system modules in the initramfs. This results in a very
annoying silent boot hang.
By moving re-generation of dracut into finalise.d phase, we run inside
the final image where / is the loop-device; the root file-system gets
detected correctly and the ext4 module is included correctly.
[1] http://git.kernel.org/cgit/boot/dracut/dracut.git/tree/dracut.sh?h=RHEL-7#n1041
Change-Id: Iaf2a1e8470f642bfaaaad3f9b7f26cfc8cc445c9
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Tripleo-image-elements have an install.d file '05-heat-cfntools' that runs
the following command:
virtualenv --setuptools $VENV
With the recent change to diskimage-builder (moving the install of pip
and virtualenv to the 10- range) virtualenv is no longer available for
this elementr; as a side-effect, the trove kick-start command is now
broken and gate jobs are failing.
The solutions is to move the (now) 10-install-pip to 04-install-pip.
This should still alleviate the race condition that
https://review.openstack.org/#/c/408277/ attempted to fix, as all
*-package-installs files are 00-, 01- or 02-.
Change-Id: Ia4e01f00c4c5e9a2087df1e2a91d9154480a0422
Closes-Bug: #1650008
