Commit graph

2363 commits

Author SHA1 Message Date
Ian Wienand
c110a56ea9 Print unparsable file in pkg-map
As you can see in the CI failures for
Ib11b9df84b593ab25232729a570c812f1b4b8774, you can not see what file
is causing the problems when the parser fails.  Add a print, but raise
the error as it contains the cause.

Change-Id: I127ff7d57d2d898969195464c6e774d496e872e6
2016-01-14 16:21:25 +11:00
Gregory Haynes
cd9fdf05e9 Fill out bootloader pkg-map
Our bootloader install fails on non-gentoo builds due to missing pkg-map
for grub-pc. This map should really live in the bootloader element, so
move it there and fill it out.

Change-Id: Ib11b9df84b593ab25232729a570c812f1b4b8774
2016-01-14 15:31:50 +11:00
Jenkins
d5da803ff8 Merge "Prune old branches when updating cache" 2016-01-14 00:03:28 +00:00
Matthew Thode
176ae7bff6
Initial add of gentoo support for diskimage-builder
uses upstream's stage4 images, includes all the needed bells and
whistles for openstack on kvm.

Change-Id: Ibca43173c30c2a74a73a2e2d9dd6d6d832c62694
Closes-Bug: 1530911
2016-01-12 19:15:00 -06:00
Jenkins
023cfc3541 Merge "Add pip-and-virtualenv element" 2016-01-12 10:37:11 +00:00
Jenkins
8903a7bc65 Merge "deploy-ironic: Fix syntax error when checking for root device hints" 2016-01-12 00:55:49 +00:00
Ian Wienand
4573d0d930 Prune old branches when updating cache
dib builds are failing in nodepool at the moment trying to download a
really old Fedora image.  What happens is that the devstack cache
script [1] goes through all the branches of the devstack checkout and
uses the inbuilt image-list tool [2] to get its list of images to
pre-download.  One of the old Fedora images required on the retired
branches has now gone, so the build halts with a 404 trying to get it.

Thus make sure we do a --prune when we fetch changes so that old
branches are removed from the cache.

[1] https://git.openstack.org/cgit/openstack-infra/project-config/tree/nodepool/elements/cache-devstack/extra-data.d/55-cache-devstack-repos
[2] https://git.openstack.org/cgit/openstack-dev/devstack/tree/tools/image_list.sh

Change-Id: Ieb6a6e9f55bd93f63c3d0a71828c276c2d02e1b9
2016-01-12 11:25:43 +11:00
Abel Lopez
71bd8b3a33 Correct rhel-common for rhel6
Hardcoding subscription-manager to use rhel-7-server-rpms causes
users building rhel6 with Satellite6 to fail. This setting cannot
be overridden with environment variables, therefore needs to be
smarter. Setting RHEL_MAJ_VER in the rhel/rhel7 environment fixes.

Change-Id: Ifbd88bc76ef8b38a739272ba6e045a12849d68df
Closes-Bug: 1404364
2016-01-11 12:03:45 -08:00
Jenkins
beae0b21f1 Merge "Properly account for pipefail during cleanup" 2016-01-08 01:23:59 +00:00
Jenkins
14560600a6 Merge "Fix discoverd bug when dmidecode reports GB" 2016-01-08 01:22:54 +00:00
Jenkins
af5d3bd0c4 Merge "Make check for image-should-fail quiet" 2016-01-08 01:22:37 +00:00
Lucas Alvares Gomes
5064026b73 deploy-ironic: Fix syntax error when checking for root device hints
This patch is fixing a syntax error in the 70-ironic-root-device init
script for the deploy-ironic element.

Change-Id: I767486ca5893605720fba41bee3af72725a26377
Closes-Bug: #1531835
2016-01-07 12:06:19 +00:00
Gregory Haynes
db3ee03672 Add pip-and-virtualenv element
This element allows installation of pip and virtualenv from either
distro packages or git.

Change-Id: Id294f0936c8fef8a3b27a415bfcc93b3f327e104
Depends-On: I731cc8a0f5bfeda8f17a78c33b9f44062323a361
2016-01-06 18:39:19 +00:00
Gregory Haynes
d1e32f80a6 Run package-intalls with py3k if we must
Use dib-python to run package-installs using the provided python
version. Automatically detect the python version for our
package-installs-squash since that runs outside the chroot.

Change-Id: I926022bcf8cbcd81b051026ffd5d6477650045ad
2016-01-04 23:37:30 +00:00
Ryan Hallisey
2b28993fb8 The mirror for installing epel is timing out
Fedora has changed the location of epel, shorting the link
from 'download.fedoraproject.org' to 'dl.fedoraproject.org'.
This change updates the epel mirror to prevent it from timing
out.

Change-Id: I87090282a2f5f757495daec6ad14123b436b1aa0
2016-01-04 07:45:34 -05:00
Gregory Haynes
966f12dc4c Properly account for pipefail during cleanup
Due to a bug in how we were running some of our phases we were not
detecting some nonzero exit's. When this is fixed, dib fails early during
cleanup (leaving some resources attatched) due to not propertly
accounting for pipefail.

Change-Id: Icc0b35acbe035cac12a9291e2d07b6c690c3a6ad
2015-12-31 19:43:55 +00:00
Gregory Haynes
09888ebd5b Make dkms element depend on dkms package
It currently does not and it is, obviously, a needed dependency.

Change-Id: I0c94573bd0878a84fccb3c2675c0bcaeeb5088fb
2015-12-26 22:07:19 +00:00
janonymous
d2bc111d6a py26 is no longer supported by Infra's CI
Python 2.6 support would be dropped by
Infra team from mitaka,CI would no longer be testing it,
so projects should drop it too.

Change-Id: I916150ec5508a095fd850c9cb85dbeb464cdb826
2015-12-26 13:44:33 +05:30
Jenkins
f389b3a04d Merge "Document byte-to-inode ratio" 2015-12-23 16:40:30 +00:00
Joey D
802f14862c Fix discoverd bug when dmidecode reports GB
This fix uses dmidecode and awk to simply multiply by 1024 when
the value is represented in GB, otherwise it returns the given
value.  I should note that I've only observered this occurence
on "some" SuperMicro Hardware

Closes-Bug: #1486689

Change-Id: I352b1891326f72af3a56c7bbe8b7f3c422169404
2015-12-22 16:34:41 -06:00
Jenkins
59456684e7 Merge "Extend root device hints for different types of WWN" 2015-12-22 22:22:42 +00:00
Jenkins
7b20224cd6 Merge "Don't print trace unless trace is on in pkg-map" 2015-12-22 22:02:09 +00:00
Jenkins
3027093644 Merge "Package installs defaults to tracing off" 2015-12-22 22:02:03 +00:00
Jenkins
96d56efccc Merge "Fix growroot device detection" 2015-12-22 22:01:57 +00:00
Jenkins
b0eff0df07 Merge "yum-minimal : install selinux policy packages" 2015-12-22 20:16:11 +00:00
Jenkins
d450751617 Merge "yum-minimal: leave behind dummy /etc/resolv.conf" 2015-12-22 20:16:05 +00:00
Jenkins
441069bc17 Merge "yum-minimal: do not configure eth0 & eth1 for DHCP automatically" 2015-12-22 20:08:59 +00:00
Ian Wienand
fd2f55ee41 yum-minimal : install selinux policy packages
Install selinux policy packages as part of the base-installs.  selinux
is part of the base-system and the kernel boots by default in selinux
mode.

Without both of these, we can get in a situation where later scripts
(particuarly, some of the infra scripts) might install systemd-policy
without a base policy (targeted), leading to a messed up situation
where systemd will halt during boot due to missing policy files.

Change-Id: I6bf156304d1134fb328fba9b12dc364701b13696
2015-12-22 08:45:20 +11:00
Ian Wienand
1f499360fc yum-minimal: do not configure eth0 & eth1 for DHCP automatically
Add an environment variable to control the creation of eth0/1
interface enablement scripts.

With a tool such as glean, the presence of these scripts will indicate
the interface is configured and configuration-drive settings will not
be applied.  This means in a non-dhcp situation like on Rackspace,
network is broken.

On Fedora, where later systemd provides "predictable network interface
names" [1] eth0 & eth1 ironically aren't predictable so this just
confuses things.  You really need cloud-init or glean or something to
bring up your interfaces in a sane fashion.

This maintains the status-quo on centos-minimal, but disables creation
for fedora-minimal.

[1] http://www.freedesktop.org/wiki/Software/systemd/PredictableNetworkInterfaceNames/

Change-Id: I3f1ffeb6de3b1f952292a144efab9554f7f99a5f
2015-12-22 08:39:04 +11:00
Ian Wienand
5f3855f6f5 yum-minimal: leave behind dummy /etc/resolv.conf
As described in the comment, systemd will create a broken
/etc/resolv.conf link if there is no file in the base-image (as you
can read in the bug, it is debated if this is a bug or a feature).

The solution is to leave a dummy /etc/resolv.conf file in the image.
Whatever network manager you choose (NetworkManager, glean,
cloud-config, etc) will overwrite this anyway.

It's just that some tools, such as dhclient, get confused with the
broken symlink.  This affects you if you're using glean to configure
the network in a DHCP situation, for example -- dhclient won't
configure nameservers and everything goes to heck.

Change-Id: I734834d03e7fdb13f9ab2e86f877b07bf4a84ff9
2015-12-21 15:28:52 +11:00
Gregory Haynes
858fc3db30 Fix growroot device detection
We are incorrectly detecting major/minor device numbers for the growroot
rootfs. This can also be simplified by querying udev for partition
information.

Change-Id: I68059bf11f2563872f6b4d0e23fa09a15de980a8
2015-12-20 08:23:54 +00:00
Gregory Haynes
40799839e0 Package installs defaults to tracing off
There isn't a good reason we should be defaulting to trace on for these
scripts.

Change-Id: I1b9ac9388f51a49e6912d800914a0e0f2faec604
2015-12-20 00:03:51 +00:00
Gregory Haynes
6def86562a Don't print trace unless trace is on in pkg-map
The detection logic in pkg-map for DIB_DEBUG_TRACE assumes that this
variable being unset means tracing is on, when in fact this means
tracing is off.

Change-Id: I584a634c57bbe03e26a6ee94cef473e634616885
2015-12-19 23:50:15 +00:00
Jenkins
c8c14edfe3 Merge "Support building ACIs" 2015-12-18 13:53:38 +00:00
James Arendt
12cb94a75d Fix unmount/remove race in cleanup_build_dir
With a slow file system, umount can return 0 and the immediately
following remove can fail with a "Device or resource busy" error.
This happened in DevStack in disk-image-create where unmount_image
is followed by an immediate cleanup_build_dir.

Solution is to apply same logic from bug 1332521 to allow the
remove to retry on failure (up to 5s) in case the umount has not
completed.

Change-Id: I3337e2b4ad0111e77f79dc179439cdfea8ebdeda
Closes-Bug: #1527721
2015-12-17 20:55:02 -08:00
Gregory Haynes
0537563c27 Make check for image-should-fail quiet
We perform a check for this file after an image build fails to see if it
was intentional. This outputs a message which looks like the failure is
a result of the check when the file is missing which is very confusing
to users. Lets be more quiet about it.

Change-Id: I9b1f6c7c57021d11b4f91cd3c33846fae692b928
2015-12-17 02:31:41 +00:00
Abel Lopez
5e9c451d5f Document byte-to-inode ratio
End user docs would benefit from a section about the byte-to-inode
ratio, and why it's set the way it is. This update explains why
and how to manipulate the ratio depending on the intended use.

Change-Id: Iffb5ef6f4c7c74f4aa6e25912d4991d7a611c8fe
Closes-bug: 1512841
2015-12-16 13:58:17 -08:00
Jenkins
3a110bf96f Merge "Split vm and bootloader elements" 2015-12-16 16:22:56 +00:00
Jenkins
e6677afb8e Merge "Deprecated tox -downloadcache option removed" 2015-12-16 01:26:25 +00:00
Jenkins
a72d59c862 Merge "Fix fedora-minimal on CentOS builds" 2015-12-16 01:25:13 +00:00
Jenkins
134d38f74c Merge "Allow grub2 to build with opensuse" 2015-12-16 01:25:10 +00:00
Jenkins
d716893e30 Merge "Add dib-python element" 2015-12-15 23:24:25 +00:00
Yolanda Robla
c54b335fcc Split vm and bootloader elements
In order to add more flexibility to the vm and bootloader
elements, split the functionality in two different ones, and
make vm depend on bootloader element.
This will allow to construct more elements that depend on
bootloader, and develop both elements independently.

Change-Id: Iad2503b7b8fe53b768a3bc79e4cb839700fbd747
2015-12-15 20:49:03 +01:00
Ondřej Nový
b35f3a6c4b Deprecated tox -downloadcache option removed
Caching is enabled by default from pip version 6.0

More info:
https://testrun.org/tox/latest/config.html#confval-downloadcache=path
https://pip.pypa.io/en/stable/reference/pip_install/#caching

Change-Id: If624cea7246e0a16355923af122b2f8586718d5e
2015-12-11 22:58:50 +01:00
Jenkins
be5b669af1 Merge "Add a new element hpdsa" 2015-12-10 21:46:06 +00:00
Jenkins
3ebd0ba902 Merge "Follow up patch for 25d3ee547176528e86d42eb026c99a134dff9452" 2015-12-10 21:25:03 +00:00
Jenkins
aabf5b6ad7 Merge "Add proliant-tools element" 2015-12-10 21:24:27 +00:00
Nisha Agarwal
4509e6f826 Add a new element hpdsa
This element enables creation of Ubuntu deploy ramdisk and
user images which could be used to deploy the HP Proliant
Servers with Dynamic Smart Array Controllers. Without this driver
the disk with the Dynamic Smart Array Controller is
not visible to the ramdisk.

Closes bug: #1492803

Change-Id: Ibb3b298cd379cd7333279484df6ae30e9d7f6aaa
2015-12-10 20:12:14 +00:00
Gregory Haynes
3afbeeaf4e Add dib-python element
Creating an element which we can use in #! lines to refer to either
python2 or python3 depending on what it available.

Change-Id: Ic47e18ad21c33ab9f0d11c04260a33725aeee814
2015-12-10 19:39:38 +00:00
Robert Li
1b8a53ff3a Allow grub2 to build with opensuse
Specify grub2 in grub2 element pkg-map

Change-Id: Ia8bc1cbe11b35802f5c979ab4bb03e4e75077e83
Closes-Bug: #1519209
2015-12-10 17:29:52 +00:00