Commit Graph

1247 Commits

Author SHA1 Message Date
Robert Collins
c7875398b3 Fix resource exhaustion with upstart.
When high VM churn is encountered dhcp-all-interfaces got tangled up
and leaked instances until upstart run out of filedescriptors.

This happened because we were locking on every invocation, and the
lock logic was disabling the single-interface code - so we always
probed every interface every time - and this was serialized across
every instance of dhcp-all-interfaces.

Change-Id: I4ef1fb83d978cc99d4cb9cc2666bde302c6db7d1
2014-04-01 22:45:12 +13:00
Jenkins
6090f520d1 Merge "Fix "(None)" seed hostname on Debian" 2014-03-31 15:41:08 +00:00
Jon-Paul Sullivan
843085a7cd Write a dpkg manifest to list installed packages
To create an auditable log of packages installed in an image use
dpkg-query to create a json data file with the package names, version
and install status.

Change-Id: I59018221edcb944a876ad47b402538c63c426bc7
2014-03-28 18:12:02 +00:00
Jenkins
b9189a204f Merge "Uses policy-rc.d to prevent dpkg starting daemons" 2014-03-27 18:38:06 +00:00
Jenkins
eacdb2a4a5 Merge "dhcp-all-interfaces: correct ifquery return stmt" 2014-03-27 18:01:52 +00:00
Jonathan Brownell
5af25b5f46 Fix "(None)" seed hostname on Debian
The cloud-init package on Debian Linux overwrites the system
hostname to "(None)" unless a hostname is specified in one
of the /etc/cloud/cloud.cfg.d/* files. This change applies
a default hostname of "debian" using cloud-init.

Change-Id: I9b411df44cfa05f899e708900f95719c0bcd6267
2014-03-27 10:08:03 -07:00
Jenkins
5feeb06738 Merge "Make stable-interface-names its own element." 2014-03-27 16:14:08 +00:00
Dan Prince
6f764d9119 dhcp-all-interfaces: correct ifquery return stmt
Corrects an issue in the config_exists function that could
cause duplicate interfaces to be configured on Ubuntu.

Previously we used 'return ifquery' directly which caused
a silent 'numberic argument required' error to go unnoticed.
This would also return 1 meaning a new interface would get
configured.

The new logic should handle things properly.

Change-Id: I625225e15113d7e184e3bcb5054df1616dec008a
Closes-bug: #1298430
2014-03-27 10:47:55 -04:00
Jenkins
54d16fa9f8 Merge "Adds package mapping for mariadb packages" 2014-03-27 13:30:14 +00:00
Jenkins
57ff15ea80 Merge "rename udev.rules to dhcp-all-interfaces-udev" 2014-03-27 07:24:40 +00:00
Dan Prince
99c54323a0 Make stable-interface-names its own element.
Virtual baremetal instances have their NICs show up as ens4
which doesn't play nicely as a default in devtest at
the moment.

Ideally we'd want it to be em1 like real baremetal
(which is why we put this in the baremetal element
to begin with). Turns out the baremetal element is
required in devtest so we can extract the ramdisk and kernel
so lets leave it alone for now and move this elsewhere
until we get a better solution for fake 'baremetal'
testing.

Closes-bug: #1298152
Change-Id: Ia71e1d32b93db0c4c844a6dc1ebcd04ab0c13f05
2014-03-26 22:25:39 -04:00
Jenkins
f93147795c Merge "debian: install cloud-init on Wheezy" 2014-03-26 23:28:39 +00:00
Thom Leggett
46a14ae0e6 Uses policy-rc.d to prevent dpkg starting daemons
Currently the dpkg element fakes start-stop-daemon, initctl and
invoke-rc.d to stop dpkg from starting a bunch of daemons in the
chroot. This is problematic when packages use service, start, stop or
restart commands.

This patch uses a policy-rc.d instead of faking invoke-rc.d to achieve
the same thing. This approach now aligns exactly with debootstrap.
Without this patch DIB runs on some debian distributions fail
when trying to umount the chroot loop device as there are daemons
running.

The log will now show "invoke-rc.d: policy-rc.d denied execution of
start." instead of "Warning: Fake invoke-rc.s called, doing nothing."

Change-Id: I6db192127aca19b5b7915179b781f5192078bfc7
Related-Bug: #1211564
2014-03-26 20:59:49 +00:00
Ben Nemec
e6e5076698 Enforce alphabetical ordering of element-deps
We've started to require this in reviews, so we should really have
automation in place to catch it right away.

Change-Id: I43fd90647acba400cea11c665fb587856514b0ee
2014-03-26 15:23:18 -05:00
Ben Nemec
63230414d2 Alphabetize all element-deps
Change-Id: Idc31d60daeea61d6df0b70dac4a5671fa5a804ee
2014-03-26 15:23:18 -05:00
Ben Nemec
f961d745e3 Add dib-lint script
This will provide a place to put checks that catch common errors
in elements.  To start, this just checks that files starting with
a shebang are chmod +x so they can actually be run.

Change-Id: I4116a8f38f7bdfc5866764354c459fad8ca18e92
2014-03-26 15:23:05 -05:00
Jan Provaznik
507b717c61 Adds package mapping for mariadb packages
Change-Id: I6fb0e176da3f73cb2388cd833395f5880c4ce3c3
2014-03-26 21:12:58 +01:00
Trent Geerdes
8007882726 Adding pypi-mirror dependencies for redhat
Change-Id: I030f27460c92ce0b59188e9469206b293271697a
2014-03-26 06:44:15 -06:00
Derek Higgins
85dadd0d90 Revert "Only create a tmpfs big enough for DIB_MIN_TMPFS"
This didn't pass CI and is now causing failures as it moves us back to a
4G tmpfs.

Change-Id: Ie92fd909ad21216b1fda55c21457893df38e882f
2014-03-26 10:07:41 +00:00
Jenkins
a78e33f617 Merge "Add console kernel parameters to extlinux configuration" 2014-03-26 09:47:40 +00:00
Jenkins
0749cf4d11 Merge "Adds "element-provides" support to element dependencies" 2014-03-26 09:30:27 +00:00
Gonéri Le Bouder
2514b822d2 debian: install cloud-init on Wheezy
cloud-init packages are not available in the Debian Wheezy repository.
However, they are available in wheezy-backports.

With this change, the cloud-init packages are installed from
wheezy-backports if DIB_RELEASE is wheezy.

Change-Id: I3daa3b0ba185ba3f7c1bf6b8f9b1f8c167fcb340
2014-03-26 10:21:13 +01:00
Jenkins
bfba0ccd03 Merge "Remove an excess cp of disk images." 2014-03-26 08:38:52 +00:00
Jenkins
73c3b2c404 Merge "serial-console: Use udev rules to startup getty" 2014-03-26 05:33:52 +00:00
Jenkins
25615b6403 Merge "Only create a tmpfs big enough for DIB_MIN_TMPFS" 2014-03-26 04:00:23 +00:00
Jenkins
440de95f41 Merge "Set +x on executable files" 2014-03-25 20:54:41 +00:00
Jenkins
593680e936 Merge "DHCP: make udev rules want dhcp-interface@.service" 2014-03-25 20:23:14 +00:00
Robert Collins
dc93feee69 Remove an excess cp of disk images.
save_image is used to copy kernel and ramdisks out of the image, which
we will sometimes want to keep the source, and sometimes not. However
for the main image itself, the temp copy is never kept, so use mv
  rather than cp and avoid the excess IO.

Change-Id: I5a9f0d69ffee3e6b872a8927537ac17f02f5aa4d
2014-03-25 17:20:37 +00:00
Jenkins
8c5aff82a2 Merge "baremetal: correct the path of ifcfg-eth0" 2014-03-25 15:50:31 +00:00
Jan Provaznik
604926b38f Adds "element-provides" support to element dependencies
An element can define a list of elements provided by itself. This allows
using an alternative element and keeping element dependencies at the
same time. This will be needed for example for mariadb element which will be
an alternative to mysql element - mysql dependency can be kept everywhere
but user can choose mariadb explicitly as a replacement by putting "mysql"
into "element-provides" file.

Change-Id: I7f193612f279dd71d9f1f1edfc60009838a00c5a
2014-03-25 15:35:52 +01:00
Luigi Toscano
be3efed80b Add console kernel parameters to extlinux configuration
Those parameters allow to access the console for VMs using
extlinux (CentOS 6 for example). They are already added to
Fedora images.

Change-Id: I035170bd101062e004fb3782b5f8e5a7944a4f03
Closes-Bug: 1296899
2014-03-25 14:24:43 +01:00
Robert Collins
ee18fe41da Permit use of wheel mirrors in pypi element.
pypi-mirror creates a separate mirror index for wheels (one per OS
that mirrors are built on). To be able to use it one then needs to be
able to export multiple mirrors for inclusion in pip.conf. As a drive
by I made it possible to disable the use of the pypi.python.org index
without using --offline (as --offline has larger impact).

Change-Id: I3e85a8069b18cafd7eae4cd0591821acc3b5a739
2014-03-25 16:48:18 +13:00
Dan Prince
6957bd8db3 baremetal: correct the path of ifcfg-eth0
Remove the correct ifcfg-eth0 file on Fedora.

This will require an associated incubator change as well
to ensure the undercloud/overcloud now use em1.

Change-Id: I4b4bb7b6a25b058eebe889fd8ce1a403aaf193fd
2014-03-24 09:20:40 -04:00
Jan Provaznik
2f24062753 Adds libmariadb-dev package mapping
libmariadb-dev is named mariadb-devel on Fedora. mariadb-devel
package is needed for upcoming mariadb element.

Change-Id: Ib62071a964a4449c4e32ef1646ab7a0fb0f5f7b7
2014-03-24 10:39:43 +01:00
Dan Prince
8378072446 rename udev.rules to dhcp-all-interfaces-udev
Renames to udev.rules files to avoid potential
install.d collisions in DIB.

Change-Id: I2d70aa8b6e3e5fe957cb81c57f378d9623809c03
2014-03-21 15:13:35 -04:00
Dan Prince
576fa64abf serial-console: Use udev rules to startup getty
Update the systemd serial port element implementation
so that it uses udev rules. Much nicer than
hard coding ttyS0 and ttyS1.

The udev rule only adds real serial device types
(i.e. == 4). See:

  define TTY_DRIVER_TYPE_SERIAL 0x0003

from include/linux/tty_driver.h.

Change-Id: I5d7de4815fd09c01693b7da4b8949f22fee20745
2014-03-21 15:01:39 -04:00
Ben Nemec
81957625e7 Set +x on executable files
Change-Id: I6007aa3f4e6326b1c7030ad0463c7e8f49fbd507
2014-03-20 15:38:30 -05:00
Jenkins
94805549d8 Merge "Add RHEL 7 image element" 2014-03-20 11:33:24 +00:00
Jenkins
e970fc8ea0 Merge "Correct README.md markdown errors" 2014-03-20 02:46:50 +00:00
Jenkins
bc3950c7a5 Merge "Use redhat-common in fedora element" 2014-03-20 02:13:25 +00:00
Jenkins
62f8f4b120 Merge "Add redhat-common element" 2014-03-20 02:10:52 +00:00
Dan Prince
00d853e5fd DHCP: make udev rules want dhcp-interface@.service
Updates the dhcp-all-interfaces element to fix a race
with the recent udev rules implementation on Fedora.

With the new approach we make the udev rule want (require
to startup) a generic dhcp-interface@.service template which
can be started individually for each interface that is
discovered.

The dhcp-interface@.service is setup such that it:

1) It calls dhcp-all-interfaces <iface> directly with
 a pre-exec script. This creates the ifcfg file right
 before we need it but avoids the case where network.service
 might get greedy and try to start it itself.

2) Only runs if the ifcfg script doesn't already exist. This
 is important because we only need to bootstrap the DHCP configs...
 Once they exist the network.service will take care of starting them
 on reboots, upgrades, etc.

3) On initial boot ensure that the initial DHCP interfaces come
 up after network.service. Since we really only want
 dhcp-all-interfaces to help bootstrap that haven't already
 been configured this seems reasonable.

4) We also try to ensure that cloud-init
 comes up after the DHCP interfaces. Cloud init has a decently
 long timeout that this wasn't a functional problem but it keeps
 log file spew down.

Change-Id: I71b026f027182aad49c3435bb903e5e38e524685
Closes-bug: #1294803
2014-03-19 14:33:14 -04:00
Jenkins
39964b975e Merge "Update openSUSE package mapping for libffi-dev" 2014-03-19 01:18:17 +00:00
Ben Nemec
3b77e1b3c5 Add RHEL 7 image element
This is being added as a separate element because RHEL 7 is too
different from either RHEL 6 or Fedora to share all of the
configuration they do.  It uses the redhat-common element for
pieces that are duplicated.

Some current limitations of the element:
-RHEL 7 is currently still in beta, so final image locations are
 not available.  This currently defaults to the RHEL 6.5 image,
 but that will need to be overridden using the DIB_* env vars.
-There are additional tripleo-image-elements changes that will be
 needed to be able to use this for building tripleo images.
-Open vSwitch packages are not currently available in the official
 RHEL 7 repos.
-RHEL does not support qemu without kvm acceleration, so at this
 time it cannot be used as a compute node for virtualized testing.

Change-Id: I705fd475303576525a27ce6801c673b5721792c9
2014-03-17 16:14:41 -05:00
Ben Nemec
94f07c5962 Use redhat-common in fedora element
Have the fedora element use the common steps in the redhat-common
element.

Change-Id: Ic71b11c03b5c3868cc9e82e049f2700c1b67263a
2014-03-17 16:14:41 -05:00
Ben Nemec
ed15edf165 Add redhat-common element
Add an element intended for use in both Fedora and RHEL.  This
allows them to share install steps that are common to both.

Change-Id: Ie4e820a7b777b8701514351b1f802cfe57c3812e
2014-03-17 16:14:36 -05:00
Jenkins
5af4c268f9 Merge "Support adding DHCP interfaces one at a time." 2014-03-17 19:52:41 +00:00
James Slagle
b46930aeff Correct README.md markdown errors
Fixes a few syntax errors in the markdown rendering related to the new
verbage about install types. Also gives a better example of using a
DIB_INSTALLTYPE_* environment variable.

Change-Id: I4b60f458adc08b601e9d913f3e448bcf620314db
2014-03-17 11:51:37 -04:00
Jenkins
f2578f2614 Merge "Move install type enablement into base element" 2014-03-17 13:07:07 +00:00
Dan Prince
f10e614579 Support adding DHCP interfaces one at a time.
Refactors dhcp-all-interfaces.sh so that if an optional
INTERFACE argument (the first argument) is passed to the script
it only inspects that single interface. If no argument is
passed then the previous default behaviour is used which
causes all interfaces to be inspected.

To avoid a collision with the previous $1 we move to using
$FLOCKED for the exec flock command which runs on distributions
using ENI.

Also sets PATH so that the commands within the script
can all be found if it isn't set properly (/sbin/ip, /bin/cat, etc.)

This is a move towards using udev rules to add these types
of interfaces automatically.

Change-Id: I3ec8fd2cc2071bfc6943c744ca619e31b71146fc
2014-03-14 13:30:17 -04:00