Commit Graph

2753 Commits

Author SHA1 Message Date
Gregory Haynes
6a40b60717 Create ubuntu/fedora test for pip-and-virtualenv
This element does some funky stuff WRT python2 vs python3 so lets get
some multi-distro testing in place.

Change-Id: I1e3c3bfa0a109419d4cbee7fa32a18392b7e1a93
2016-12-15 18:00:41 +00:00
Ian Wienand
a72645f431 Recreate initramfs within loopback image
dracut has a loop [1] where it probes top-level directories, tries to
find what block device they are on, then determines the file-system of
that block device.  It then puts those file-system modules into the
initramfs for boot.

Since we install the kernel package during the chroot phase, / there
is not a block device and thus this loop matches nothing and we end up
with no file-system modules in the initramfs.  This results in a very
annoying silent boot hang.

By moving re-generation of dracut into finalise.d phase, we run inside
the final image where / is the loop-device; the root file-system gets
detected correctly and the ext4 module is included correctly.

[1] http://git.kernel.org/cgit/boot/dracut/dracut.git/tree/dracut.sh?h=RHEL-7#n1041

Change-Id: Iaf2a1e8470f642bfaaaad3f9b7f26cfc8cc445c9
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2016-12-15 12:37:00 -05:00
Peter Stachowski
280896759a Pip install as 10- incompatible with 05-heat-cfntools
Tripleo-image-elements have an install.d file '05-heat-cfntools' that runs
the following command:

virtualenv --setuptools $VENV

With the recent change to diskimage-builder (moving the install of pip
and virtualenv to the 10- range) virtualenv is no longer available for
this elementr; as a side-effect, the trove kick-start command is now
broken and gate jobs are failing.

The solutions is to move the (now) 10-install-pip to 04-install-pip.
This should still alleviate the race condition that
https://review.openstack.org/#/c/408277/ attempted to fix, as all
*-package-installs files are 00-, 01- or 02-.

Change-Id: Ia4e01f00c4c5e9a2087df1e2a91d9154480a0422
Closes-Bug: #1650008
2016-12-14 20:50:00 +00:00
Saverio Proto
3417bd6298 Support sysv init system used by Debian Wheezy
Change-Id: Ia6ca11ab78f16a51aba7b627c72c615c184d338d
2016-12-14 15:53:04 +01:00
Markos Chandras
339ecee2b2 elements: dib-python: Add python2 as the default version for openSUSE
Commit 6278371eaa13("Make dib-python use the default python for distro")
added default python version for various distros but it missed openSUSE
which leads to build failures since the openSUSE elements are pulling
python2 packages. Add openSUSE to the list of python2 distributions
until python3 support for the openSUSE elements is in place.

Change-Id: I95f1fa849a22607c430387a2a915f9d19c9c209f
2016-12-14 09:38:45 +00:00
Jenkins
5a64c9e9cf Merge "Fix pip-and-virtualenv to work with python3" 2016-12-14 07:18:32 +00:00
Jenkins
d1ca1b1957 Merge "Catch errors in DIB_INIT_SYSTEM export" 2016-12-14 07:14:10 +00:00
Gregory Haynes
3e777cd8f4 Fix pip-and-virtualenv to work with python3
We are explicitly calling python in this element which does not work on
systems which only have python3.

Change-Id: Ia730850a48e2478fd5461710a9d2619408725cd8
2016-12-14 17:14:02 +11:00
Jenkins
8565867734 Merge "Allow package-installs to parse DIB_PYTHON_VERSION" 2016-12-14 02:23:47 +00:00
Jenkins
1c0ad1d4b0 Merge "Disable centos6 testing" 2016-12-14 01:14:03 +00:00
Gregory Haynes
ecae8dcbd5 Allow package-installs to parse DIB_PYTHON_VERSION
Now that we are explicit about what python version we intend to use
for dib we can have package installs optionally install packages
depending on this.  Add a new dib_python_version that matches on the
DIB_PYTHON_VERSION string set by dib-python.

Co-Authored-By: Adam Harwell <flux.adam@gmail.com>
Change-Id: I70659aab7d12924bdb9bc0489a7f02d5fd0dbb39
2016-12-14 12:13:40 +11:00
Jenkins
d981d6b0cc Merge "Add install-types as pip-and-virtualenv dep" 2016-12-14 00:57:45 +00:00
Jenkins
95b6c1b00b Merge "Move pip-and-virtualenv source install to 10-" 2016-12-14 00:36:19 +00:00
Gregory Haynes
a1dfe505ea Add install-types as pip-and-virtualenv dep
This element supports install-types so we need to depend on it.

Change-Id: Ib1193673ca1c1a1cafe0006eabef981a01c87781
2016-12-14 11:21:11 +11:00
Gregory Haynes
c3293b3f7e Disable centos6 testing
Centos6 is no longer reasonable to expect to function with openstack,
and infra does not host a mirror for it.

Change-Id: I95ccf5840807fee73d6e78d596c82709e476bb3a
2016-12-14 11:10:54 +11:00
Gregory Haynes
6ddbb457d6 Move pip-and-virtualenv source install to 10-
We currently have this as a 01- script which causes it to race with
package-installs (the deps are installed after the script runs).

Change-Id: I7b04b4c186eaae783b8e2bda1aa724c0d7823eab
2016-12-14 11:07:12 +11:00
Jenkins
b34c5db441 Merge "Update sysctl-write-value to do conflict checking" 2016-12-14 00:04:55 +00:00
Jenkins
a52a82036e Merge "DIB element to support cinder local attach/detach functionality" 2016-12-13 23:58:36 +00:00
Jenkins
5db62528f5 Merge "Put MKFS_OPTS after filesystem type" 2016-12-13 23:58:30 +00:00
Jenkins
8ddbc6425d Merge "Make dib-python use the default python for distro" 2016-12-13 23:28:50 +00:00
Ian Wienand
8e3e66dfbd Speed up chroot checking loop
It was noticed on a very busy system this can take about 1s per loop.
This starts to add up on thousands of processes.

Firstly, prune out all the kernel threads.  Then introduce a very
small inline python script to find any pids that seem to be in the
chroot without forking to examine each one.  After that the existing
loop just kills anything as before.

Change-Id: Icc7bc7eda80ffcd636f97e6542d70c220e9c225e
2016-12-13 12:47:46 +11:00
Ben Nemec
bf5af6155c Don't set the executable bit on dhcp-interface@.service
systemd doesn't like it when service files have the executable bit
so this causes it to spam the journal with messages like:

Configuration file /usr/lib/systemd/system/dhcp-interface@.service is
marked executable. Please remove executable permission bits.
Proceeding anyway.

Removing the executable bit from the install permissions should
eliminate those messages.

Change-Id: Ie1bc39465b3fcb55dcda5cee9e46a128a6ccffcb
2016-12-12 10:55:03 -06:00
Gregory Haynes
6278371eaa Make dib-python use the default python for distro
Right now dib-python works by trying to find any python on a system in
an order of precedence. A much better way is if we are explicit about
the python we intend to be there which will allow us to make better
decisions in other elements (such as allowing for package-installs to
take into account DIB_PYTHON_VERSION) as well as allow for users to
specify a preferred python version.

Co-Authored-By: Adam Harwell <flux.adam@gmail.com>
Change-Id: Ie609de51cc5fcde701296c9474e315981d9778a2
2016-12-09 09:25:37 -08:00
Gregory Haynes
cd1965f7b7 Allow disto-specific mirror settings
Most of our distros support DIB_DISTRIBUTION_MIRROR for setting the
mirror to be used when building. This has a major downside where if
someone would like to create a static configuration for building various
distro images it cannot set DIB_DISTRIBUTION_MIRROR (think of the case
where we want to run our tests).

By adding support for DIB_$DISTRO_DISTRIBUTION_MIRROR this problem is
resolved.

Change-Id: I92964b17ec3e47cf97e3a3091f054b2a205ac768
2016-12-08 16:02:15 -08:00
Ian Wienand
5e2330d89c Put MKFS_OPTS after filesystem type
mkfs's arguments are

 mkfs [options] [-t type] [fs-options] device [size]

So it seems our MKFS_OPTS are really supposed to be fs-options, rather
than options to mkfs itself.

Why didn't we notice?  It's quite a trap -- mkfs.ext2 has a "-t"
option, so when we're calling

 $ mkfs -i 4096 ... -t ext4 ...

We actually just fall-back to the default from the mkfs wrapper which
is mkfs.ext2 which works!  But when you make that, say, xfs, we're not
calling the right wrapper at all.

Also update documentation

Closes-Bug: #1648287
Change-Id: I3ea5807088ab361bd9c235c07fb1553fbaf9178b
2016-12-09 07:22:38 +11:00
Aparna
cb3e1076e3 Update hpssacli to ssacli in proliant-tools element
This commit updates the CLI utility name hpssacli to
ssacli  as part of company name change from HP to HPE.
ssacli will work seamlessly for all the cases where
hpssacli is used.

It also updates the custom URL location for ssacli
utility with new location.

Change-Id: I2de4d3430e7308d25e36dc6960783699aa17dfae
2016-12-07 15:56:41 +00:00
Michael Johnson
2e82d7f214 Update sysctl-write-value to do conflict checking
Adds conflict checking to the sysctl-write-value script
to detect settings from multiple elements conflicting.

Change-Id: If312d199388036d6f4103e94dca99249cb3bcbaf
2016-12-06 22:58:20 +00:00
Jenkins
8c74c8e409 Merge "Delete deprecated Hacking in tox.ini" 2016-12-06 03:18:58 +00:00
Jenkins
b25044fe11 Merge "elements: Drop executable bits from environment files" 2016-12-06 03:18:33 +00:00
Jenkins
e19a3d823a Merge "add option to configure cloud-init to allow password authentication" 2016-12-05 06:54:50 +00:00
gengchc2
b2c3a3afdd Delete deprecated Hacking in tox.ini
Some hacking have been removed,so we can delete them.
More details:
    https://github.com/openstack-dev/hacking/blob/master/setup.cfg

Change-Id: I2ba6b00aa2a968ffb53d1061f895a593ea57a063
2016-12-04 13:04:50 +08:00
Jenkins
fc7b7cf46d Merge "Changed author and author-email" 2016-12-02 17:02:58 +00:00
Jenkins
753ac91ec8 Merge "Replace six.iteritems() with .items()" 2016-12-02 16:38:22 +00:00
Markos Chandras
e22faa0f77 elements: Drop executable bits from environment files
Files in $element/environment.d are meant to be sourced, so drop
the executable bit. Moreover, drop the executable bit from a couple
of other scripts that are either meant to be sourced or simply because
they are configuration files.

Change-Id: I7f724dd9d409f4a835a136f12f48a84aa9acc41e
2016-12-01 23:06:56 +00:00
Jenkins
ef1effd202 Merge "elements: Drop unneeded DIB_INIT_SYSTEM usage" 2016-12-01 19:44:47 +00:00
Jenkins
f7e095e880 Merge "elements: pip-and-virtualenv: Add python-xml dependency" 2016-12-01 07:34:55 +00:00
Jenkins
ff688c67c5 Merge "elements: zypper-minimal: Add ca-certificates-mozilla package" 2016-12-01 05:28:32 +00:00
Ian Wienand
5adfc9365a Fix --version display
We do not have the concept of "not installed" in v2, so remove the
obsolete code looking for a now non-existent variable.

Also, log the version at startup.  This can help when
debugging from logs

Change-Id: I964c4cf207c10666afc5bc7ab9f2bfb9b1897c1e
2016-12-01 04:37:04 +00:00
Jenkins
c867c8cdb5 Merge "Trace package install in package-installs-v2" 2016-12-01 01:52:03 +00:00
Anshul Jain
34bdc7df90 DIB element to support cinder local attach/detach functionality
This element adds python-brick-cinderclient-ext to the make customized image
to support cinder local attach/detach functionality. Currently it has the
dependency on known bug<https://launchpad.net/bugs/1623549>, which would be
resolved with next release of python-brick-cinderclient-ext.

Change-Id: Idfe83bafa2843c781c18b83f1a3aece3ae852f78
2016-11-30 08:46:13 +00:00
Jenkins
fffe15e763 Merge "Perform package install outside of debootstrap" 2016-11-30 05:32:09 +00:00
Jenkins
389b372976 Merge "Improve checksum performance for images" 2016-11-30 05:07:21 +00:00
Jenkins
3e68c5aaa5 Merge "Fix a typo" 2016-11-30 04:20:56 +00:00
Gregory Haynes
45df304d48 Perform package install outside of debootstrap
Debootstrap only supports one apt repository to install packages from.
As a result, we do not consider the updates repo during debootstrap
causing us install a second kernel when we do an apt-get dist-upgrade
during build.

Lets use debootstrap to get us a minimal chroot, then add our repos and
install the correct packages from the start.

We also have to reorder the dpkg root.d scripts which configure apt so
they run before we perform our package installs.

Change-Id: I6a592db6f0a01d3b19d8e0786e63f1315a1ef647
Closes-Bug: #1637516
2016-11-30 15:16:46 +11:00
Paul Belanger
22952b7ea0 Improve checksum performance for images
Do md5 and sha256 in parallel to speed things up for larger images.

Change-Id: Ib782fe54e4286ba2749a7ab7247f5d41a887a370
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2016-11-30 03:42:24 +00:00
Ian Wienand
04208e7c79 Activate virtualenv in disk-image-create
Because we're still fundamentally a python program calling a
shell-script, there's some oddities like not having the virtualenv
bin/ in the $PATH if we call disk-image-create directly.

We can detect this, however, and activate the virtualenv before we
fork the disk-image-create shell script so everything "just works".

See also nodepool change I0537cbf167bb18edf26f84ac269cbd9c8a1ea6a2

Change-Id: Ibfea6cf6a6fd0c7f1e468d501c61ae0b58992042
2016-11-30 06:11:09 +11:00
Markos Chandras
35e878b6d9 elements: zypper-minimal: Add ca-certificates-mozilla package
It's important to have the CA certificates on the target for ssl
crypto apps to work. Plus it's also important during bootstrapping
with diskimage-builder as tools like 'pip' etc need the certificates
in place in order to work properly. This fixes opensuse-minimal
image generation with the 'simple-init' element which was causing the
following error:

Download error on https://pypi.python.org/simple/: [SSL:
CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)

Change-Id: Ie94cd3556f8ae523f60ce0155ba18ed752e6fbb6
2016-11-29 16:14:53 +00:00
Cao Xuan Hoang
cdfd757ae9 Changed author and author-email
Changed author to OpenStack and author_email to
author-email

Change-Id: Id1a398bfd644e8db41a8ab816c8c6ff3c322f7b4
2016-11-29 11:04:52 +07:00
Ian Wienand
448a2602fe Merge remote-tracking branch 'origin/master' into merge-branch
Change-Id: I05cc51c699008018d557ed3874d71af26fd60240
2016-11-29 07:43:46 +11:00
Jenkins
7644681815 Merge "elements: Add new openssh-server element" 2016-11-28 17:09:15 +00:00