Commit Graph

2069 Commits

Author SHA1 Message Date
Jenkins
d716893e30 Merge "Add dib-python element" 2015-12-15 23:24:25 +00:00
Yolanda Robla
c54b335fcc Split vm and bootloader elements
In order to add more flexibility to the vm and bootloader
elements, split the functionality in two different ones, and
make vm depend on bootloader element.
This will allow to construct more elements that depend on
bootloader, and develop both elements independently.

Change-Id: Iad2503b7b8fe53b768a3bc79e4cb839700fbd747
2015-12-15 20:49:03 +01:00
Ondřej Nový
b35f3a6c4b Deprecated tox -downloadcache option removed
Caching is enabled by default from pip version 6.0

More info:
https://testrun.org/tox/latest/config.html#confval-downloadcache=path
https://pip.pypa.io/en/stable/reference/pip_install/#caching

Change-Id: If624cea7246e0a16355923af122b2f8586718d5e
2015-12-11 22:58:50 +01:00
Jenkins
be5b669af1 Merge "Add a new element hpdsa" 2015-12-10 21:46:06 +00:00
Jenkins
3ebd0ba902 Merge "Follow up patch for 25d3ee547176528e86d42eb026c99a134dff9452" 2015-12-10 21:25:03 +00:00
Jenkins
aabf5b6ad7 Merge "Add proliant-tools element" 2015-12-10 21:24:27 +00:00
Nisha Agarwal
4509e6f826 Add a new element hpdsa
This element enables creation of Ubuntu deploy ramdisk and
user images which could be used to deploy the HP Proliant
Servers with Dynamic Smart Array Controllers. Without this driver
the disk with the Dynamic Smart Array Controller is
not visible to the ramdisk.

Closes bug: #1492803

Change-Id: Ibb3b298cd379cd7333279484df6ae30e9d7f6aaa
2015-12-10 20:12:14 +00:00
Gregory Haynes
3afbeeaf4e Add dib-python element
Creating an element which we can use in #! lines to refer to either
python2 or python3 depending on what it available.

Change-Id: Ic47e18ad21c33ab9f0d11c04260a33725aeee814
2015-12-10 19:39:38 +00:00
Robert Li
1b8a53ff3a Allow grub2 to build with opensuse
Specify grub2 in grub2 element pkg-map

Change-Id: Ia8bc1cbe11b35802f5c979ab4bb03e4e75077e83
Closes-Bug: #1519209
2015-12-10 17:29:52 +00:00
Jenkins
a95b96f57f Merge "Use ironic-agent for source-repositories" 2015-12-09 23:18:58 +00:00
Jenkins
da7b125fea Merge "Add kmod to package-installs of ironic-agent" 2015-12-09 16:22:23 +00:00
Dmitry Tantsur
901344ef7c Add kmod to package-installs of ironic-agent
The modprobe utility is required by the rtslib package (iSCSI Linux-IO).
It will also be required for inspection.

Change-Id: I6760c86160d1ceba45aedde62597a711bcb4543d
2015-12-09 14:27:21 +00:00
Jenkins
a9c8e6d55d Merge "Load the 8021q kernel module in simple-init" 2015-12-09 01:31:40 +00:00
Jenkins
93a54e32b9 Merge "Fix grub-efi-amd64-signed install failure" 2015-12-09 01:07:35 +00:00
Colleen Murphy
808e8f8bdc Load the 8021q kernel module in simple-init
Vlan support was recently added to glean. However, if the 8021q module
is not loaded, glean will fail to bring up a tagged interfaced defined
in /etc/network/interfaces.d/. Manually attempting to bring up the
interface results in an error[1]. This patch ensures that the 8021q
module is loaded so that tagged interfaces can be brought up at boot.

[1] http://paste.openstack.org/show/480027/

Change-Id: I15d805c07d4b5e1161d831f0393d027e4325137f
2015-12-08 12:59:06 -08:00
Paul Belanger
2be1bdc701 Add openssh-server package-install to local-config
Since we are modifing SSH keys, it should be safe to assume
openssh-server should be installed too.

Change-Id: I17ff05642bb2f0868d4c17819cd91b179068399a
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2015-12-07 15:23:00 -05:00
Jane.zhang
7eb70becd3 Fix grub-efi-amd64-signed install failure
When build ubuntu iso image, it will install grub-efi-amd64-signed
and grub-efi-amd64 packages. Both of the postinst script will try
to find root device and install grub which will definitely fail in
such a chroot environment.
So the workaround is to skip error and remove postinst script.
And confirm the package be installed successfully at last.

Change-Id: Ie0aecb212b22362046db55b5ad8c64c3211c28e5
Closes-Bug: #1491280
Co-Authored-By: Jane.zhang <jian.zhang8@hpe.com>
2015-12-01 23:53:35 -08:00
Ian Wienand
ce781fbbc4 Fix fedora-minimal on CentOS builds
As described in the comments, CentOS overrides the "distroverpkg"
variable in yum.conf.  This is the package that yum queries to
establish the value of the $releasever variable.  On other platforms,
this defaults to "redhat-release" (which "fedora-release" provides) so
everything works.  It is only when the base-system "distroverpkg"
refers to a package not in the chroot we hit the issue.

We can avoid this by setting the releasever variable via the
commandline.

Change-Id: I231c3277960992cd479b8aff7838f246397936f2
2015-12-02 12:16:43 +11:00
Jenkins
3a3e4e4787 Merge "Add dynamic-login element" 2015-12-01 14:27:01 +00:00
Lucas Alvares Gomes
40197fa7f1 Follow up patch for 25d3ee5471
This patch is a follow up patch fixing some nits left by the review
25d3ee5471.

It does:

* Fix the README file to say that the password *must* be encrypted and
  the option values *must* be quoted

* Adds Type=oneshot in the upstart service config file so that upstart
  will not try to restart the service over and over.

* Enable setu, sete and setpipefail in the dynamic-login script

Change-Id: Iee5d75daef24469ccf47ca12de6ead37bf9d8d6f
2015-12-01 14:11:40 +00:00
OpenStack Proposal Bot
019ed49298 Updated from global requirements
Change-Id: Id7edb1f8838d093168eff50938ed26014f1bde27
2015-12-01 05:00:47 +00:00
Jenkins
403c37ba25 Merge "Add DIB_LOCAL_CONFIG_USERNAME to local-config" 2015-11-30 05:38:36 +00:00
Jenkins
0072628c5a Merge "Move install-types doc to user guide" 2015-11-25 17:45:23 +00:00
Paul Belanger
35b363698b Add DIB_LOCAL_CONFIG_USERNAME to local-config
Allow a user to override the username on where .ssh/authorized_keys is
installed.

Change-Id: I030d5a89260aed8b23a35c4cdc2d67629934b076
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2015-11-24 10:42:14 -05:00
Dan Prince
d9dc8f1334 Use ironic-agent for source-repositories
I recently built a ramdisk for IPA and was confused by
the fact that the source-repositories name did not
match the element name. (this is a convention,
confusing when they don't match but certainly not
required).

This patch makes it so you can use DIB_REPOREF_ironic_agent to
customize the IPA ramdisk sources when building ramdisks.

For backwards compat if DIB_REPOREF_agent is set it automatically
sets the new DIB_REPOREF_ironic_agent to that value as well.

Change-Id: I082d989d0d85601f5984dc7c3767b8d66a3d5438
2015-11-22 15:03:31 -05:00
Lucas Alvares Gomes
25d3ee5471 Add dynamic-login element
Troubleshooting an image can be quite hard, specially if you can not get
a prompt you can enter commands to find out what went wrong. By default,
the images (specially ramdisks) doesn't have any SSH key or password for
any user. Of course one could use the ``devuser`` element to generate
an image with SSH keys and user/password in the image but that would be
a massive security hole and very it's discouraged to run in production
with a ramdisk like that.

This commit is adding a new element called dynamic-login, which inserts
a helper script into the image to allow operators to inject a SSH key
and/or change the root password dynamically when it boots via parameters
in the kernel command line.

Those parameters are:

sshkey = If the operator append sshkey="$PUBLIC_SSH_KEY" to the kernel
command line on boot, the helper script will append this key to the root
user authorized_keys.

rootpwd = If the operator append rootpwd="$ENCRYPTED_PASSWORD" to the
kernel command line on boot, the helper script will set the root password
to the one specified by this option. Note that this password should be
an encrypted password.

Change-Id: I6b87a1b90163d79745f30dfacd37516051fa0aea
2015-11-20 17:02:54 +00:00
Ian Wienand
1d476dd994 Remove fedora-minimal/install.d/99-ramdisk
When the kernel gets installed on Fedora, the rpm post scripts call
"/bin/kernel-install" [1] to install it.  This is a script provided by
systemd.

However, in [2], Fedora ships a patch to kernel-install that makes a
call-out to /sbin/new-kernel-pkg -- the install script provided by
grubby [3]

Without grubby installed, systemd's kernel-install script goes off and
runs dracut plugins directly [4], which eventually creates the initrd.
For reasons that are not clearly explained, the initrd will end up in
a a "machine-id" sub-directory of /boot (possibly, so you can symlink
it?).  It is also called "initrd", even though it's an initramfs, for
historical reasons in dracut I think.

It is at this point that I think 99-ramdisk has been written to move
the generated initrd file back into /boot.  Later on, when we build
the image, we run grub-install and it picks up the kernel and the
initrd and installs everything.

grubby's new-kernel-pkg [6] it's very similar -- it uses dracut to
make the initramfs ... but in this case it is put in /boot and is
actually called initramfs.

The subtle change that led me down this path is that dracut has been
modified to have a "Recommends" for grubby for >F22 [7].  After
discussing this change with the author, it turns out it was *always*
intended to use the grubby-based kernel install scripts for Fedora --
our builds have been incorrect in not including the package.  The
author got sick of people removing the package and making unbootable
systems, hence the change.

Thus this removes the workarounds in 99-ramdisk and replace it with an
install of the grubby package.  grubby's kernel install script will
put the kernel & generated initramfs in /boot, and it will be
installed correctly via the usual grub install later when we build the
disk image.

I have built F22 & F23 fedora-minimal images with this and they boot.

[1] http://pkgs.fedoraproject.org/cgit/kernel.git/tree/kernel.spec#n1832
[2] http://pkgs.fedoraproject.org/cgit/systemd.git/tree/kernel-install-grubby.patch
[3] http://linux.die.net/man/8/new-kernel-pkg
[4] https://github.com/haraldh/dracut/blob/master/50-dracut.install
[5] 81516adcb7
[6] https://github.com/rhinstaller/grubby/blob/master/new-kernel-pkg
[7] 47ff68e78b

Change-Id: I1a6e45d04755515286b3d49f8280c16b527e2f48
2015-11-19 21:03:45 +11:00
Lucas Alvares Gomes
ff988ac45c Extend root device hints for different types of WWN
This patch is extending the root device hints to also look at
ID_WWN_WITH_EXTENSION and ID_WWN_VENDOR_EXTENSION from udev.

Prior to this patch the bash ramdisk only cared about ID_WWN but in some
systems in some platforms with a RAID controller, this ID can be same
even if they are different disks (see bug 1516641).

Related-Bug: #1516641
Change-Id: I45b3910d03d164d880b32169b91e94e88812e183
2015-11-17 22:42:44 +00:00
Gregory Haynes
930b597220 Move install-types doc to user guide
Install-types are a user facing feature, not just for developers. Lets
move the docs on them in to the user guide.

Change-Id: I6ee8f657c270cf90da9c0729494740bb23aa47c5
2015-11-13 18:45:28 +00:00
Jenkins
2ff566b80a Merge "Selectively prune /root for ironic-agent ramdisk" 2015-11-13 03:28:13 +00:00
Jenkins
5e571d9f44 Merge "Fixup RPM db path when building Fedora on Ubuntu" 2015-11-10 11:03:22 +00:00
Ian Wienand
3f3850aa0f Fixup RPM db path when building Fedora on Ubuntu
On Debian/Ubuntu installs of RPM, /usr/lib/rpm/macros sets

  %_dbpath  %(echo $HOME/.rpmdb)

which makes quite a bit of sense, because RPM is not the system
packager and thus RPM is setup to install things into a hierarchy in
the users homedir.

However, this messes things up when building a Fedora chroot on an
Ubuntu platform.

We use RPM & yum from the base-system to bootstrap the Fedora chroot.
While both obey --root flags, they still pick up the %_dbpath macro
and so end up creating the RPM database in <chroot>/home/user/.rpmdb

After we have bootstrapped yum/dnf, we execute further installation
commands from inside the chroot -- where we now have the Fedora
version of /usr/lib/rpm/macros and hence have _dbpath set to
/var/lib/rpm -- except there is no rpm database there.

Should anyone be finding this in the future, the actual issue that
appears is

  $ sudo chroot /opt/dib_tmp/image.b6B5S3f6/mnt dnf makecache
   Error: Failed to synchronize cache for repo 'fedora' from \
    'https://mirrors.fedoraproject.org/metalink?repo=fedora-$releasever&arch=x86_64': \
    Cannot prepare internal mirrorlist: file "repomd.xml" was not found in metalink

Note the issue there is that $releasever is not expanded, because the
rpmdb where this info is kept is not populated.

The trick is to make sure we override this value when using the host
rpm/yum to setup the chroot.  The bare rpm calls, which we use to
install the repos, have a --dbpath argument where we can override
this.  yum does not however, so we override this in the global
~/.rpmmacros while we are installing the packaging tools and
dependencies into the chroot.

Copious comments are included, because this is super-confusing.

Change-Id: I20801150ea02d1c64f118eb969fb2aec473476f7
2015-11-10 08:54:44 +00:00
Ian Wienand
8ee21cb9fd Remove unused RELEASE_RPMS variable
It was noticed during reviews of
Ic7aa8cbe13e4347b447e84bb9c12483a4e125228 these are unused

Change-Id: I9e0fa9d3e4864e54c6fe23f8b6e781e8d5d24bda
2015-11-10 07:17:52 +00:00
Jenkins
e90be5a595 Merge "Fix fedora-minimal kernel-install on older platforms" 2015-11-10 05:14:28 +00:00
Jenkins
6f21f47850 Merge "Clarify what fedora-minimal/install.d/99-ramdisk is doing" 2015-11-10 05:14:25 +00:00
Jenkins
3412ced2f0 Merge "dib-lint: ignore blank lines in element ordering" 2015-11-10 00:32:04 +00:00
Jenkins
484341c5ab Merge "Fix diskimage-builder image size" 2015-11-09 19:41:54 +00:00
Michael Johnson
590dce8dcc Fix diskimage-builder image size
This patch fixes the calculation of the resultant image size
when building an image with diskimage-builder on ext4 a
filesystem.

Prior to this, using the '--image-size 2' (2GB) setting would
generate an image that would not boot under a 2GB nova flavor.

Change-Id: I7a753bdef84c6300ccea73ae4a92bf330dcd77cb
Closes-Bug: #1513622
2015-11-09 16:20:18 +00:00
Ramakrishnan G
2ebfb12b84 Add proliant-tools element
This new element installs hpssacli utility (for configuring
RAID) and installs proliantutils python module (which has
ironic-python-agent hardware manager for HP ProLiant hardware).

This module also exposes a new environment variable DIB_HPSSACLI_URL
which allows operator to pass a custom HTTP(S) URL for RPM of hpssacli
utility.

NOTE: This module currently supports only installing from source.

Change-Id: I0494e3db623fdd7ea9182ffba21c0652aaad113c
2015-11-09 07:58:00 +00:00
Jenkins
dfb40eb5aa Merge "Add --force to grub-install" 2015-11-09 07:14:13 +00:00
Jenkins
6f3185f251 Merge "Add missing six requirement for svc-map element" 2015-11-05 18:18:09 +00:00
Jenkins
8c42e60df5 Merge "Fix uniqueness check of initrd in fedora-minimal" 2015-11-05 03:36:13 +00:00
Martin André
8309df5f25 Add missing six requirement for svc-map element
Add constraint for versions >= 1.9.0 to be in line with global
requirements.

Change-Id: I91bcbfa00b1b03774d507dd5c9c5fcd1c4d809b4
2015-11-05 01:58:00 +00:00
Jenkins
63610993ed Merge "debian: cloud hostname ignored by Jessie" 2015-11-04 21:46:54 +00:00
Jenkins
29f42e2f15 Merge "Add support for Xen PV disks" 2015-11-04 21:07:25 +00:00
Jenkins
45928ab5d1 Merge "Add a tox target to run functional tests locally" 2015-11-03 20:56:02 +00:00
Jenkins
654d80a40f Merge "Define a default for $YUM" 2015-11-03 20:33:38 +00:00
Ian Wienand
f307bb4d8b Fix fedora-minimal kernel-install on older platforms
fedora-minimal fails to build on Ubuntu Trusty due do being unable to
find the initrd (see Id4c04d7ae20068643df34d2fa31068e8a917a52d).

This is a rather obscure problem that comes from the intersection of
several things.

The first thing to note is that the post-install scripts of the
kernel-core package use kernel-install [1].  For whatever reason, this
installs the kernel to /boot/MACHINE-ID/KERNEL-VERSION

MACHINE-ID comes from /etc/machine-id; a UUID that should have been
created by the systemd post-inst scripts with systemd-machine-id-setup
[2].

The chroot environment provided for root.d elements has no kernel
file-systems like /proc or /dev mounted.  This is where differences in
the base-system come into play -- on more recent systems that
implement getrandom() systemd does not need /dev/urandom to generate
the machine-id [3]; we get a value and /etc/machine-id is populated.

On older platforms (Trusty), systemd-machine-id-setup fails (unable to
access /dev/urandom) and we end up with a blank /etc/machine-id.  This
ends up making kernel-install (the script) fail during yum's
installation of kernel-core, which means the initrd is not installed
correctly.

We end up bailing out in fedora-minimal/install.d/99-ramdisk, where we
try to put the installed ramdisk in /boot for the later grub install
scripts to find.

The solution here is to mount the standard kernel file-systems within
the chroot before we try installing.

[1] http://www.freedesktop.org/software/systemd/man/kernel-install.html
[2] http://www.freedesktop.org/software/systemd/man/systemd-machine-id-setup.html
[3] https://github.com/systemd/systemd/blob/master/src/basic/random-util.c

Change-Id: Ibcce35da928f64e6a719b070bcc833346ee7ee92
2015-11-04 06:23:17 +11:00
Ian Wienand
a67ee17d62 Clarify what fedora-minimal/install.d/99-ramdisk is doing
Clarify what this script is doing.  It currently fails on some
platforms due to earlier errors, see
Ibcce35da928f64e6a719b070bcc833346ee7ee92

Change-Id: Id4c04d7ae20068643df34d2fa31068e8a917a52d
2015-11-04 06:23:17 +11:00
Martin André
d8a0e13261 Fix uniqueness check of initrd in fedora-minimal
The check suffered from various flaws.

First, due to missing quotes around $initrd, 'wc -l' would always see
1 line no matter how many results the find returned.

Second, echo adds a line break making 'wc -l' count 1 even for empty
string. We need to add a check for empty string.

Change-Id: Ib2c67960f566dbdc471d9585a4cef1beb1cc38ab
Closes-Bug: #1506692
2015-11-04 06:23:17 +11:00