Commit Graph

2206 Commits

Author SHA1 Message Date
Colleen Murphy
808e8f8bdc Load the 8021q kernel module in simple-init
Vlan support was recently added to glean. However, if the 8021q module
is not loaded, glean will fail to bring up a tagged interfaced defined
in /etc/network/interfaces.d/. Manually attempting to bring up the
interface results in an error[1]. This patch ensures that the 8021q
module is loaded so that tagged interfaces can be brought up at boot.

[1] http://paste.openstack.org/show/480027/

Change-Id: I15d805c07d4b5e1161d831f0393d027e4325137f
2015-12-08 12:59:06 -08:00
Paul Belanger
2be1bdc701 Add openssh-server package-install to local-config
Since we are modifing SSH keys, it should be safe to assume
openssh-server should be installed too.

Change-Id: I17ff05642bb2f0868d4c17819cd91b179068399a
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2015-12-07 15:23:00 -05:00
Jane.zhang
7eb70becd3 Fix grub-efi-amd64-signed install failure
When build ubuntu iso image, it will install grub-efi-amd64-signed
and grub-efi-amd64 packages. Both of the postinst script will try
to find root device and install grub which will definitely fail in
such a chroot environment.
So the workaround is to skip error and remove postinst script.
And confirm the package be installed successfully at last.

Change-Id: Ie0aecb212b22362046db55b5ad8c64c3211c28e5
Closes-Bug: #1491280
Co-Authored-By: Jane.zhang <jian.zhang8@hpe.com>
2015-12-01 23:53:35 -08:00
Ian Wienand
ce781fbbc4 Fix fedora-minimal on CentOS builds
As described in the comments, CentOS overrides the "distroverpkg"
variable in yum.conf.  This is the package that yum queries to
establish the value of the $releasever variable.  On other platforms,
this defaults to "redhat-release" (which "fedora-release" provides) so
everything works.  It is only when the base-system "distroverpkg"
refers to a package not in the chroot we hit the issue.

We can avoid this by setting the releasever variable via the
commandline.

Change-Id: I231c3277960992cd479b8aff7838f246397936f2
2015-12-02 12:16:43 +11:00
Jenkins
3a3e4e4787 Merge "Add dynamic-login element" 2015-12-01 14:27:01 +00:00
Lucas Alvares Gomes
40197fa7f1 Follow up patch for 25d3ee5471
This patch is a follow up patch fixing some nits left by the review
25d3ee5471.

It does:

* Fix the README file to say that the password *must* be encrypted and
  the option values *must* be quoted

* Adds Type=oneshot in the upstart service config file so that upstart
  will not try to restart the service over and over.

* Enable setu, sete and setpipefail in the dynamic-login script

Change-Id: Iee5d75daef24469ccf47ca12de6ead37bf9d8d6f
2015-12-01 14:11:40 +00:00
OpenStack Proposal Bot
019ed49298 Updated from global requirements
Change-Id: Id7edb1f8838d093168eff50938ed26014f1bde27
2015-12-01 05:00:47 +00:00
Jenkins
403c37ba25 Merge "Add DIB_LOCAL_CONFIG_USERNAME to local-config" 2015-11-30 05:38:36 +00:00
Jenkins
0072628c5a Merge "Move install-types doc to user guide" 2015-11-25 17:45:23 +00:00
Paul Belanger
35b363698b Add DIB_LOCAL_CONFIG_USERNAME to local-config
Allow a user to override the username on where .ssh/authorized_keys is
installed.

Change-Id: I030d5a89260aed8b23a35c4cdc2d67629934b076
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2015-11-24 10:42:14 -05:00
Dan Prince
d9dc8f1334 Use ironic-agent for source-repositories
I recently built a ramdisk for IPA and was confused by
the fact that the source-repositories name did not
match the element name. (this is a convention,
confusing when they don't match but certainly not
required).

This patch makes it so you can use DIB_REPOREF_ironic_agent to
customize the IPA ramdisk sources when building ramdisks.

For backwards compat if DIB_REPOREF_agent is set it automatically
sets the new DIB_REPOREF_ironic_agent to that value as well.

Change-Id: I082d989d0d85601f5984dc7c3767b8d66a3d5438
2015-11-22 15:03:31 -05:00
Lucas Alvares Gomes
25d3ee5471 Add dynamic-login element
Troubleshooting an image can be quite hard, specially if you can not get
a prompt you can enter commands to find out what went wrong. By default,
the images (specially ramdisks) doesn't have any SSH key or password for
any user. Of course one could use the ``devuser`` element to generate
an image with SSH keys and user/password in the image but that would be
a massive security hole and very it's discouraged to run in production
with a ramdisk like that.

This commit is adding a new element called dynamic-login, which inserts
a helper script into the image to allow operators to inject a SSH key
and/or change the root password dynamically when it boots via parameters
in the kernel command line.

Those parameters are:

sshkey = If the operator append sshkey="$PUBLIC_SSH_KEY" to the kernel
command line on boot, the helper script will append this key to the root
user authorized_keys.

rootpwd = If the operator append rootpwd="$ENCRYPTED_PASSWORD" to the
kernel command line on boot, the helper script will set the root password
to the one specified by this option. Note that this password should be
an encrypted password.

Change-Id: I6b87a1b90163d79745f30dfacd37516051fa0aea
2015-11-20 17:02:54 +00:00
Ian Wienand
1d476dd994 Remove fedora-minimal/install.d/99-ramdisk
When the kernel gets installed on Fedora, the rpm post scripts call
"/bin/kernel-install" [1] to install it.  This is a script provided by
systemd.

However, in [2], Fedora ships a patch to kernel-install that makes a
call-out to /sbin/new-kernel-pkg -- the install script provided by
grubby [3]

Without grubby installed, systemd's kernel-install script goes off and
runs dracut plugins directly [4], which eventually creates the initrd.
For reasons that are not clearly explained, the initrd will end up in
a a "machine-id" sub-directory of /boot (possibly, so you can symlink
it?).  It is also called "initrd", even though it's an initramfs, for
historical reasons in dracut I think.

It is at this point that I think 99-ramdisk has been written to move
the generated initrd file back into /boot.  Later on, when we build
the image, we run grub-install and it picks up the kernel and the
initrd and installs everything.

grubby's new-kernel-pkg [6] it's very similar -- it uses dracut to
make the initramfs ... but in this case it is put in /boot and is
actually called initramfs.

The subtle change that led me down this path is that dracut has been
modified to have a "Recommends" for grubby for >F22 [7].  After
discussing this change with the author, it turns out it was *always*
intended to use the grubby-based kernel install scripts for Fedora --
our builds have been incorrect in not including the package.  The
author got sick of people removing the package and making unbootable
systems, hence the change.

Thus this removes the workarounds in 99-ramdisk and replace it with an
install of the grubby package.  grubby's kernel install script will
put the kernel & generated initramfs in /boot, and it will be
installed correctly via the usual grub install later when we build the
disk image.

I have built F22 & F23 fedora-minimal images with this and they boot.

[1] http://pkgs.fedoraproject.org/cgit/kernel.git/tree/kernel.spec#n1832
[2] http://pkgs.fedoraproject.org/cgit/systemd.git/tree/kernel-install-grubby.patch
[3] http://linux.die.net/man/8/new-kernel-pkg
[4] https://github.com/haraldh/dracut/blob/master/50-dracut.install
[5] 81516adcb7
[6] https://github.com/rhinstaller/grubby/blob/master/new-kernel-pkg
[7] 47ff68e78b

Change-Id: I1a6e45d04755515286b3d49f8280c16b527e2f48
2015-11-19 21:03:45 +11:00
Lucas Alvares Gomes
ff988ac45c Extend root device hints for different types of WWN
This patch is extending the root device hints to also look at
ID_WWN_WITH_EXTENSION and ID_WWN_VENDOR_EXTENSION from udev.

Prior to this patch the bash ramdisk only cared about ID_WWN but in some
systems in some platforms with a RAID controller, this ID can be same
even if they are different disks (see bug 1516641).

Related-Bug: #1516641
Change-Id: I45b3910d03d164d880b32169b91e94e88812e183
2015-11-17 22:42:44 +00:00
Gregory Haynes
930b597220 Move install-types doc to user guide
Install-types are a user facing feature, not just for developers. Lets
move the docs on them in to the user guide.

Change-Id: I6ee8f657c270cf90da9c0729494740bb23aa47c5
2015-11-13 18:45:28 +00:00
Jenkins
2ff566b80a Merge "Selectively prune /root for ironic-agent ramdisk" 2015-11-13 03:28:13 +00:00
Jenkins
5e571d9f44 Merge "Fixup RPM db path when building Fedora on Ubuntu" 2015-11-10 11:03:22 +00:00
Ian Wienand
3f3850aa0f Fixup RPM db path when building Fedora on Ubuntu
On Debian/Ubuntu installs of RPM, /usr/lib/rpm/macros sets

  %_dbpath  %(echo $HOME/.rpmdb)

which makes quite a bit of sense, because RPM is not the system
packager and thus RPM is setup to install things into a hierarchy in
the users homedir.

However, this messes things up when building a Fedora chroot on an
Ubuntu platform.

We use RPM & yum from the base-system to bootstrap the Fedora chroot.
While both obey --root flags, they still pick up the %_dbpath macro
and so end up creating the RPM database in <chroot>/home/user/.rpmdb

After we have bootstrapped yum/dnf, we execute further installation
commands from inside the chroot -- where we now have the Fedora
version of /usr/lib/rpm/macros and hence have _dbpath set to
/var/lib/rpm -- except there is no rpm database there.

Should anyone be finding this in the future, the actual issue that
appears is

  $ sudo chroot /opt/dib_tmp/image.b6B5S3f6/mnt dnf makecache
   Error: Failed to synchronize cache for repo 'fedora' from \
    'https://mirrors.fedoraproject.org/metalink?repo=fedora-$releasever&arch=x86_64': \
    Cannot prepare internal mirrorlist: file "repomd.xml" was not found in metalink

Note the issue there is that $releasever is not expanded, because the
rpmdb where this info is kept is not populated.

The trick is to make sure we override this value when using the host
rpm/yum to setup the chroot.  The bare rpm calls, which we use to
install the repos, have a --dbpath argument where we can override
this.  yum does not however, so we override this in the global
~/.rpmmacros while we are installing the packaging tools and
dependencies into the chroot.

Copious comments are included, because this is super-confusing.

Change-Id: I20801150ea02d1c64f118eb969fb2aec473476f7
2015-11-10 08:54:44 +00:00
Ian Wienand
8ee21cb9fd Remove unused RELEASE_RPMS variable
It was noticed during reviews of
Ic7aa8cbe13e4347b447e84bb9c12483a4e125228 these are unused

Change-Id: I9e0fa9d3e4864e54c6fe23f8b6e781e8d5d24bda
2015-11-10 07:17:52 +00:00
Jenkins
e90be5a595 Merge "Fix fedora-minimal kernel-install on older platforms" 2015-11-10 05:14:28 +00:00
Jenkins
6f21f47850 Merge "Clarify what fedora-minimal/install.d/99-ramdisk is doing" 2015-11-10 05:14:25 +00:00
Jenkins
3412ced2f0 Merge "dib-lint: ignore blank lines in element ordering" 2015-11-10 00:32:04 +00:00
Jenkins
484341c5ab Merge "Fix diskimage-builder image size" 2015-11-09 19:41:54 +00:00
Michael Johnson
590dce8dcc Fix diskimage-builder image size
This patch fixes the calculation of the resultant image size
when building an image with diskimage-builder on ext4 a
filesystem.

Prior to this, using the '--image-size 2' (2GB) setting would
generate an image that would not boot under a 2GB nova flavor.

Change-Id: I7a753bdef84c6300ccea73ae4a92bf330dcd77cb
Closes-Bug: #1513622
2015-11-09 16:20:18 +00:00
Ramakrishnan G
2ebfb12b84 Add proliant-tools element
This new element installs hpssacli utility (for configuring
RAID) and installs proliantutils python module (which has
ironic-python-agent hardware manager for HP ProLiant hardware).

This module also exposes a new environment variable DIB_HPSSACLI_URL
which allows operator to pass a custom HTTP(S) URL for RPM of hpssacli
utility.

NOTE: This module currently supports only installing from source.

Change-Id: I0494e3db623fdd7ea9182ffba21c0652aaad113c
2015-11-09 07:58:00 +00:00
Jenkins
dfb40eb5aa Merge "Add --force to grub-install" 2015-11-09 07:14:13 +00:00
Jenkins
6f3185f251 Merge "Add missing six requirement for svc-map element" 2015-11-05 18:18:09 +00:00
Jenkins
8c42e60df5 Merge "Fix uniqueness check of initrd in fedora-minimal" 2015-11-05 03:36:13 +00:00
Martin André
8309df5f25 Add missing six requirement for svc-map element
Add constraint for versions >= 1.9.0 to be in line with global
requirements.

Change-Id: I91bcbfa00b1b03774d507dd5c9c5fcd1c4d809b4
2015-11-05 01:58:00 +00:00
Jenkins
63610993ed Merge "debian: cloud hostname ignored by Jessie" 2015-11-04 21:46:54 +00:00
Jenkins
29f42e2f15 Merge "Add support for Xen PV disks" 2015-11-04 21:07:25 +00:00
Jenkins
45928ab5d1 Merge "Add a tox target to run functional tests locally" 2015-11-03 20:56:02 +00:00
Jenkins
654d80a40f Merge "Define a default for $YUM" 2015-11-03 20:33:38 +00:00
Ian Wienand
f307bb4d8b Fix fedora-minimal kernel-install on older platforms
fedora-minimal fails to build on Ubuntu Trusty due do being unable to
find the initrd (see Id4c04d7ae20068643df34d2fa31068e8a917a52d).

This is a rather obscure problem that comes from the intersection of
several things.

The first thing to note is that the post-install scripts of the
kernel-core package use kernel-install [1].  For whatever reason, this
installs the kernel to /boot/MACHINE-ID/KERNEL-VERSION

MACHINE-ID comes from /etc/machine-id; a UUID that should have been
created by the systemd post-inst scripts with systemd-machine-id-setup
[2].

The chroot environment provided for root.d elements has no kernel
file-systems like /proc or /dev mounted.  This is where differences in
the base-system come into play -- on more recent systems that
implement getrandom() systemd does not need /dev/urandom to generate
the machine-id [3]; we get a value and /etc/machine-id is populated.

On older platforms (Trusty), systemd-machine-id-setup fails (unable to
access /dev/urandom) and we end up with a blank /etc/machine-id.  This
ends up making kernel-install (the script) fail during yum's
installation of kernel-core, which means the initrd is not installed
correctly.

We end up bailing out in fedora-minimal/install.d/99-ramdisk, where we
try to put the installed ramdisk in /boot for the later grub install
scripts to find.

The solution here is to mount the standard kernel file-systems within
the chroot before we try installing.

[1] http://www.freedesktop.org/software/systemd/man/kernel-install.html
[2] http://www.freedesktop.org/software/systemd/man/systemd-machine-id-setup.html
[3] https://github.com/systemd/systemd/blob/master/src/basic/random-util.c

Change-Id: Ibcce35da928f64e6a719b070bcc833346ee7ee92
2015-11-04 06:23:17 +11:00
Ian Wienand
a67ee17d62 Clarify what fedora-minimal/install.d/99-ramdisk is doing
Clarify what this script is doing.  It currently fails on some
platforms due to earlier errors, see
Ibcce35da928f64e6a719b070bcc833346ee7ee92

Change-Id: Id4c04d7ae20068643df34d2fa31068e8a917a52d
2015-11-04 06:23:17 +11:00
Martin André
d8a0e13261 Fix uniqueness check of initrd in fedora-minimal
The check suffered from various flaws.

First, due to missing quotes around $initrd, 'wc -l' would always see
1 line no matter how many results the find returned.

Second, echo adds a line break making 'wc -l' count 1 even for empty
string. We need to add a check for empty string.

Change-Id: Ib2c67960f566dbdc471d9585a4cef1beb1cc38ab
Closes-Bug: #1506692
2015-11-04 06:23:17 +11:00
Jenkins
449b4e205e Merge "Preserve env when calling yum with sudo" 2015-11-03 10:36:33 +00:00
OpenStack Proposal Bot
943ab4b88e Updated from global requirements
Change-Id: I93e693c255787272399d53d841627459e46e8974
2015-11-03 04:29:38 +00:00
Jenkins
ce16f9d38c Merge "Remove quotes from subshell call in bash script" 2015-11-02 21:07:20 +00:00
Jenkins
727b4cddf6 Merge "Prettyfy source-repositories doc" 2015-11-02 20:14:10 +00:00
Jenkins
40e9705655 Merge "Prettify 'Caches and offline mode' documentation" 2015-11-02 20:14:03 +00:00
Jenkins
2d826fb1f2 Merge "Prettify 'Developing Elements' documentation" 2015-11-02 20:14:00 +00:00
Jenkins
4d986ce3a5 Merge "Enable decimal value for $DIB_IMAGE_SIZE." 2015-11-02 20:13:22 +00:00
Antoine Musso
23be09ce35 debian: cloud hostname ignored by Jessie
5af25b5f fixed the hostname of Debian images to "debian" since a lack of
hostname definition set the hostname to "(None)".

It has been done by introducing /etc/cloud/cloud.cfg.d/01_hostname.cfg
with content:

  hostname: debian

Review supposed the hostname would be overriden by cloud meta-data. That
might have stand true for Wheezy but it is not the case for Jessie.

cloud-init 0.7.6 ignores cloud metadata whenever "hostname" or "fqdn"
are set in a config file. Roughly:

    # no fqdn set, get fqdn from cloud
    # get hostname from cfg if available otherwise cloud
    fqdn = cloud.get_hostname(fqdn=True)
    if "hostname" in cfg:
        # hashar: set from config file NOT cloud
        hostname = cfg['hostname']
    else:
        # fallback to cloud
        hostname = cloud.get_hostname()

Relevant code is
https://github.com/number5/cloud-init/blob/0.7.6/cloudinit/util.py#L839-L860

Only inject "hostname: debian" for the Wheezy release.

Bug: https://phabricator.wikimedia.org/T117283
Change-Id: I6e2522bd725cbf9651f11c76ecdc72ecbc92f402
2015-10-30 22:34:10 +01:00
Martin André
bc2199144e Selectively prune /root for ironic-agent ramdisk
Previously all files in /root were ignored when building the
ironic-agent ramdisk. This prevented for example to use the
local-config element to connect to the ramdisk via ssh as root user.

This commit change the exclude rule on /root to only ignore the
/root/.cache directory.

Change-Id: I18d839e8d97636f5f2164ba407f252407d9bc956
Closes-Bug: #1451668
2015-10-26 11:19:28 +09:00
Dmitry Tantsur
0e122e8e35 Add a tox target to run functional tests locally
Now 'tox -efunc' can be invoked to run all functional tests in
the 'venv' tox environment. Also `tox -efunc element-name` can be
used to run function tests for one element (e.g. ironic-agent).

Change-Id: Ia685d1b2a7deef2f8b98876ac09792134dd30f2f
2015-10-23 12:05:23 +02:00
Ian Wienand
94a7a7cf67 Create YUM_CACHE_DIR in yum-minimal
yum-minimal/root.d/08-yum-chroot runs before yum/root.d/50-yum-cache,
and thus if run on a completely fresh system will fail in
08-yum-chroot as the YUM_CACHE directory isn't made.

This is probably hidden by testing & nodepool builds, because it sets
DIB_IMAGE_CACHE.  It was hidden from me because locally I have done
builds using the "yum" element previously, which had created the
cache.

Change-Id: I333f5f7e67d198f75a522cc296c118c2e94a5ecb
2015-10-23 15:17:18 +11:00
Jenkins
9b45586f0e Merge "Use DIB_EPEL_MIRROR when finding the epel-release package" 2015-10-21 15:08:16 +00:00
Jenkins
48a9dd4d3d Merge "Fix tests/test_functions.bash" 2015-10-20 22:15:46 +00:00
Derek Higgins
0c0c7275a9 Use DIB_EPEL_MIRROR when finding the epel-release package
download.fedoraproject.org uses dns round robin and occasionally
hits a bad server. Using DIB_EPEL_MIRROR when finding the
epel-release package will allow us to avoid it e.g. in ci.

Change-Id: I756223b3e669532476663c05e79c238449b8a0db
2015-10-20 17:25:58 +01:00