Commit Graph

1428 Commits

Author SHA1 Message Date
Jane.zhang
7eb70becd3 Fix grub-efi-amd64-signed install failure
When build ubuntu iso image, it will install grub-efi-amd64-signed
and grub-efi-amd64 packages. Both of the postinst script will try
to find root device and install grub which will definitely fail in
such a chroot environment.
So the workaround is to skip error and remove postinst script.
And confirm the package be installed successfully at last.

Change-Id: Ie0aecb212b22362046db55b5ad8c64c3211c28e5
Closes-Bug: #1491280
Co-Authored-By: Jane.zhang <jian.zhang8@hpe.com>
2015-12-01 23:53:35 -08:00
Jenkins
2ff566b80a Merge "Selectively prune /root for ironic-agent ramdisk" 2015-11-13 03:28:13 +00:00
Jenkins
5e571d9f44 Merge "Fixup RPM db path when building Fedora on Ubuntu" 2015-11-10 11:03:22 +00:00
Ian Wienand
3f3850aa0f Fixup RPM db path when building Fedora on Ubuntu
On Debian/Ubuntu installs of RPM, /usr/lib/rpm/macros sets

  %_dbpath  %(echo $HOME/.rpmdb)

which makes quite a bit of sense, because RPM is not the system
packager and thus RPM is setup to install things into a hierarchy in
the users homedir.

However, this messes things up when building a Fedora chroot on an
Ubuntu platform.

We use RPM & yum from the base-system to bootstrap the Fedora chroot.
While both obey --root flags, they still pick up the %_dbpath macro
and so end up creating the RPM database in <chroot>/home/user/.rpmdb

After we have bootstrapped yum/dnf, we execute further installation
commands from inside the chroot -- where we now have the Fedora
version of /usr/lib/rpm/macros and hence have _dbpath set to
/var/lib/rpm -- except there is no rpm database there.

Should anyone be finding this in the future, the actual issue that
appears is

  $ sudo chroot /opt/dib_tmp/image.b6B5S3f6/mnt dnf makecache
   Error: Failed to synchronize cache for repo 'fedora' from \
    'https://mirrors.fedoraproject.org/metalink?repo=fedora-$releasever&arch=x86_64': \
    Cannot prepare internal mirrorlist: file "repomd.xml" was not found in metalink

Note the issue there is that $releasever is not expanded, because the
rpmdb where this info is kept is not populated.

The trick is to make sure we override this value when using the host
rpm/yum to setup the chroot.  The bare rpm calls, which we use to
install the repos, have a --dbpath argument where we can override
this.  yum does not however, so we override this in the global
~/.rpmmacros while we are installing the packaging tools and
dependencies into the chroot.

Copious comments are included, because this is super-confusing.

Change-Id: I20801150ea02d1c64f118eb969fb2aec473476f7
2015-11-10 08:54:44 +00:00
Ian Wienand
8ee21cb9fd Remove unused RELEASE_RPMS variable
It was noticed during reviews of
Ic7aa8cbe13e4347b447e84bb9c12483a4e125228 these are unused

Change-Id: I9e0fa9d3e4864e54c6fe23f8b6e781e8d5d24bda
2015-11-10 07:17:52 +00:00
Jenkins
e90be5a595 Merge "Fix fedora-minimal kernel-install on older platforms" 2015-11-10 05:14:28 +00:00
Jenkins
6f21f47850 Merge "Clarify what fedora-minimal/install.d/99-ramdisk is doing" 2015-11-10 05:14:25 +00:00
Jenkins
dfb40eb5aa Merge "Add --force to grub-install" 2015-11-09 07:14:13 +00:00
Jenkins
8c42e60df5 Merge "Fix uniqueness check of initrd in fedora-minimal" 2015-11-05 03:36:13 +00:00
Jenkins
63610993ed Merge "debian: cloud hostname ignored by Jessie" 2015-11-04 21:46:54 +00:00
Jenkins
29f42e2f15 Merge "Add support for Xen PV disks" 2015-11-04 21:07:25 +00:00
Jenkins
654d80a40f Merge "Define a default for $YUM" 2015-11-03 20:33:38 +00:00
Ian Wienand
f307bb4d8b Fix fedora-minimal kernel-install on older platforms
fedora-minimal fails to build on Ubuntu Trusty due do being unable to
find the initrd (see Id4c04d7ae20068643df34d2fa31068e8a917a52d).

This is a rather obscure problem that comes from the intersection of
several things.

The first thing to note is that the post-install scripts of the
kernel-core package use kernel-install [1].  For whatever reason, this
installs the kernel to /boot/MACHINE-ID/KERNEL-VERSION

MACHINE-ID comes from /etc/machine-id; a UUID that should have been
created by the systemd post-inst scripts with systemd-machine-id-setup
[2].

The chroot environment provided for root.d elements has no kernel
file-systems like /proc or /dev mounted.  This is where differences in
the base-system come into play -- on more recent systems that
implement getrandom() systemd does not need /dev/urandom to generate
the machine-id [3]; we get a value and /etc/machine-id is populated.

On older platforms (Trusty), systemd-machine-id-setup fails (unable to
access /dev/urandom) and we end up with a blank /etc/machine-id.  This
ends up making kernel-install (the script) fail during yum's
installation of kernel-core, which means the initrd is not installed
correctly.

We end up bailing out in fedora-minimal/install.d/99-ramdisk, where we
try to put the installed ramdisk in /boot for the later grub install
scripts to find.

The solution here is to mount the standard kernel file-systems within
the chroot before we try installing.

[1] http://www.freedesktop.org/software/systemd/man/kernel-install.html
[2] http://www.freedesktop.org/software/systemd/man/systemd-machine-id-setup.html
[3] https://github.com/systemd/systemd/blob/master/src/basic/random-util.c

Change-Id: Ibcce35da928f64e6a719b070bcc833346ee7ee92
2015-11-04 06:23:17 +11:00
Ian Wienand
a67ee17d62 Clarify what fedora-minimal/install.d/99-ramdisk is doing
Clarify what this script is doing.  It currently fails on some
platforms due to earlier errors, see
Ibcce35da928f64e6a719b070bcc833346ee7ee92

Change-Id: Id4c04d7ae20068643df34d2fa31068e8a917a52d
2015-11-04 06:23:17 +11:00
Martin André
d8a0e13261 Fix uniqueness check of initrd in fedora-minimal
The check suffered from various flaws.

First, due to missing quotes around $initrd, 'wc -l' would always see
1 line no matter how many results the find returned.

Second, echo adds a line break making 'wc -l' count 1 even for empty
string. We need to add a check for empty string.

Change-Id: Ib2c67960f566dbdc471d9585a4cef1beb1cc38ab
Closes-Bug: #1506692
2015-11-04 06:23:17 +11:00
Jenkins
449b4e205e Merge "Preserve env when calling yum with sudo" 2015-11-03 10:36:33 +00:00
Jenkins
ce16f9d38c Merge "Remove quotes from subshell call in bash script" 2015-11-02 21:07:20 +00:00
Jenkins
727b4cddf6 Merge "Prettyfy source-repositories doc" 2015-11-02 20:14:10 +00:00
Antoine Musso
23be09ce35 debian: cloud hostname ignored by Jessie
5af25b5f fixed the hostname of Debian images to "debian" since a lack of
hostname definition set the hostname to "(None)".

It has been done by introducing /etc/cloud/cloud.cfg.d/01_hostname.cfg
with content:

  hostname: debian

Review supposed the hostname would be overriden by cloud meta-data. That
might have stand true for Wheezy but it is not the case for Jessie.

cloud-init 0.7.6 ignores cloud metadata whenever "hostname" or "fqdn"
are set in a config file. Roughly:

    # no fqdn set, get fqdn from cloud
    # get hostname from cfg if available otherwise cloud
    fqdn = cloud.get_hostname(fqdn=True)
    if "hostname" in cfg:
        # hashar: set from config file NOT cloud
        hostname = cfg['hostname']
    else:
        # fallback to cloud
        hostname = cloud.get_hostname()

Relevant code is
https://github.com/number5/cloud-init/blob/0.7.6/cloudinit/util.py#L839-L860

Only inject "hostname: debian" for the Wheezy release.

Bug: https://phabricator.wikimedia.org/T117283
Change-Id: I6e2522bd725cbf9651f11c76ecdc72ecbc92f402
2015-10-30 22:34:10 +01:00
Martin André
bc2199144e Selectively prune /root for ironic-agent ramdisk
Previously all files in /root were ignored when building the
ironic-agent ramdisk. This prevented for example to use the
local-config element to connect to the ramdisk via ssh as root user.

This commit change the exclude rule on /root to only ignore the
/root/.cache directory.

Change-Id: I18d839e8d97636f5f2164ba407f252407d9bc956
Closes-Bug: #1451668
2015-10-26 11:19:28 +09:00
Ian Wienand
94a7a7cf67 Create YUM_CACHE_DIR in yum-minimal
yum-minimal/root.d/08-yum-chroot runs before yum/root.d/50-yum-cache,
and thus if run on a completely fresh system will fail in
08-yum-chroot as the YUM_CACHE directory isn't made.

This is probably hidden by testing & nodepool builds, because it sets
DIB_IMAGE_CACHE.  It was hidden from me because locally I have done
builds using the "yum" element previously, which had created the
cache.

Change-Id: I333f5f7e67d198f75a522cc296c118c2e94a5ecb
2015-10-23 15:17:18 +11:00
Derek Higgins
0c0c7275a9 Use DIB_EPEL_MIRROR when finding the epel-release package
download.fedoraproject.org uses dns round robin and occasionally
hits a bad server. Using DIB_EPEL_MIRROR when finding the
epel-release package will allow us to avoid it e.g. in ci.

Change-Id: I756223b3e669532476663c05e79c238449b8a0db
2015-10-20 17:25:58 +01:00
Jenkins
b902189eaa Merge "Fix devuser pubkey defaults" 2015-10-20 01:41:16 +00:00
Bob Ball
b5794f3665 Add support for Xen PV disks
Xen paravirtualised disks (supported by most modern kernels) have the
"xvd" prefix (e.g. xvda0).  The functions to strip partitions need to
match on Xen PV disks otherwise the device name is discarded.

Change-Id: I5539d2afba3fae30d1ddb49dcbf077113d38bbf7
Closes-Bug: #1498576
2015-10-19 15:38:00 +01:00
Gregory Haynes
757b8cee3b Add --force to grub-install
Grub fails to install the bootloader due to it being on the root
partition of a block device. This is not actually a problem for us, so
we need to force it to succeed.

Change-Id: I335ef04ca8a8a8a5c242d3444b09bcce0a9f51e7
2015-10-18 02:26:31 +00:00
Derek Higgins
63641aa2ce Preserve env when calling yum with sudo
In particular we need [http|https]_proxy to be preserved.

Change-Id: I5bcd1b1deac917a1be8d8155f1283e330e3d5862
2015-10-16 17:36:09 +01:00
Derek Higgins
614661ec5b Define a default for $YUM
This is set by elements that require it not to be yum.

Change-Id: Ie01f357eef382bd1549fbe2b911129a3c48818e0
2015-10-16 09:55:00 +01:00
Jenkins
21f5e6146c Merge "Reset yum/dnf cache to correct location" 2015-10-15 13:21:57 +00:00
Jenkins
2ed4c1d860 Merge "Remove extra install of release pkgs in fedora-minimal" 2015-10-15 04:01:08 +00:00
Jenkins
654fb0311d Merge "Update default fedora-minimal to f22" 2015-10-15 04:01:05 +00:00
Jenkins
98bbd432a9 Merge "Update rhel7 element readme" 2015-10-14 19:03:38 +00:00
Colleen Murphy
a6e6570102 Fix devuser pubkey defaults
Without this patch, the devuser element attempts to find public keys by
iterating over the string "rsa dsa". When two keys are grouped together
in quotes, a bash for loop treats it as a single key. You can see the
issue this causes when debug output is turned on:

   + for fmt in '"rsa dsa"'
   + '[' -f '/home/krinkle/.ssh/id_rsa dsa.pub' ']'

This is not a reasonably named key to look for, so this patch removes
the quotes so that the loop will look for id_rsa.pub and id_dsa.pub
separately.

Change-Id: I0b5b1abd14013de85d90e76a95918a8071a5e013
2015-10-13 17:54:09 -07:00
Ian Wienand
283c9705c0 Reset yum/dnf cache to correct location
Make sure we reset the yum/dnf cache to /var/cache/${YUM}, not just
/var/cache/yum

This was resulting in the F22 fedora-minimal image being larger than
the base-image.  Because F22 fedora-minimal does some installs with
dnf when bootstrapping the chroot before we set "cachedir=" to the
bind-mounted external cache, we have "/var/cache/dnf" created and and
populated with the package meta-data, etc.

When we globally point dnf to /var/cache/yum here, we effectively
orphan the /var/cache/dnf created in those first steps.  dnf doesn't
care, but we end up with two copies of all the package metadata, etc
in "/var/cache/dnf" & "/var/cache/yum".

This also cleans up the sed a bit, by just replacing the lines.

Change-Id: Icc98fe30c34cb941aed4b987647ab67ac34af15a
2015-10-14 06:36:06 +11:00
Ian Wienand
c40aa76d83 Remove extra install of release pkgs in fedora-minimal
I'm not sure why we try to do an extra install of these, it is done
inside the chroot in _install_repos.  Currently it just gets skipped
saying the packages are already installed.

Change-Id: Ic7aa8cbe13e4347b447e84bb9c12483a4e125228
2015-10-14 06:36:06 +11:00
Ian Wienand
b00258cfc1 Update default fedora-minimal to f22
Testing shows this works

 $ ./bin/disk-image-create -x vm fedora-minimal
 [wait]
 $ ls -lh image.qcow2
 -rw-r--r--. 1 545M Sep 24 14:11 image.qcow2

Change-Id: Ifab2afa439e62e9ee2909460ac6bb4b07bc41662
2015-10-14 06:36:06 +11:00
Ian Wienand
ebdf48623e Add Fedora 22 support to yum-minimal
Add basic F22/dnf support to yum-minimal path.  We extract common
code, add some comments and reduce duplication.

Change-Id: If4bd5f88e26bd6f2168958f1ec1efff1072de7ba
2015-10-14 06:35:22 +11:00
Ben Nemec
e23274921d Update rhel7 element readme
Evidently the readme file hasn't been updated since rhel7 finished
beta, so this is long overdue.

In addition, since it's not possible to download the base image
file directly, let's stop pretending we can and bail out if the user
didn't set the necessary env vars.

Also updated the README to use the new table format instead of free text

Co-Authored-By: Augustina Ragwitz <aragwitz+lp@pobox.com>
Change-Id: Ie8343ee2ce1715583c28de7f59daed7e58c8ca0f
2015-10-12 15:31:09 -05:00
Jenkins
89d1eaf5a5 Merge "Adds debian support to iso element" 2015-10-12 20:15:07 +00:00
Jenkins
bf4c57dfb8 Merge "Update apt-preferences element README from free text to table formatting" 2015-10-12 19:26:06 +00:00
Jenkins
32a96a536f Merge "Update apt-conf elements README from free text to table formatting" 2015-10-12 19:23:45 +00:00
Jenkins
caaf20926a Merge "Move yum-based install into function" 2015-10-10 00:22:22 +00:00
Ian Wienand
829d626f0a Move yum-based install into function
Move yum-based install into a function, to make way for a second
related function where use dnf later

Change-Id: Iad09f3753ecdfa0c10cb8a0970a3c8e5a2dccab1
2015-10-09 12:03:19 +11:00
Jenkins
ff6abe3e8a Merge "Fix title of env vars section of redhat-common README to match template" 2015-10-09 00:51:19 +00:00
Augustina Ragwitz
b1958bfbbe Update apt-preferences element README from free text to table formatting
Change-Id: Iae04b809c8008487fabe1a173123928ab157c78d
2015-10-08 17:14:16 -07:00
Augustina Ragwitz
b3c448e765 Update apt-conf elements README from free text to table formatting
Change-Id: Ie89ce94fd063e04d57158a2e0faf11719537d450
2015-10-08 17:12:46 -07:00
Augustina Ragwitz
b4cc1a59ec Fix title of env vars section of fedora README to match template
Change-Id: I5cfc977dd21eb6f6b737fa272469a34e6cbc5e24
2015-10-08 17:10:05 -07:00
Augustina Ragwitz
7df60c31b9 Fix title of env vars section of redhat-common README to match template
Change-Id: I37b33eba8af507c03088bc0025280a696e85b04f
2015-10-08 17:05:57 -07:00
Jenkins
77edd112f6 Merge "Avoid transcending /proc with find" 2015-10-08 16:32:29 +00:00
Derek Higgins
bdcd03d604 Avoid transcending /proc with find
Find doesn't like listings disappearing while its trying to find them,
in this case if a PID directory disappears while find is running. Using
-xdev prevents find from going into ./proc and as a side effect /dev
will also be avoided which is mounted on boot so not needed either.

Change-Id: Iaa282e58d81d533ad4445da0a44200dd14bf0850
Closes-bug: #1502142
2015-10-08 12:16:23 +01:00
Nisha Agarwal
1cd45803f2 Reorder the script number of 'elements/dkms/post-install.d/99-dkms'
Reorder the script number of 'elements/dkms/post-install.d/99-dkms'
to 'elements/dkms/post-install.d/97-dkms' to ensure that
it will always get executed before the
'elements/ramdisk/post-install.d/99-build-ramdisk'. This
would make sure that the DKMS module is there in the ramdisk.

Closes bug: #1492904

Change-Id: I2145d0ac29646335f76745a7678d169a62f13d44
2015-10-06 09:57:06 -07:00