iso element to build bootable ISO images

This commit adds a new element named 'iso' to build a bootable
ISO image for the kernel/ramdisk emitted by the 'baremetal' or
'ramdisk' element.

Change-Id: I89d175a29e2d0bc64b47fe527f0d0f6875f6849a

README.md

@@ -21,7 +21,7 @@ What tools are there?
 * disk-image-create [-a i386|amd64|armhf] -o filename {element} [{element} ...]
   Create an image of element {element}, optionally mixing in other elements.
   Element dependencies are automatically included. Support for other
-  architectures depends on your environment being able to run binaries of that 
+  architectures depends on your environment being able to run binaries of that
   platform. For instance, to enable armhf on Ubuntu install the qemu-user-static
   package. The default output format from disk-image-create is qcow2. To instead
   output a tarball pass in "-t tar". This tarball could then be used as an image
@@ -104,7 +104,7 @@ disk-image-builder for caching. When invoking disk-image-builder the --offline
 option will instruct disk-image-builder to not refresh cached resources.
 
 Note that we don't maintain operating system package caches, instead depending
-on your local infrastructure (e.g. Squid cache, or an APT or Yum proxy) to 
+on your local infrastructure (e.g. Squid cache, or an APT or Yum proxy) to
 facilitate caching of that layer, so you need to arrange independently for
 offline mode.
 
@@ -222,7 +222,7 @@ contents can be modelled as three distinct portions:
 The goal of the image building tools is to create machine images that contain
 the correct global content and are ready for 'last-mile' configuration by the
 nova metadata API, after which a configuration management system can take over
-(until the next deploy, when it all starts over from scratch). 
+(until the next deploy, when it all starts over from scratch).
 
 Existing elements
 -----------------
@@ -269,7 +269,7 @@ two-digit numeric prefix, and are executed in numeric order.
 
 * root.d: Create or adapt the initial root filesystem content. This is where
   alternative distribution support is added, or customisations such as
-  building on an existing image. 
+  building on an existing image.
 
   Only one element can use this at a time unless particular care is taken not
   to blindly overwrite but instead to adapt the context extracted by other
@@ -369,7 +369,7 @@ Each element can use the following files to define or affect dependencies:
 
 Ramdisk elements support the following files in their element directories:
 
-* binary-deps.d : text files listing executables required to be fed into the 
+* binary-deps.d : text files listing executables required to be fed into the
   ramdisk. These need to be present in $PATH in the build chroot (i.e. need to
   be installed by your elements as described above).
 
@@ -399,6 +399,10 @@ Global image-build variables
   the operation should still be attempted as the user may have an external
   cache able to keep the operation functional.
 
+* DIB\_IMAGE\_ROOT\_FS\_UUID : this contains the UUID of the root fs, when
+  diskimage-builder is building a disk image. This works only for ext
+  filesystems.
+
 Structure of an element
 -----------------------
 
@@ -512,7 +516,7 @@ Copyright
 =========
 
 Copyright 2012 Hewlett-Packard Development Company, L.P.
-Copyright (c) 2012 NTT DOCOMO, INC. 
+Copyright (c) 2012 NTT DOCOMO, INC.
 
 All Rights Reserved.
 

bin/disk-image-create

@@ -145,6 +145,10 @@ if [ "${#IMAGE_TYPES[@]}" = "1" ]; then
   export IMAGE_NAME=${IMAGE_NAME%%\.${IMAGE_TYPES[0]}}
 fi
 
+# NOTE: Tuning the rootfs uuid works only for ext filesystems.
+# Rely on the below environment variable only for ext filesystems.
+export DIB_IMAGE_ROOT_FS_UUID=$(uuidgen -r)
+
 # FS_TYPE isn't available until after we source img-defaults
 if [ -z "$DIB_ROOT_LABEL" ]; then
     # NOTE(bnemec): XFS has a limit of 12 characters for filesystem labels
@@ -206,6 +210,10 @@ export EXTRA_UNMOUNT="detach_loopback $LOOPDEV"
 export IMAGE_BLOCK_DEVICE=$LOOPDEV
 eval_run_d block-device "IMAGE_BLOCK_DEVICE="
 sudo mkfs $MKFS_OPTS -t $FS_TYPE -L ${DIB_ROOT_LABEL} ${IMAGE_BLOCK_DEVICE}
+# Tuning the rootfs uuid works only for ext filesystems.
+if echo "$FS_TYPE" | grep -q "^ext"; then
+    sudo tune2fs ${IMAGE_BLOCK_DEVICE} -U ${DIB_IMAGE_ROOT_FS_UUID}
+fi
 mkdir $TMP_BUILD_DIR/mnt
 sudo mount ${IMAGE_BLOCK_DEVICE} $TMP_BUILD_DIR/mnt
 sudo mv -t $TMP_BUILD_DIR/mnt ${TMP_BUILD_DIR}/built/*

elements/iso/README.md

@@ -0,0 +1,38 @@
+Generates a bootable ISO image from the kernel/ramdisk generated by the
+elements ``baremetal`` or ``ramdisk``. It uses isolinux to boot on BIOS
+machines and grub to boot on EFI machines.
+
+This element has been tested on the following distro(s):
+* ubuntu
+* fedora
+
+**NOTE**: For other distros, please make sure the ``isolinux.bin`` file
+exists at ``/usr/lib/syslinux/isolinux.bin``.
+
+baremetal element
+-----------------
+When used with ``baremetal`` element, this generates a bootable ISO image
+named ``<image-name>-boot.iso`` booting the generated kernel and ramdisk.
+It also automatically appends kernel command-line argument
+'root=UUID=<uuid-of-the-root-partition>'.  Any more kernel command-line
+arguments required may be provided by specifying them in
+``DIB_BOOT_ISO_KERNEL_CMDLINE_ARGS``.
+
+**NOTE**: It uses pre-built efiboot.img by default to work for UEFI machines.
+This is because of a bug in latest version of grub[1].  The user may choose
+to avoid using pre-built binary and build efiboot.img on their own machine
+by setting the environment variable DIB\_UEFI\_ISO\_BUILD\_EFIBOOT to 1 (this
+might work only on certain versions of grub). The current efiboot.img was
+generated by the method build\_efiboot\_img() in 100-build-iso on
+Ubuntu 13.10 with grub 2.00-19ubuntu2.1.
+
+ramdisk element
+---------------
+When used with ``ramdisk`` element, this generates a bootable ISO image
+named ``<image-name>.iso`` booting the generated kernel and ramdisk. It also
+automatically appends kernel command-line argument 'boot\_method=vmedia'
+which is required for Ironic drivers ``iscsi_ilo``.
+
+**REFERENCES**
+
+[1] https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1378658

elements/iso/cleanup.d/100-build-iso

@@ -0,0 +1,160 @@
+#!/bin/bash
+#
+# Copyright 2014 Hewlett-Packard Development Company, L.P.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+
+set -eux
+set -o pipefail
+
+function build_efiboot_img() {
+
+    cat > "$TMP_BUILD_DIR/grub-embedded.cfg" << END_CONFIG
+search --file --set=root /vmlinuz
+set prefix=(\$root)/EFI/BOOT/
+END_CONFIG
+
+    grub-mkimage --format=x86_64-efi --output=$TMP_BUILD_DIR/bootx64.efi \
+    --config=$TMP_BUILD_DIR/grub-embedded.cfg --compression=xz \
+    --prefix=/EFI/BOOT part_gpt part_msdos fat ext2 hfs hfsplus \
+    iso9660 udf ufs1 ufs2 zfs chain linux boot appleldr ahci \
+    configfile normal regexp minicmd reboot halt search \
+    search_fs_file search_fs_uuid search_label gfxterm gfxmenu \
+    efi_gop efi_uga all_video loadbios gzio echo true probe \
+    loadenv bitmap_scale font cat help ls png jpeg tga test \
+    at_keyboard usb_keyboard
+
+    # Create a FAT formatted image that contains bootx64.efi in the /EFI/BOOT
+    # directory.  This is used to bootstrap GRUB from the ISO image.
+    dd if=/dev/zero of=$TMP_BUILD_DIR/efiboot.img bs=1K count=1440
+    mkdosfs -F 12 $TMP_BUILD_DIR/efiboot.img
+
+    # Create a temporary mount point:
+    MOUNTPOINT=$TMP_BUILD_DIR/tmpmount
+    mkdir $MOUNTPOINT
+    sudo mount -o loop $TMP_BUILD_DIR/efiboot.img $MOUNTPOINT
+    sudo mkdir -p $MOUNTPOINT/EFI/BOOT
+    sudo cp $TMP_BUILD_DIR/bootx64.efi $MOUNTPOINT/EFI/BOOT
+    sudo umount $MOUNTPOINT
+    rmdir $MOUNTPOINT
+    cp $TMP_BUILD_DIR/efiboot.img $TMP_IMAGE_DIR/isolinux
+}
+
+function build_iso() {
+
+    KERNEL=$1
+    INITRD=$2
+    KERNEL_CMDLINE_ARGS=$3
+    OUTPUT_FILENAME=$4
+
+    SCRIPTNAME=$(basename $0)
+    SCRIPTDIR=$(dirname $0)
+    MKISOFS="/usr/bin/mkisofs"
+    EFI_BOOT_DIR="EFI/BOOT"
+
+    # Create a temporary build directory for holiding the contents of iso
+    TMP_IMAGE_DIR="$TMP_BUILD_DIR/image"
+    echo "Creating temporary directory $TMP_IMAGE_DIR"
+    mkdir -p "$TMP_IMAGE_DIR"
+
+    # Copy isolinux bin to the isolinux directory
+    mkdir -p "$TMP_IMAGE_DIR/isolinux"
+    echo "Copying isolinux.bin"
+
+    # TODO(rameshg87): Something similar to pkg-map can be used here.
+    # But pkg-map doesn't work for cleanup scripts right now.
+    if [ $DISTRO_NAME = "fedora" ]; then
+        ISOLINUX_BIN=/usr/share/syslinux/isolinux.bin
+    else
+        ISOLINUX_BIN=/usr/lib/syslinux/isolinux.bin
+    fi
+
+    cp $ISOLINUX_BIN "$TMP_IMAGE_DIR/isolinux"
+
+    # Copy initrd, kernel
+    echo "Copying kernel to $TMP_IMAGE_DIR/vmlinuz"
+    cp $KERNEL "$TMP_IMAGE_DIR/vmlinuz"
+
+    echo "Copying initrd to $TMP_IMAGE_DIR/initrd"
+    cp $INITRD "$TMP_IMAGE_DIR/initrd"
+
+    # Generate isolinux.cfg for default booting
+    echo "Generating isolinux.cfg"
+    cat > "$TMP_IMAGE_DIR/isolinux/isolinux.cfg" << END_CONFIG
+DEFAULT install
+LABEL install
+menu label "Install image"
+    kernel /vmlinuz
+    append initrd=/initrd $KERNEL_CMDLINE_ARGS --
+TIMEOUT 5
+PROMPT 0
+END_CONFIG
+
+
+    echo "Creating EFI/BOOT directory"
+    mkdir -p "$TMP_IMAGE_DIR/$EFI_BOOT_DIR"
+
+    # Generate grub.cfg for default booting
+    echo "Generating grub.cfg"
+    cat > "$TMP_IMAGE_DIR/$EFI_BOOT_DIR/grub.cfg" << END_CONFIG
+set default="0"
+set timeout="5"
+set hidden_timeout_quiet=false
+
+menuentry "install" {
+    linux /vmlinuz  $KERNEL_CMDLINE_ARGS --
+    initrd /initrd
+}
+END_CONFIG
+
+
+    DIB_UEFI_ISO_BUILD_EFIBOOT=${DIB_UEFI_ISO_BUILD_EFIBOOT:-}
+    if [ -n "$DIB_UEFI_ISO_BUILD_EFIBOOT" ]; then
+        build_efiboot_img
+    else
+        cp $SCRIPTDIR/efiboot.img $TMP_IMAGE_DIR/isolinux
+    fi
+
+    # Create the ISO
+    echo "Generating the ISO"
+    $MKISOFS -r -V "INSTALL_IMAGE" -cache-inodes -J -l \
+    -b isolinux/isolinux.bin  -no-emul-boot \
+    -boot-load-size 4 -boot-info-table \
+    -eltorito-alt-boot -e isolinux/efiboot.img \
+    -no-emul-boot -o $OUTPUT_FILENAME $TMP_IMAGE_DIR
+
+}
+
+IMAGE_NAME=${IMAGE_NAME:-'image'}
+
+if echo $IMAGE_ELEMENT | grep -q '\bramdisk\b'; then
+    EMITTED_KERNEL=$IMAGE_NAME.kernel
+    EMITTED_RAMDISK=$IMAGE_NAME.initramfs
+    EMITTED_KERNEL_CMDLINE_ARGS="boot_method=vmedia"
+    EMITTED_ISO_FILENAME=$IMAGE_NAME.iso
+elif echo $IMAGE_ELEMENT | grep -q '\bbaremetal\b'; then
+    EMITTED_KERNEL=${IMAGE_NAME}.vmlinuz
+    EMITTED_RAMDISK=${IMAGE_NAME}.initrd
+    EMITTED_KERNEL_CMDLINE_ARGS="root=UUID=$DIB_IMAGE_ROOT_FS_UUID"
+    DIB_BOOT_ISO_KERNEL_CMDLINE_ARGS=${DIB_BOOT_ISO_KERNEL_CMDLINE_ARGS:-}
+    if [ -n "$DIB_BOOT_ISO_KERNEL_CMDLINE_ARGS" ]; then
+        EMITTED_KERNEL_CMDLINE_ARGS="$EMITTED_KERNEL_CMDLINE_ARGS $DIB_BOOT_ISO_KERNEL_CMDLINE_ARGS"
+    fi
+    export EMITTED_ISO_FILENAME="$IMAGE_NAME-boot.iso"
+else
+    echo "Cannot find the kernel/ramdisk to build the iso image. "
+    echo "Please use 'iso' element with either 'baremetal' or 'ramdisk' elements"
+fi
+
+build_iso "$EMITTED_KERNEL" "$EMITTED_RAMDISK" "$EMITTED_KERNEL_CMDLINE_ARGS" \
+    "$EMITTED_ISO_FILENAME"