Document an interface for offline operation.

Complex image builds can download hundreds of MB of data from the
internet with many separate lookups. It would be nice to allow users
to ask for a fast build where those lookups are entirely avoided,
using locally cached resources (where possible). This new interface
allows users to signal to elements that they wish to operate without
updating cached resources, which will in turn allow us to avoid
checking for stale data at all.

As part of this I've also documented where we cache data, so that
things like the ccache cache dir and image cache files are not a
surprise to users.

Change-Id: I27f5de6ceaa4e9c6390721b7c434fe0908df84f5

README.md

@@ -79,6 +79,21 @@ uncompressed cloud image and about 20% of that for working files.
 
 \* As reported by /proc/meminfo MemTotal
 
+Caches and offline mode
+=======================
+
+Since retrieving and transforming operating system image files, git
+repositories, Python or Ruby packages, and so on can be a significant overhead,
+we cache many of the inputs to the build process in ~/.cache/image-create/. The
+writing an element documention describes the interface within
+disk-image-builder for caching. When invoking disk-image-builder the --offline
+option will instruct disk-image-builder to not refresh cached resources.
+
+Note that we don't maintain operating system package caches, instead depending
+on your local infrastructure (e.g. Squid cache, or an APT or Yum proxy) to 
+facilitate caching of that layer, so you need to arrange independently for
+offline mode.
+
 Design
 ======
 
@@ -222,6 +237,14 @@ Ramdisk elements support the following files in their element directories:
 
 * udev.d : udev rules files that will be copied into the ramdisk.
 
+Global image-build variables
+----------------------------
+
+* DIB\_OFFLINE : this is always set. When not empty, any operations that
+  perform remote data access should avoid it if possible. If not possible
+  the operation should still be attempted as the user may have an external
+  cache able to keep the operation functional.
+
 Structure of an element
 -----------------------
 

bin/disk-image-create

@@ -43,6 +43,7 @@ function show_options () {
     echo "    -u -- uncompressed; do not compress the image - larger but faster"
     echo "    -c -- clear environment before starting work"
     echo "    --no-tmpfs -- do not use tmpfs to speed image build"
+    echo "    --offline -- do not update cached resources"
     if [ "$IS_RAMDISK" == "0" ]; then
         echo "    -n skip the default inclusion of the 'base' element"
         echo "    -p package[,package,package] -- list of packages to install in the image"
@@ -71,6 +72,7 @@ while true ; do
         -n) shift; export SKIP_BASE="1";;
         -p) IFS="," read -a INSTALL_PACKAGES <<< "$2"; export INSTALL_PACKAGES ; shift 2 ;;
         --no-tmpfs) shift; export DIB_NO_TMPFS=1;;
+        --offline) shift; export DIB_OFFLINE=1;;
         --) shift ; break ;;
         *) echo "Internal error!" ; exit 1 ;;
     esac

lib/common-defaults

@@ -39,3 +39,4 @@ export DIB_NO_TMPFS=${DIB_NO_TMPFS:-0}
 _BASE_ELEMENT_DIR=$(dirname $0)/../elements
 ELEMENTS_PATH=${ELEMENTS_PATH:+"$ELEMENTS_PATH:$_BASE_ELEMENT_DIR"}
 export ELEMENTS_PATH=${ELEMENTS_PATH:-$_BASE_ELEMENT_DIR}
+export DIB_OFFLINE=''