Support defining the free space in the image

Currently diskimage-builder supports two ways to specify the image
size. One is defining a fixed image size using DIB_IMAGE_SIZE, the
other one is auto-detection while adding a security margin of 60% as
free space. This means when building larger images (e.g. >100GB) with
unknown size upfront we end up with much wasted space, IO and network
traffic when uploading the images to several cloud providers. This can
be optimized by adding a third way by defining DIB_IMAGE_EXTRA_SIZE to
specify the free space in GB. This makes it possible to easily build
images of varying sizes while still minimizing the overhead by keeping
the free space constant to e.g. 1GB.

Change-Id: I114c739d11d0cfe3b8d8abc6df5ff989edfb67f2
This commit is contained in:
Tobias Henkel 2019-04-23 14:30:48 +02:00 committed by Ian Wienand
parent 11142f75b4
commit 778d007150
3 changed files with 21 additions and 3 deletions

View File

@ -62,6 +62,7 @@ function show_options () {
echo " --logfile -- save run output to given logfile (implies DIB_QUIET=1)" echo " --logfile -- save run output to given logfile (implies DIB_QUIET=1)"
echo " --checksum -- generate MD5 and SHA256 checksum files for the created image" echo " --checksum -- generate MD5 and SHA256 checksum files for the created image"
echo " --image-size size -- image size in GB for the created image" echo " --image-size size -- image size in GB for the created image"
echo " --image-extra-size size -- extra image size in GB for the created image"
echo " --image-cache directory -- location for cached images(default ~/.cache/image-create)" echo " --image-cache directory -- location for cached images(default ~/.cache/image-create)"
echo " --max-online-resize size -- max number of filesystem blocks to support when resizing." echo " --max-online-resize size -- max number of filesystem blocks to support when resizing."
echo " Useful if you want a really large root partition when the image is deployed." echo " Useful if you want a really large root partition when the image is deployed."
@ -148,6 +149,7 @@ while true ; do
-p) IFS="," read -a _INSTALL_PACKAGES <<< "$2"; export INSTALL_PACKAGES=( ${INSTALL_PACKAGES[@]} ${_INSTALL_PACKAGES[@]} ) ; shift 2 ;; -p) IFS="," read -a _INSTALL_PACKAGES <<< "$2"; export INSTALL_PACKAGES=( ${INSTALL_PACKAGES[@]} ${_INSTALL_PACKAGES[@]} ) ; shift 2 ;;
--checksum) shift; export DIB_CHECKSUM=1;; --checksum) shift; export DIB_CHECKSUM=1;;
--image-size) export DIB_IMAGE_SIZE=$2; shift 2;; --image-size) export DIB_IMAGE_SIZE=$2; shift 2;;
--image-extra-size) export DIB_IMAGE_EXTRA_SIZE=$2; shift 2;;
--image-cache) export DIB_IMAGE_CACHE=$2; shift 2;; --image-cache) export DIB_IMAGE_CACHE=$2; shift 2;;
--max-online-resize) export MAX_ONLINE_RESIZE=$2; shift 2;; --max-online-resize) export MAX_ONLINE_RESIZE=$2; shift 2;;
--mkfs-journal-size) export DIB_JOURNAL_SIZE=$2; shift 2;; --mkfs-journal-size) export DIB_JOURNAL_SIZE=$2; shift 2;;
@ -378,8 +380,15 @@ else
echo "Calculating image size (this may take a minute)..." echo "Calculating image size (this may take a minute)..."
sudo du -a -c -x ${TMP_BUILD_DIR}/built > ${du_output} sudo du -a -c -x ${TMP_BUILD_DIR}/built > ${du_output}
# the last line is the total size from "-c". # the last line is the total size from "-c".
# scale this by 0.6 to create a slightly bigger image if [ -n "$DIB_IMAGE_EXTRA_SIZE" ]; then
du_size=$(tail -n1 ${du_output} | cut -f1 | awk '{print int($1 / 0.6)}') # add DIB_IMAGE_EXTRA_SIZE to create a bigger image as requested
du_extra_size=$(echo "$DIB_IMAGE_EXTRA_SIZE" | awk '{printf("%d\n",$1 * 1024 *1024)}')
du_size_tmp=$(tail -n1 ${du_output} | cut -f1)
du_size=$(echo "$du_size_tmp $du_extra_size" | awk '{print int($1 + $2)}')
else
# scale this by 0.6 to create a slightly bigger image
du_size=$(tail -n1 ${du_output} | cut -f1 | awk '{print int($1 / 0.6)}')
fi
$xtrace $xtrace
fi fi

View File

@ -617,7 +617,7 @@ more complicated block-device layouts with multiple partitions, you
may need to take into account the special behaviour described below. may need to take into account the special behaviour described below.
The ``local_loop`` module will take it's default size from the The ``local_loop`` module will take it's default size from the
following argument. following arguments:
``--image-size`` ``--image-size``
The size of loopback device which the image will be generated in, The size of loopback device which the image will be generated in,
@ -625,6 +625,11 @@ following argument.
from the on-disk size of the image and then scaled up by a fixed from the on-disk size of the image and then scaled up by a fixed
60% factor. Can also set ``DIB_IMAGE_SIZE``. 60% factor. Can also set ``DIB_IMAGE_SIZE``.
``--image-extra-size``
Extra space to add when automatically calculating image size, in
gigabytes. This overrides the default 60% scale up as described
above for ``--image-size``. Can also set ``DIB_IMAGE_EXTRA_SIZE``.
The special node named ``mkfs_root`` is affected by the following; The special node named ``mkfs_root`` is affected by the following;
this reflects that the standard layout has only a single root this reflects that the standard layout has only a single root
partition so the options are, in effect, global for the default partition so the options are, in effect, global for the default

View File

@ -0,0 +1,4 @@
---
features:
- The ``--image-extra-size`` option is provided to override the default
60% padding growth of the image size with a fixed gigabyte value.