Currently when dib-lint finds a problem it does something like:
echo "ERROR: Problem found"
rc=1
This is repetitive and error-prone since it's easy to forget to set
rc to actually fail the check. This change makes those two steps
a single function call.
Change-Id: I40b5bf39348a69add1f955c49f310e3bda21be0e
With this change, dib-lint raises an error if when finding an element
using space indentation that is not multiple of 4.
Co-Authored-By: Jon-Paul Sullivan <jonpaul.sullivan@hp.com>
Change-Id: I470e1fdfc38a3f3c7ba5644c5103f2a9ef073005
This will hopefully catch bugs where they happen rather than
allowing scripts to continue on and fail later.
Change-Id: Idacd9274415b21db285a198dafff19b1d19a4a68
With this change, dib-lint ensure elements do not use tab
indentation. The following files are checked:
- executable file
- .md file
Change-Id: I071262ff9f6599548f869f5439ee127f64eeb46f
If scripts are not set -e then errors can be ignored, causing more
confusing failures later.
Also adds an exclusion comment to the ramdisk init script since we
don't want that to exit on failure.
Change-Id: Idf43993bd10b1ef16c1d3b0d9df8d0ad94c46458
There are certain scripts (such as the ramdisk init script) that
need to ignore linting rules for one reason or another. This adds
support for exclusions via a comment in the file like:
"# dib-lint: disable=executable sete"
There should be no ", but I need those to prevent git from treating
that as a comment. This syntax is similar to the pylint exclusion
mechanism.
Exclusion support is added to the executable check, but not
the alphabetical ordering one because I can't imagine a reason we
would need to disable that, and I don't know that comments are
supported in those files anyway.
Change-Id: I9ecfb47269841dc75a005855455ac26ad2cbc642
The phrase is no longer needed as of August 23, 2000 with Nicaragua's
joining of the Berne Convention.
Additionally, in at least one instance,
elements/cache-url/bin/cache-url, its existence in the file between
Copyright lines is just weird and feels misleading, even though it is
not.
Remove all of the lines, because sanity.
Change-Id: I24fd76c2b4f66b8036010b5079db39ead729abee
We've started to require this in reviews, so we should really have
automation in place to catch it right away.
Change-Id: I43fd90647acba400cea11c665fb587856514b0ee
This will provide a place to put checks that catch common errors
in elements. To start, this just checks that files starting with
a shebang are chmod +x so they can actually be run.
Change-Id: I4116a8f38f7bdfc5866764354c459fad8ca18e92
save_image is used to copy kernel and ramdisks out of the image, which
we will sometimes want to keep the source, and sometimes not. However
for the main image itself, the temp copy is never kept, so use mv
rather than cp and avoid the excess IO.
Change-Id: I5a9f0d69ffee3e6b872a8927537ac17f02f5aa4d
A recent change[1] relies on IMAGE_NAME being available when
98-source-repositories was run. However IMAGE_NAME was previously only
exported if the -o option had been used.
[1] I7dbe9e163ad38a418cf2869a81e720de2c27dfb1
Change-Id: Ife3d94fd45a2e0e0948c17414c369e0e6e040442
In fb246a02eb we introduced an
ext4 option to allow root filesystems to be resized up to 1PB.
This appears to cause an ext4 resize2fs bug in some images.
When the issue occurs an image will hit a kernel bug when
cloud-init runs the resize2fs command during first boot:
kernel BUG at fs/ext4/resize.c:409!
In this commit we add a new option for max-online-resize
which can be used if a really large root partition is
desirable.
The root cause of all this is really a design problem in
DIB/TripleO at the moment in that we shouldn't have to worry about the
max size of the root file system when creating our images. Ideally we'd
just mkfs on the root file system itself. Much more efficient,
avoids this problem altogether...
I think the best thing to do today to avoid this is make setting
max-online-resize an option in DIB. This will allow us to stick
to the (well tested) ext4 defaults for most cases, and if someone has
need for a large root filesystem they can easily bump the setting. This
may be temporary until we either fix the design... or the ext4 fix is
released.
Change-Id: I371f62555d2753cec48790c8fd811c4342af925c
Closes-bug: #1280709
Fixes issues with option parsing. The --min-tmpfs and --image-cache
options where previously broken.
Closes-bug: 1282077
Change-Id: I40c62b16d854335902d1b6b5ceab7f0e1992623a
Rather than using a script to mount the image using nbd to extract the
kernel and ramdisk, make a new element called baremetal, which contains
a cleanup.d script that will copy them out to <image name>.{vmlinuz,initrd}.
Closes-Bug: 1224669
Change-Id: I8f3569aa12148d18b1c8242b6fbbd8857894b26f
With smaller base images, such as Debian, the padding value used to
size the root filesystem does not leave enough room for grub which is
installed after the filesystem is created.
Change-Id: Ic2ab9e2efc9bf1b02f802ee47a36e3fff9c3512e
Occasionally cloud-init fails to resize the disk on first boot, this is
occuring because the filesystem has only 10% free. Nudging the disk size
up a bit more should give us a little more headroom so this is less
likely to occur.
I havn't been able to find out what value ensures we wont hit this
problem but I can say for sure this fixes all casses of the problem I
have seen.
Change-Id: Ib23f3a654151338ad91839e49b323b65b4054245
disk-image-get-kernel is pretty noisy and you see e.g.:
$> load-image overcloud-compute.qcow2
XXX -d '/tmp/image.lWGCgPoj' -o 'tmp' -i '/home/stack/overcloud-compute.qcow2' --
Extracting kernel + ramdisk from /home/stack/overcloud-compute.qcow2 and writing them to /tmp/image.lWGCgPoj
nbd 17554 0
nbd 17554 0
basename: missing operand
Try 'basename --help' for more information.
/dev/nbd0 disconnected
tmp-vmlinuz,tmp-initrd
Clean all this up so we just get:
$> load-image overcloud-compute.qcow2
Extracting kernel + ramdisk from /home/stack/overcloud-compute.qcow2 to tmp-vmlinuz and tmp-initrd in /tmp/image.g6b0lG88
Change-Id: I8971ec0bbcd87157b07fc17254c56bb9f9f2a597
Converts our existing default root element code to be just a check
which exits with a failure message if no root/distribution element
is found.
Change-Id: I954a6abfd7871d5807b1a171a03fa98932410cff
Or environment variables like OS_USERNAME OS_PASSWORD will be stored
in /etc/dib_environment also. It's potentially unsafe.
Change-Id: I3a65d5bc0e4469a071db9ac23ebc82196a3f3feb
Adds an option for --image-size, which sets the DIB_IMAGE_SIZE
environment variable. Having the cli option makes disk-image-create
more consistent in that parameters are able to be specified via a flag
on the command line or an environment variable. Also, you may not be
able to remember all the environment variables that you can tweak, yet
--help will tell you about the cli parameters. Preserves the old
behavior as well where if DIB_IMAGE_SIZE is set in your environment, it
will be honored.
Change-Id: I195c9144a80ce7b8bd5809b57f2bed71a2cbdf26
This commit refactors the python code in diskimage_builder slightly. This is
in preparation for a larger review that adds more functionality to the python
code, namely the ability to apply elements to the current system as opposed to
a chroot. Further, the refactorings can stand on their own for better clarity.
They include:
- renaming elements.py to element_dependencies.py. Adds clarity about the
purpose of this module.
- updating other code for this rename.
- move tests into a tests submodule.
Change-Id: I5519cc52398e442b24e33802bae42070d64b0c1d
Commit 0210be22ae introduced the
possibility of basename being called on an empty string. Because
that can happen in normal operation (such as an x86_64 kernel,
in which case the PAE check will find nothing), don't allow the
basename error to kill the script. Also ignore failures in the
second basename call so the proper error message is echoed.
Change-Id: I38a18af09ab24fda9c98cbf3ace8fd7acc6faef5
disk-image-get-kernel picks the wrong version of kernel and initramfs
if there's a PAE and non-PAE version present. Only affects i386 images.
Change-Id: I06e08fdf038988759b620f549261499cb0a69b34
Closes-Bug: #1240873
the problem is that the journal isn't large enough to allow online
resizing. Solution is straight forward. So the file system can be
resized successfully to disk size specified in flavor.
Fixes bug #1233008
Change-Id: Ie84fb8aea8d334706574d1a8006ec9eaee5bb5be
DIB_IMAGE_CACHE will be a user override for the location where images
are cached. Default location is ~/.cache/image-create
Change-Id: I3e9b9f970864d555c9ec9436344b53f6d3d66dfa
For example if there are following kernels in undercloud/overcloud
image:
/boot/vmlinuz-3.9.5-301.fc19.i686.PAE
/boot/vmlinuz-3.10.10-200.fc19.i686.PAE
then disk-image-get-kernel picks vmlinuz-3.9.5. It should use the
newest one.
Change-Id: I7bbf06705e85370d66c7dd8a5d4f8d6c93b21c0c
Fixes: bug #1224365
In some scenarios, the required space in the tmpfs partition can be
larger (or smaller) than the default one, producing errors due to
the lack of enough space (or performance penalties for not using
tmpfs).
Using --min-tmpfs <size>, we can hint the working set size we'll need
and let dib choose to avoid or use tmpfs.
Change-Id: I7d5fe498302a100c8555ae542268e14b21f3a0c5
When compressing an image, this is done in the same dir where the raw
image resides, doubling the amount of space needed (scarce when
using tmpfs), and then it's moved to the .cache folder in disk.
Combining these two functions, we reduce the amount of space needed
in the tmpfs partition (when in use), and the compressed image is
created directly on the .cache folder disk, so there is no need to
move the compressed image after the process into disk.
Change-Id: I451d24bdd6fa0983414244135dff5e96c0549833
A user running di-b several times while developing an element may not
want to drop to a shell in all cases but may only want to do so if one
of their in target hooks failed.
This patch gives them the ability to do so, If break=after-error is set
then a user will be provided a in target shell taking over from where the
last failed command left off.
Change-Id: Ia2f7ac4c21b64b971f87f4ae9cb867981b13eb5e
(Based on review https://review.openstack.org/#/c/36009)
Scripts test for existence of ../share/diskimage-builder and
fall-back to ../ if not found. This allows scripts to run unmodified
from a packaged installation or a local archive/repository.
Change-Id: I0cf4c1fdb8e42ec284c56860cb15818632b93b9e
- Ensures /sbin and friends are in $PATH when invoked (without this,
various sudo invocations fail in exciting ways).
- Use dib-run-parts in lib/common-functions instead of run-parts
(neither SLES nor openSUSE ship run-parts).
- Ensure dib-run-parts doesn't descend into subdirectories (same
behaviour as run-parts).
- Move dib-run-parts from root.d to bin (cleaner, consistent with
other elements with separate bin scripts).
- Tested by building Ubuntu image on openSUSE 12.3.
- Note: this doesn't add support for creating SUSE images, it just
lets you run disk-image-create on SUSE-based distros.
Change-Id: I906c6bc3cf51cdf2c4415adeae1ca250faac25e1
I missed the getopt parameter and forgot defaults are imported after
option processing. Untested code is broken code!
Change-Id: I133a691909d38e834c204950276a57f4884fc4ed
Complex image builds can download hundreds of MB of data from the
internet with many separate lookups. It would be nice to allow users
to ask for a fast build where those lookups are entirely avoided,
using locally cached resources (where possible). This new interface
allows users to signal to elements that they wish to operate without
updating cached resources, which will in turn allow us to avoid
checking for stale data at all.
As part of this I've also documented where we cache data, so that
things like the ccache cache dir and image cache files are not a
surprise to users.
Change-Id: I27f5de6ceaa4e9c6390721b7c434fe0908df84f5
Ramdisks are now built inside a chroot which is built by the normal
image build process. Doing so improves our independence of the
precise state of the build host.
This fixes bug 1194055.
Change-Id: Ibc254fbb9e7b404b5f38c1b35bcde8a4136e8e28
