Commit Graph

54 Commits

Author SHA1 Message Date
Adam Williamson
6ee0e2217d createhdds: use a temp file for virt-builder image creation
Summary:
Creating the virt-builder images directly with their final file
name means that if we're rebuilding them for age and the build
fails, we'll lose the existing image: it seems better to keep
it than have no image at all, when this happens. It also means
that while the rebuild is in process, the file might exist but
be useless and cause any tests that happen to be running at the
time to fail. So just like the guestfs images, create the file
with a .tmp extension initially and rename it after a successful
build.

Test Plan:
Do some image builds and check they work, check that
temp file is cleaned up on failure and ctrl-c if you can...

Reviewers: jskladan, garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D858
2016-05-23 06:55:17 -07:00
Adam Williamson
731cd6e69c add FreeIPA test disks (with kickstarts) and update InstallGuide
Summary:
This goes along with the openqa_fedora commit to add the tests.
I didn't update the Docker instructions yet because I don't
quite remember how that goes. It might need a whole different
setup using some other networking...thing...

Test Plan: See https://phab.qadevel.cloud.fedoraproject.org/D831

Reviewers: jskladan, garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D832
2016-05-04 11:55:22 -07:00
Adam Williamson
055c79fc58 createhdds: use temp file correctly when uploading
Summary:
With the previous code, the temporary file wasn't closed before
we try to upload its contents to the disk image. This seems to
cause problems when run in Python 2; the file is truncated on
upload. So rejig things a bit so we use tempfile.mkstemp, close
the file after writing to it, then remove it ourselves after
doing the upload. Tested with Python 2 and Python 3 that this
creates a correct image.

If something goes wrong at the wrong time the temp file will
be left around, but hey, it's a temp file - they get cleaned
up on system shutdown. So doesn't seem like a big deal.

Test Plan:
Check building the two images that involve a file
upload in both Python 2 and Python 3, make sure the file is
complete and correct in all cases.

Reviewers: jskladan, garretraziel

Reviewed By: garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D786
2016-03-21 08:53:13 -07:00
Adam Williamson
326dd08e25 rewrite createhdds in python, make it shinier
Summary:
createhdds.sh was just too damn simple and understandable, so
I thought I'd make it three times longer, object oriented,
and hard to understand!

OK, OK, that's not great sales. Alright. The main thing was to
make it smarter. This rewrite lets it do these things:

* Only create the images that are missing (not rebuild all)
* Work out the releases to build images for
* Rename images when appropriate
* Rebuild images when they need rebuilding
* Remove old / abandoned images

It can figure out what images ought to be present - including
working out the 'next' release and figuring out from that what
releases it needs images for - and build only the missing ones.
There's a 'version' concept for images; if the existing image
is older than the version given in the data file, it'll be
rebuilt. The data file can list 'rename' pairs, allowing images
to be renamed (like when we move from a single image to multiple
label/filesystem variants). This code uses fedfind's ability
to find the current release version to figure out what releases
we need virtbuilder images for (so you don't have to pass it
in). And it can find image files that aren't in the 'currently
expected' set and wipe them. Images can also have a 'maxage',
triggering a rebuild when they exceed it - this is intended
for the virtbuilder images, so we get a rebuild with the
latest updates every so often (default is two weeks).

The point of all this is to help with unattended deployment/
maintenance, i.e. the ansible deployment we have in infra;
the idea is that we can just set that up to run the 'all'
subcommand every so often, and it'll remove old images, create
new ones, and rebuild ones that are outdated.

I kept the ability to build a single image (or a whole image
'group'), and included the ability to just run a check without
actually doing a rebuild. There's a few little weird things
and holes here as it's not really the focus of the tool.

Test Plan:
Build all images, do a full test run, and see if
it works OK. Test out all the variations of building single
images / image groups, and using the 'check' command.

Reviewers: jskladan, garretraziel

Reviewed By: jskladan, garretraziel

Subscribers: tflink

Differential Revision: https://phab.qadevel.cloud.fedoraproject.org/D687
2015-12-17 13:25:30 -08:00