Commit Graph

81 Commits

Author SHA1 Message Date
Yolanda Robla
6d0b9abc0f Apply setfiles on all mountpoints
With new block device definition, where content of the image
can be mounted on different partitions, is not enough with
executing setfiles on root directory. Instead of that, expose
all the mountpoints on the image, and apply setfiles on them.

Change-Id: I153f979722eaec49eab93d7cd398c5589b9bfc44
2017-05-16 07:51:48 +02:00
Andreas Florath
e4e23897a1 Refactor: block-device filesystem creation, mount and fstab
This patch finalizes the block device refactoring.  It moves the three
remaining levels (filesystem creation, mount and fstab handling) into
the new python module.

Now it is possible to use any number of disk images, any number of
partitions and used them mounted to different directories.

Notes:

 * unmount_dir : modified to only unmount the subdirs mounted by
   mount_proc_sys_dev().  dib-block-device unmounts
   $TMP_MOUNT_PATH/mnt (see I85e01f3898d3c043071de5fad82307cb091a64a9)

Change-Id: I592c0b1329409307197460cfa8fd69798013f1f8
Signed-off-by: Andreas Florath <andreas@florath.net>
Closes-Bug: #1664924
2017-05-12 13:52:02 +02:00
Ian Wienand
b9c065de28 Remove args from BlockDevice() init
The args agument was only used to find the symbol for the getval
command.  Have the command pass the symbol to find in directly.  We
can therefore remove the args paramater to the BlockDevice() creation.

Change-Id: I8e357131b70a00e4a2c4792c009f6058d1d5ae9e
2017-05-12 09:36:23 +10:00
Ian Wienand
c74ba2fe74 Move to subparsers
Move argument parsing to subparsers, rather than positional arguments.
This better reflects the tool's role as a driver and allows
sub-commands to deal with arguments in a natural way.

Change-Id: Iae8c368e0f3fe47abfddb9e0a1558bd5b3423aee
2017-05-11 21:03:33 +10:00
Ian Wienand
f76d68020b Clear __init__.py from cmd move
I accidentally dropped the clearing of this file when it moved to
cmd.py during rebase of I1919f6e865acae14ee95cd025c9c7b75ca266a9c

Change-Id: Ibe9fcde594770cb51c732cc253987308dc038083
2017-05-11 18:52:14 +10:00
Ian Wienand
ff5b30db8a Create BlockDeviceCmd object
This creates a BlockDeviceCmd object to hold the main() function.
This doesn't really do anything different right now, but sets a base
for using argparse subparsers to handle the command-line

Change-Id: I4acf95ff4d554a3b4e7e2244ab1706631b98458f
2017-05-11 15:48:47 +10:00
Ian Wienand
a06c610a8c Move YAML parsing into cmd.py; default to env
This moves the YAML parameter parsing into the command-line driver.
It makes the argument optional so it can be taken from the environment
variable directly.  The parsed YAML is passed to the BlockDevice
object.

Change-Id: I6fa5e5b7d1fccfc7cf47d6e4a1fa6e560734680d
2017-05-11 15:22:41 +10:00
Ian Wienand
a9d8e51ec2 Move dib-block-device implementation into cmd.py
Move the command-line driver components into cmd.py.  No functional
change.

Change-Id: I1919f6e865acae14ee95cd025c9c7b75ca266a9c
2017-05-11 15:22:41 +10:00
Ian Wienand
47140293b6 Move blockdevicesetupexception.py into blockdevice.py
Less is more when it comes to code :)

Change-Id: I925ed62bdc808f0e07862f6e0905e80b50fbe942
2017-05-11 15:22:41 +10:00
Jenkins
fae8484e6b Merge "block_device: reorder imports" 2017-05-11 04:53:40 +00:00
Jenkins
bd7d7b052a Merge "Remove unused val_else_none" 2017-05-11 03:24:35 +00:00
Ian Wienand
9a8184ab4c Remove unused val_else_none
This function is unused

Change-Id: I8fe3e5452b95a639618a37a02f34b5b9f63ca43a
2017-05-11 11:53:08 +10:00
Ian Wienand
2a185ec6b6 block_device: reorder imports
Reorder imports to hacking standard (stdlib, third-party, project) [1]

[1] https://docs.openstack.org/developer/hacking/#import-order-template

Change-Id: I4aa73321e1e796ef6b8b079e42f90bf5c75388fe
2017-05-11 10:38:55 +10:00
Ian Wienand
0d8c4270c0 exec_sudo: check cmd for str, log output and raise exception
To avoid any confusion, commands passed to exec_sudo() should be a
list of "str"s.  Log a message if we see unicode issues.

This also adds a debug trace of all output.  stderr is captured.

This is modified to raise CalledProcessError on failure, like
check_call().  Calls that are ok to fail will need to explicitly catch
and ignore this.

The two calls that we expect to fail are wrapped

We wish to try rolling back if one of these command raises an
exception.  Modify the create handler to initiate rollback on all
exceptions.

Change-Id: Iee4fa41ffaf243e4728bf3a5eeec5c8fa8d2dadc
2017-05-11 09:45:25 +10:00
Yolanda Robla
59a1fc6546 Add sort_mount_point method
As this method can be introduced without any dependency,
provide it on an independent change to simplify reviews.
This is a partial refactor based on
I592c0b1329409307197460cfa8fd69798013f1f8

Change-Id: Idaf3d2b3b3e23d0b9d6bc071d67b961a829ae422
Co-Authored-By: Andreas Florath <andreas@florath.net>
2017-05-05 17:29:53 +00:00
Yolanda Robla
9b75eda51a Introduce exec_sudo command
Add a new method in the block device library called
exec_sudo, so it can be reused.

This is a partial refactor of change
I592c0b1329409307197460cfa8fd69798013f1f8

Change-Id: Id621f6d029e1275a35c4fd3f19b57c8518076134
Co-Authored-By: Andreas Florath <andreas@florath.net>
2017-05-05 16:11:41 +02:00
Adam Harwell
196b44f38a Fix py3 error in block-device
This is breaking the octavia py3 gates.
Introduced by: I7aa4fe0466e44846d8fa3194575d446fe4b5b2e6

Change-Id: I3d86482a2999197a60a81d42afc5ef7a6e71e313
2017-05-04 15:55:22 -07:00
Yolanda Robla
e21935626b Refactor block-device base functions.
As part of the final steps, refactor the bits belonging
to block device and functions. This is a partial refactor
from I3600c6a3d663c697b59d91bd3fbb5e408af345e4

Change-Id: I7aa4fe0466e44846d8fa3194575d446fe4b5b2e6
Co-Authored-By: Andreas Florath <andreas@florath.net>
2017-05-04 19:54:18 +00:00
Yolanda Robla
08c36e4bf8 Add refactor of tree-like vs graph
Introducing the refactors of the block device to allow a tree-like
configuration, and start using it for the partitions level.
Based on patch I3600c6a3d663c697b59d91bd3fbb5e408af345e4

Change-Id: I58bb3c256a1dfd100d29266571c333c2d43334f7
Co-Authored-By: Andreas Florath <andreas@florath.net>
2017-05-03 05:27:43 +00:00
Yolanda Robla
943f1ccf04 Refactor block_device: isolate the getval call
Add a new getval call that allows to retrieve values
from the block device. Also isolating the block device
information into a 'blockdev' dictionary entry, to better
return it with the getval command.

This is a refactor from the original code at
I3600c6a3d663c697b59d91bd3fbb5e408af345e4.

Change-Id: I93d33669a3a0ae644eab9f9b955bb5a9470cadeb
Co-Authored-By: Andreas Florath <andreas@florath.net>
2017-05-01 12:22:52 +02:00
Andreas Florath
803d40b0c6 Refactor block_device: passing command line parameters
The original approach was to pass each and every command
line parameter to the block device.  While the block device
functionality gets extended, this is not any longer practical.

Instead of passing in all the parameters separately this patch
collects these in a YAML file that is passed in to the block device
layer.

Change-Id: I9d07593a01441b62632234468ac25a982cf1a9f0
Signed-off-by: Andreas Florath <andreas@florath.net>
2017-04-23 09:51:18 +00:00
Andreas Florath
57c9e0bb41 Use stevedore for plugin config of block device
This patch introduces stevedore plugin mechanism for use
with the block device layer.  This makes it possible that
other projects pass in their own block device plugins.

Change-Id: Id3ea56aaf75f5a20a4e1b6ac2a68adb12c56b574
Signed-off-by: Andreas Florath <andreas@florath.net>
2017-03-24 21:42:47 +00:00
Ian Wienand
a8f2eaded8 Capture output in _exec_sudo
The stdout of the script is captured, so anything coming out from
these commands needs to be captured.  Move to check_process and show
the output as part of an error log in failure case.

Change-Id: I1150375cdc479d4f19b8ddeb49a824ab16fdf831
2017-03-23 09:31:37 +11:00
Ian Wienand
b04c58eca4 blockdevice.py: python3 fixes
keys() is an iterator in python3, so we need to make it list before
finding the first element

Change-Id: Ic158c7b2901c1a34ff417d3432fcefed4760ce24
2017-03-13 19:33:07 +11:00
Yolanda Robla
96504a4de0 Use OrderedDict for partitions instead of simple dictionary
The order of the partitions is important, it needs to be preserved.
If using a simple dict, this is not happening. As a consequence,
checks like 'primary partition being first' are failing because the
dictionary sorts the partitions randomly.
Switched to OrderedDict solved the problem, as it preserves the
ordering it gets from the yaml blob.

Change-Id: Icfa9bd95ffd0203d7c3f6af95de3a6f848c2a954
2017-03-07 15:55:20 +01:00
Andreas Florath
866a06f92d Refactor: block-device partitioning cleanup
Now that the main partitioning refactor patch is merged, there is
a small relict of handling partitions still in the disk-image-create
main.

This patch moves the functionality from disk-image-create to the
block-device/partitioning module: it is mostly a rewrite of the
original bash code in python.

Change-Id: Ia73baeca74180a7bc9ea487da03ff56d6a3070ce
Signed-off-by: Andreas Florath <andreas@florath.net>
2017-03-07 18:43:09 +11:00
Jenkins
0f224d3ae6 Merge "Check return of _load_state" into feature/v2 2017-02-06 23:41:50 +00:00
Andreas Florath
df1f583903 Check return of _load_state
In error scenarios there might be no loadable state available.
This patch adds some robustness handling this scenarios
by checking the result and possible bailing out.

Change-Id: I02e2731b14bec22c22db08d60d8d15db1911a84e
Signed-off-by: Andreas Florath <andreas@florath.net>
2017-02-04 19:21:48 +00:00
Andreas Florath
a8953dd277 block-device: change top level config from dict to list
With the old configuration structure it was only possible
to use one image and one partition layout.  The new
block-device configuration uses a list at top level;
therefore it is possible to use multiple instances
of each element type.

Change-Id: I9db4327486b676887d6ce09609994116dbebfc89
Signed-off-by: Andreas Florath <andreas@florath.net>
2017-02-03 20:15:39 +00:00
Andreas Florath
ec7f56c1b2 Refactor: block-device handling (partitioning)
During the creation of a disk image (e.g. for a VM), there is the need
to create, setup, configure and afterwards detach some kind of storage
where the newly installed OS can be copied to or directly installed
in.

This patch implements partitioning handling.

Change-Id: I0ca6a4ae3a2684d473b44e5f332ee4225ee30f8c
Signed-off-by: Andreas Florath <andreas@florath.net>
2017-01-24 19:59:10 +00:00
Andreas Florath
3d48a528c1 Refactor: block-device handling (local loop)
Block device handling can be somewhat complex - especially
when taking things like md, lvm or encryption into account.

This patch factors out the creation and deletion of the local
loop image device handling into a python library.

The main propose of this patch is to implement the needed
infrastructure.  Based on this, more advanced functions can be added.
Example: (advanced) partitioning, LVM, handling different boot
scenarios (BIOS, UEFI, ...), possibility of handling multiple images
(local loop image, iSCSI, physical hard disk, ...), handling of
different filesystems for different partitions / LVs.

Change-Id: Ib626b36a00f8a5dc3dbde8df3e2619a2438eaaf1
Signed-off-by: Andreas Florath <andreas@florath.net>
2016-09-08 04:31:01 +00:00