sig_cloud/diskimage-builder
2
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
master
diskimage-builder/diskimage_builder/block_device/tests/plugin/test_a.py

82 lines
2.7 KiB
Python
Raw Permalink Normal View History

Add state object, rename "results", add unit tests A couple of things going on, but I think it makes sense to do them atomically. The NodeBase.create() argument "results" is the global state dictionary that will be saved to "state.json", and re-loaded in later phases and passed to them as the argument "state". So for consistency, call this argument "state" (this fits with the change out to start building the state dictionary earlier in the PluginBase.__init__() calls). Since the "state" is a pretty important part of how everything works, move it into a separate object. This is treated as essentially a singleton. It bundles it nicely together for some added documentation [1]. We move instantiation of this object out of the generic BlockDevice.__init__() call and into the actual cmd_* drivers. This is because there's two distinct instantiation operations -- creating a new state (during cmd_create) and loading an existing state (other cmd_*). This is also safer -- since we know the cmd_* arguments are looking for an existing state.json, we will fail if it somehow goes missing. To more fully unit test this, some testing plugins and new entry-points are added. These add known state values which we check for. These should be a good basis for further tests. [1] as noted, we could probably do some fun things in the future like make this implement a dictionary and have some saftey features like r/o keys. Change-Id: I90eb711b3e9b1ce139eb34bdf3cde641fd06828f
2017-05-30 02:06:41 +00:00
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
# plugin test case
import logging
from diskimage_builder.block_device.plugin import NodeBase
from diskimage_builder.block_device.plugin import PluginBase
logger = logging.getLogger(__name__)
class TestANode(NodeBase):
Move rollback into NodeBase object Currently we pass a reference to a global "rollback" list to create() to keep rollback functions. Other nodes don't need to know about global rollback state, and by passing by reference we're giving them the chance to mess it up for everyone else. Add a "add_rollback()" function in NodeBase for create() calls to register rollback calls within themselves. As they hit rollback points they can add a new entry. lambda v arguments is much of a muchness -- but this is similar to the standard atexit() call so with go with that pattern. A new "rollback()" call is added that the driver will invoke on each node as it works its way backwards in case of failure. On error, nodes will have rollback() called in reverse order (which then calls registered rollbacks in reverse order). A unit test is added to test rollback behaviour Change-Id: I65214e72c7ef607dd08f750a6d32a0b10fe97ac3
2017-06-02 00:57:06 +00:00
def __init__(self, config, state, test_rollback):
Add state object, rename "results", add unit tests A couple of things going on, but I think it makes sense to do them atomically. The NodeBase.create() argument "results" is the global state dictionary that will be saved to "state.json", and re-loaded in later phases and passed to them as the argument "state". So for consistency, call this argument "state" (this fits with the change out to start building the state dictionary earlier in the PluginBase.__init__() calls). Since the "state" is a pretty important part of how everything works, move it into a separate object. This is treated as essentially a singleton. It bundles it nicely together for some added documentation [1]. We move instantiation of this object out of the generic BlockDevice.__init__() call and into the actual cmd_* drivers. This is because there's two distinct instantiation operations -- creating a new state (during cmd_create) and loading an existing state (other cmd_*). This is also safer -- since we know the cmd_* arguments are looking for an existing state.json, we will fail if it somehow goes missing. To more fully unit test this, some testing plugins and new entry-points are added. These add known state values which we check for. These should be a good basis for further tests. [1] as noted, we could probably do some fun things in the future like make this implement a dictionary and have some saftey features like r/o keys. Change-Id: I90eb711b3e9b1ce139eb34bdf3cde641fd06828f
2017-05-30 02:06:41 +00:00
logger.debug("Create test 1")
Move rollback into NodeBase object Currently we pass a reference to a global "rollback" list to create() to keep rollback functions. Other nodes don't need to know about global rollback state, and by passing by reference we're giving them the chance to mess it up for everyone else. Add a "add_rollback()" function in NodeBase for create() calls to register rollback calls within themselves. As they hit rollback points they can add a new entry. lambda v arguments is much of a muchness -- but this is similar to the standard atexit() call so with go with that pattern. A new "rollback()" call is added that the driver will invoke on each node as it works its way backwards in case of failure. On error, nodes will have rollback() called in reverse order (which then calls registered rollbacks in reverse order). A unit test is added to test rollback behaviour Change-Id: I65214e72c7ef607dd08f750a6d32a0b10fe97ac3
2017-06-02 00:57:06 +00:00
super(TestANode, self).__init__(config['name'], state)
# might be a root node, so possibly no base
if 'base' in config:
self.base = config['base']
Add state to NodeBase class Making the global state reference a defined part of the node makes some parts of the block device processing easier and removes the need for other global values. The state is passed to PluginNodeBase.__init__() and expected to be passed into all nodes as they are created. NodeBase.__init__() is updated with the new paramater 'state'. The parameter is removed from the create() call as nodes can simply reference it at any point as "self.state". This is similar to 1cdc8b20373c5d582ea928cfd7334469ff36dbce, except it is based on I68840594a34af28d41d9522addcfd830bd203b97 which loads the node-list from pickled state for later cmd_* calls. Thus we only build the state *once*, at cmd_create() time as we build the node list. Change-Id: I468dbf5134947629f125504513703d6f2cdace59
2017-06-01 04:31:49 +00:00
# put something in the state for test_b to check for
state['test_init_state'] = 'here'
Add state object, rename "results", add unit tests A couple of things going on, but I think it makes sense to do them atomically. The NodeBase.create() argument "results" is the global state dictionary that will be saved to "state.json", and re-loaded in later phases and passed to them as the argument "state". So for consistency, call this argument "state" (this fits with the change out to start building the state dictionary earlier in the PluginBase.__init__() calls). Since the "state" is a pretty important part of how everything works, move it into a separate object. This is treated as essentially a singleton. It bundles it nicely together for some added documentation [1]. We move instantiation of this object out of the generic BlockDevice.__init__() call and into the actual cmd_* drivers. This is because there's two distinct instantiation operations -- creating a new state (during cmd_create) and loading an existing state (other cmd_*). This is also safer -- since we know the cmd_* arguments are looking for an existing state.json, we will fail if it somehow goes missing. To more fully unit test this, some testing plugins and new entry-points are added. These add known state values which we check for. These should be a good basis for further tests. [1] as noted, we could probably do some fun things in the future like make this implement a dictionary and have some saftey features like r/o keys. Change-Id: I90eb711b3e9b1ce139eb34bdf3cde641fd06828f
2017-05-30 02:06:41 +00:00
Move rollback into NodeBase object Currently we pass a reference to a global "rollback" list to create() to keep rollback functions. Other nodes don't need to know about global rollback state, and by passing by reference we're giving them the chance to mess it up for everyone else. Add a "add_rollback()" function in NodeBase for create() calls to register rollback calls within themselves. As they hit rollback points they can add a new entry. lambda v arguments is much of a muchness -- but this is similar to the standard atexit() call so with go with that pattern. A new "rollback()" call is added that the driver will invoke on each node as it works its way backwards in case of failure. On error, nodes will have rollback() called in reverse order (which then calls registered rollbacks in reverse order). A unit test is added to test rollback behaviour Change-Id: I65214e72c7ef607dd08f750a6d32a0b10fe97ac3
2017-06-02 00:57:06 +00:00
# If we're doing rollback testing the config has some strings
# set for us
if test_rollback:
self.add_rollback(self.do_rollback, config['rollback_one_arg'])
self.add_rollback(self.do_rollback, config['rollback_two_arg'])
# see if we're the node who is going to fail
self.trigger_rollback = True if 'trigger_rollback' in config else False
Add state object, rename "results", add unit tests A couple of things going on, but I think it makes sense to do them atomically. The NodeBase.create() argument "results" is the global state dictionary that will be saved to "state.json", and re-loaded in later phases and passed to them as the argument "state". So for consistency, call this argument "state" (this fits with the change out to start building the state dictionary earlier in the PluginBase.__init__() calls). Since the "state" is a pretty important part of how everything works, move it into a separate object. This is treated as essentially a singleton. It bundles it nicely together for some added documentation [1]. We move instantiation of this object out of the generic BlockDevice.__init__() call and into the actual cmd_* drivers. This is because there's two distinct instantiation operations -- creating a new state (during cmd_create) and loading an existing state (other cmd_*). This is also safer -- since we know the cmd_* arguments are looking for an existing state.json, we will fail if it somehow goes missing. To more fully unit test this, some testing plugins and new entry-points are added. These add known state values which we check for. These should be a good basis for further tests. [1] as noted, we could probably do some fun things in the future like make this implement a dictionary and have some saftey features like r/o keys. Change-Id: I90eb711b3e9b1ce139eb34bdf3cde641fd06828f
2017-05-30 02:06:41 +00:00
def get_edges(self):
Move rollback into NodeBase object Currently we pass a reference to a global "rollback" list to create() to keep rollback functions. Other nodes don't need to know about global rollback state, and by passing by reference we're giving them the chance to mess it up for everyone else. Add a "add_rollback()" function in NodeBase for create() calls to register rollback calls within themselves. As they hit rollback points they can add a new entry. lambda v arguments is much of a muchness -- but this is similar to the standard atexit() call so with go with that pattern. A new "rollback()" call is added that the driver will invoke on each node as it works its way backwards in case of failure. On error, nodes will have rollback() called in reverse order (which then calls registered rollbacks in reverse order). A unit test is added to test rollback behaviour Change-Id: I65214e72c7ef607dd08f750a6d32a0b10fe97ac3
2017-06-02 00:57:06 +00:00
# may not have a base, if used as root node
to = [self.base] if hasattr(self, 'base') else []
return (to, [])
def do_rollback(self, string):
# We will check this after all rollbacks to make sure they ran
# in the right order
self.state['rollback_test'].append(string)
Add state object, rename "results", add unit tests A couple of things going on, but I think it makes sense to do them atomically. The NodeBase.create() argument "results" is the global state dictionary that will be saved to "state.json", and re-loaded in later phases and passed to them as the argument "state". So for consistency, call this argument "state" (this fits with the change out to start building the state dictionary earlier in the PluginBase.__init__() calls). Since the "state" is a pretty important part of how everything works, move it into a separate object. This is treated as essentially a singleton. It bundles it nicely together for some added documentation [1]. We move instantiation of this object out of the generic BlockDevice.__init__() call and into the actual cmd_* drivers. This is because there's two distinct instantiation operations -- creating a new state (during cmd_create) and loading an existing state (other cmd_*). This is also safer -- since we know the cmd_* arguments are looking for an existing state.json, we will fail if it somehow goes missing. To more fully unit test this, some testing plugins and new entry-points are added. These add known state values which we check for. These should be a good basis for further tests. [1] as noted, we could probably do some fun things in the future like make this implement a dictionary and have some saftey features like r/o keys. Change-Id: I90eb711b3e9b1ce139eb34bdf3cde641fd06828f
2017-05-30 02:06:41 +00:00
Move rollback into NodeBase object Currently we pass a reference to a global "rollback" list to create() to keep rollback functions. Other nodes don't need to know about global rollback state, and by passing by reference we're giving them the chance to mess it up for everyone else. Add a "add_rollback()" function in NodeBase for create() calls to register rollback calls within themselves. As they hit rollback points they can add a new entry. lambda v arguments is much of a muchness -- but this is similar to the standard atexit() call so with go with that pattern. A new "rollback()" call is added that the driver will invoke on each node as it works its way backwards in case of failure. On error, nodes will have rollback() called in reverse order (which then calls registered rollbacks in reverse order). A unit test is added to test rollback behaviour Change-Id: I65214e72c7ef607dd08f750a6d32a0b10fe97ac3
2017-06-02 00:57:06 +00:00
def create(self):
Add state object, rename "results", add unit tests A couple of things going on, but I think it makes sense to do them atomically. The NodeBase.create() argument "results" is the global state dictionary that will be saved to "state.json", and re-loaded in later phases and passed to them as the argument "state". So for consistency, call this argument "state" (this fits with the change out to start building the state dictionary earlier in the PluginBase.__init__() calls). Since the "state" is a pretty important part of how everything works, move it into a separate object. This is treated as essentially a singleton. It bundles it nicely together for some added documentation [1]. We move instantiation of this object out of the generic BlockDevice.__init__() call and into the actual cmd_* drivers. This is because there's two distinct instantiation operations -- creating a new state (during cmd_create) and loading an existing state (other cmd_*). This is also safer -- since we know the cmd_* arguments are looking for an existing state.json, we will fail if it somehow goes missing. To more fully unit test this, some testing plugins and new entry-points are added. These add known state values which we check for. These should be a good basis for further tests. [1] as noted, we could probably do some fun things in the future like make this implement a dictionary and have some saftey features like r/o keys. Change-Id: I90eb711b3e9b1ce139eb34bdf3cde641fd06828f
2017-05-30 02:06:41 +00:00
# put some fake entries into state
Add state to NodeBase class Making the global state reference a defined part of the node makes some parts of the block device processing easier and removes the need for other global values. The state is passed to PluginNodeBase.__init__() and expected to be passed into all nodes as they are created. NodeBase.__init__() is updated with the new paramater 'state'. The parameter is removed from the create() call as nodes can simply reference it at any point as "self.state". This is similar to 1cdc8b20373c5d582ea928cfd7334469ff36dbce, except it is based on I68840594a34af28d41d9522addcfd830bd203b97 which loads the node-list from pickled state for later cmd_* calls. Thus we only build the state *once*, at cmd_create() time as we build the node list. Change-Id: I468dbf5134947629f125504513703d6f2cdace59
2017-06-01 04:31:49 +00:00
self.state['test_a'] = {}
self.state['test_a']['value'] = 'foo'
self.state['test_a']['value2'] = 'bar'
Move rollback into NodeBase object Currently we pass a reference to a global "rollback" list to create() to keep rollback functions. Other nodes don't need to know about global rollback state, and by passing by reference we're giving them the chance to mess it up for everyone else. Add a "add_rollback()" function in NodeBase for create() calls to register rollback calls within themselves. As they hit rollback points they can add a new entry. lambda v arguments is much of a muchness -- but this is similar to the standard atexit() call so with go with that pattern. A new "rollback()" call is added that the driver will invoke on each node as it works its way backwards in case of failure. On error, nodes will have rollback() called in reverse order (which then calls registered rollbacks in reverse order). A unit test is added to test rollback behaviour Change-Id: I65214e72c7ef607dd08f750a6d32a0b10fe97ac3
2017-06-02 00:57:06 +00:00
if self.trigger_rollback:
# The rollback test will append the strings to this as
# it unrolls, and we'll check it's value at the end
self.state['rollback_test'] = []
raise RuntimeError("Rollback triggered")
Add state object, rename "results", add unit tests A couple of things going on, but I think it makes sense to do them atomically. The NodeBase.create() argument "results" is the global state dictionary that will be saved to "state.json", and re-loaded in later phases and passed to them as the argument "state". So for consistency, call this argument "state" (this fits with the change out to start building the state dictionary earlier in the PluginBase.__init__() calls). Since the "state" is a pretty important part of how everything works, move it into a separate object. This is treated as essentially a singleton. It bundles it nicely together for some added documentation [1]. We move instantiation of this object out of the generic BlockDevice.__init__() call and into the actual cmd_* drivers. This is because there's two distinct instantiation operations -- creating a new state (during cmd_create) and loading an existing state (other cmd_*). This is also safer -- since we know the cmd_* arguments are looking for an existing state.json, we will fail if it somehow goes missing. To more fully unit test this, some testing plugins and new entry-points are added. These add known state values which we check for. These should be a good basis for further tests. [1] as noted, we could probably do some fun things in the future like make this implement a dictionary and have some saftey features like r/o keys. Change-Id: I90eb711b3e9b1ce139eb34bdf3cde641fd06828f
2017-05-30 02:06:41 +00:00
return
Remove 'state' argument from later cmd_* calls With I468dbf5134947629f125504513703d6f2cdace59 each node has a reference to the global state object. This means it gets pickled into the node-list, which is loaded for later calls. There is no need to reload the state.json it and pass it for later cmd_* calls, as the nodes can see it via the unpickled self.state Change-Id: I9e2f8910f17599d92ee33e7df8e36d8ed4d44575
2017-06-01 04:57:34 +00:00
def umount(self):
Use picked nodes for later cmd_* calls Currently the later cmd_* calls -- umount, cleanup, delete -- all recreate the node graph by parsing the config file using create_graph() There is some need, however, to have a sense of global state when building the node list. The problem is, this is a one time operation -- we do not want to rebuild that state for these later calls (see the "loaded" checks in proposed Ic3b805f9258128d5233b21ff25579c03487c7fcc). An insight here seems to be that these cmd_* calls do not actually want to re-parse the configuration file and rebuild the node list; they just want to walk the node list in reverse with the state as provided after cmd_create(). So, rather than re-creating the node list, we might as well just pickle it, save it to disk along side the state dictionary dump and reload it for cmd_*. After this, I think we can safely have PluginBase.__init__() be passed the state. We will now know that this will only be called once, during initial creation. Change-Id: I68840594a34af28d41d9522addcfd830bd203b97
2017-05-31 05:09:37 +00:00
# Umount is run in reverse. This key should exist from test_b
Remove 'state' argument from later cmd_* calls With I468dbf5134947629f125504513703d6f2cdace59 each node has a reference to the global state object. This means it gets pickled into the node-list, which is loaded for later calls. There is no need to reload the state.json it and pass it for later cmd_* calls, as the nodes can see it via the unpickled self.state Change-Id: I9e2f8910f17599d92ee33e7df8e36d8ed4d44575
2017-06-01 04:57:34 +00:00
self.state['umount'].append('test_a')
Use picked nodes for later cmd_* calls Currently the later cmd_* calls -- umount, cleanup, delete -- all recreate the node graph by parsing the config file using create_graph() There is some need, however, to have a sense of global state when building the node list. The problem is, this is a one time operation -- we do not want to rebuild that state for these later calls (see the "loaded" checks in proposed Ic3b805f9258128d5233b21ff25579c03487c7fcc). An insight here seems to be that these cmd_* calls do not actually want to re-parse the configuration file and rebuild the node list; they just want to walk the node list in reverse with the state as provided after cmd_create(). So, rather than re-creating the node list, we might as well just pickle it, save it to disk along side the state dictionary dump and reload it for cmd_*. After this, I think we can safely have PluginBase.__init__() be passed the state. We will now know that this will only be called once, during initial creation. Change-Id: I68840594a34af28d41d9522addcfd830bd203b97
2017-05-31 05:09:37 +00:00
Add state object, rename "results", add unit tests A couple of things going on, but I think it makes sense to do them atomically. The NodeBase.create() argument "results" is the global state dictionary that will be saved to "state.json", and re-loaded in later phases and passed to them as the argument "state". So for consistency, call this argument "state" (this fits with the change out to start building the state dictionary earlier in the PluginBase.__init__() calls). Since the "state" is a pretty important part of how everything works, move it into a separate object. This is treated as essentially a singleton. It bundles it nicely together for some added documentation [1]. We move instantiation of this object out of the generic BlockDevice.__init__() call and into the actual cmd_* drivers. This is because there's two distinct instantiation operations -- creating a new state (during cmd_create) and loading an existing state (other cmd_*). This is also safer -- since we know the cmd_* arguments are looking for an existing state.json, we will fail if it somehow goes missing. To more fully unit test this, some testing plugins and new entry-points are added. These add known state values which we check for. These should be a good basis for further tests. [1] as noted, we could probably do some fun things in the future like make this implement a dictionary and have some saftey features like r/o keys. Change-Id: I90eb711b3e9b1ce139eb34bdf3cde641fd06828f
2017-05-30 02:06:41 +00:00
class TestA(PluginBase):
Add state to NodeBase class Making the global state reference a defined part of the node makes some parts of the block device processing easier and removes the need for other global values. The state is passed to PluginNodeBase.__init__() and expected to be passed into all nodes as they are created. NodeBase.__init__() is updated with the new paramater 'state'. The parameter is removed from the create() call as nodes can simply reference it at any point as "self.state". This is similar to 1cdc8b20373c5d582ea928cfd7334469ff36dbce, except it is based on I68840594a34af28d41d9522addcfd830bd203b97 which loads the node-list from pickled state for later cmd_* calls. Thus we only build the state *once*, at cmd_create() time as we build the node list. Change-Id: I468dbf5134947629f125504513703d6f2cdace59
2017-06-01 04:31:49 +00:00
def __init__(self, config, defaults, state):
Use class as super() argument Fix a few typos using the inherited class for super() Change-Id: If9f2f423f136fb78ee93018d5c299d0dae603aad
2017-06-07 23:43:47 +00:00
super(TestA, self).__init__()
Move rollback into NodeBase object Currently we pass a reference to a global "rollback" list to create() to keep rollback functions. Other nodes don't need to know about global rollback state, and by passing by reference we're giving them the chance to mess it up for everyone else. Add a "add_rollback()" function in NodeBase for create() calls to register rollback calls within themselves. As they hit rollback points they can add a new entry. lambda v arguments is much of a muchness -- but this is similar to the standard atexit() call so with go with that pattern. A new "rollback()" call is added that the driver will invoke on each node as it works its way backwards in case of failure. On error, nodes will have rollback() called in reverse order (which then calls registered rollbacks in reverse order). A unit test is added to test rollback behaviour Change-Id: I65214e72c7ef607dd08f750a6d32a0b10fe97ac3
2017-06-02 00:57:06 +00:00
test_rollback = True if 'test_rollback' in defaults else False
self.node = TestANode(config, state, test_rollback)
Add state object, rename "results", add unit tests A couple of things going on, but I think it makes sense to do them atomically. The NodeBase.create() argument "results" is the global state dictionary that will be saved to "state.json", and re-loaded in later phases and passed to them as the argument "state". So for consistency, call this argument "state" (this fits with the change out to start building the state dictionary earlier in the PluginBase.__init__() calls). Since the "state" is a pretty important part of how everything works, move it into a separate object. This is treated as essentially a singleton. It bundles it nicely together for some added documentation [1]. We move instantiation of this object out of the generic BlockDevice.__init__() call and into the actual cmd_* drivers. This is because there's two distinct instantiation operations -- creating a new state (during cmd_create) and loading an existing state (other cmd_*). This is also safer -- since we know the cmd_* arguments are looking for an existing state.json, we will fail if it somehow goes missing. To more fully unit test this, some testing plugins and new entry-points are added. These add known state values which we check for. These should be a good basis for further tests. [1] as noted, we could probably do some fun things in the future like make this implement a dictionary and have some saftey features like r/o keys. Change-Id: I90eb711b3e9b1ce139eb34bdf3cde641fd06828f
2017-05-30 02:06:41 +00:00
def get_nodes(self):
return [self.node]
Reference in New Issue Copy Permalink