sig_cloud/diskimage-builder
2
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
master
diskimage-builder/diskimage_builder/block_device/tests/test_config.py

153 lines
6.0 KiB
Python
Raw Permalink Normal View History

Adding unit testing for configuration Add a range of unit-testing for configuration parsing, graph generation and mount-point generation. Unfortunately there's some global variable hacks, and some stubs, but it's a start. Change-Id: I9e4f950c2c2ea656fc0c1a14594059fb4c62fa35
2017-05-23 01:21:02 +00:00
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import logging
Move create_graph into config.py This was suggested in a review comment in I8a5d62a076a5a50597f2f1df3a8615afba6dadb2. It works out quite nicely because the BlockDevice() driver now doesn't need to know anything about stevedore or plugins, and just works on the node list. It also simplifies the unit testing by not having to call create_graph through a BlockDevice object. Change-Id: I98512f6cf42e256d2ea8225a0b496d303bf357b8
2017-05-26 00:32:59 +00:00
from diskimage_builder.block_device.config import config_tree_to_graph
from diskimage_builder.block_device.config import create_graph
Add a more generic tree->graph parser This moves to a more generic config parser that doesn't have plugins parsing part of the tree. I understand why it ended up that way; we have "partitions" key which has special semantics compared to others keys and there was a desire to keep it isolated from core tree->graph code. But this isn't really isolated; you have to reverse-engineer several module-crossing boundaries, extras classes and repetitive recursive functions. Ultimately, plugins should have access to the node graph, but not participate in configuration parsing. This way we ensure that plugins can't invent new methods of configuration parsing. Note: unit tests produce the same tree -> graph conversion as the old method. i.e. this is not intended to have a functional change. Change-Id: I8a5d62a076a5a50597f2f1df3a8615afba6dadb2
2017-05-17 05:54:02 +00:00
from diskimage_builder.block_device.exception import \
BlockDeviceSetupException
Add state object, rename "results", add unit tests A couple of things going on, but I think it makes sense to do them atomically. The NodeBase.create() argument "results" is the global state dictionary that will be saved to "state.json", and re-loaded in later phases and passed to them as the argument "state". So for consistency, call this argument "state" (this fits with the change out to start building the state dictionary earlier in the PluginBase.__init__() calls). Since the "state" is a pretty important part of how everything works, move it into a separate object. This is treated as essentially a singleton. It bundles it nicely together for some added documentation [1]. We move instantiation of this object out of the generic BlockDevice.__init__() call and into the actual cmd_* drivers. This is because there's two distinct instantiation operations -- creating a new state (during cmd_create) and loading an existing state (other cmd_*). This is also safer -- since we know the cmd_* arguments are looking for an existing state.json, we will fail if it somehow goes missing. To more fully unit test this, some testing plugins and new entry-points are added. These add known state values which we check for. These should be a good basis for further tests. [1] as noted, we could probably do some fun things in the future like make this implement a dictionary and have some saftey features like r/o keys. Change-Id: I90eb711b3e9b1ce139eb34bdf3cde641fd06828f
2017-05-30 02:06:41 +00:00
from diskimage_builder.block_device.tests.test_base import TestBase
Adding unit testing for configuration Add a range of unit-testing for configuration parsing, graph generation and mount-point generation. Unfortunately there's some global variable hacks, and some stubs, but it's a start. Change-Id: I9e4f950c2c2ea656fc0c1a14594059fb4c62fa35
2017-05-23 01:21:02 +00:00
logger = logging.getLogger(__name__)
Add state object, rename "results", add unit tests A couple of things going on, but I think it makes sense to do them atomically. The NodeBase.create() argument "results" is the global state dictionary that will be saved to "state.json", and re-loaded in later phases and passed to them as the argument "state". So for consistency, call this argument "state" (this fits with the change out to start building the state dictionary earlier in the PluginBase.__init__() calls). Since the "state" is a pretty important part of how everything works, move it into a separate object. This is treated as essentially a singleton. It bundles it nicely together for some added documentation [1]. We move instantiation of this object out of the generic BlockDevice.__init__() call and into the actual cmd_* drivers. This is because there's two distinct instantiation operations -- creating a new state (during cmd_create) and loading an existing state (other cmd_*). This is also safer -- since we know the cmd_* arguments are looking for an existing state.json, we will fail if it somehow goes missing. To more fully unit test this, some testing plugins and new entry-points are added. These add known state values which we check for. These should be a good basis for further tests. [1] as noted, we could probably do some fun things in the future like make this implement a dictionary and have some saftey features like r/o keys. Change-Id: I90eb711b3e9b1ce139eb34bdf3cde641fd06828f
2017-05-30 02:06:41 +00:00
class TestConfig(TestBase):
Adding unit testing for configuration Add a range of unit-testing for configuration parsing, graph generation and mount-point generation. Unfortunately there's some global variable hacks, and some stubs, but it's a start. Change-Id: I9e4f950c2c2ea656fc0c1a14594059fb4c62fa35
2017-05-23 01:21:02 +00:00
"""Helper for setting up and reading a config"""
def setUp(self):
super(TestConfig, self).setUp()
Move global mount tracking into state Keep track of the mount-point ordering in a state variable, rather than a global. This path is tested by existing unit tests. Note a prior change inserted the MountNode objects directly into a list in self.state, which makes sorting quite easy as it can just implement __lt__. Unfortunately we still json dump the state, and thus we can't have aribtrary objects in it (future work may be to check keys inserted into the status object...). So we have to do a bit of wrangling with tuple lists and comparision functions here, but it's not too bad. Change-Id: I0c51e0c53c4efdb7a65ab0efe09a6780cb1affa8
2017-06-01 05:53:23 +00:00
# previously we mocked some globals here ...
Adding unit testing for configuration Add a range of unit-testing for configuration parsing, graph generation and mount-point generation. Unfortunately there's some global variable hacks, and some stubs, but it's a start. Change-Id: I9e4f950c2c2ea656fc0c1a14594059fb4c62fa35
2017-05-23 01:21:02 +00:00
class TestGraphGeneration(TestConfig):
"""Extra helper class for testing graph generation"""
def setUp(self):
super(TestGraphGeneration, self).setUp()
self.fake_default_config = {
'build-dir': '/fake',
'image-size': '1000',
'image-dir': '/fake',
'mount-base': '/fake',
}
Add a more generic tree->graph parser This moves to a more generic config parser that doesn't have plugins parsing part of the tree. I understand why it ended up that way; we have "partitions" key which has special semantics compared to others keys and there was a desire to keep it isolated from core tree->graph code. But this isn't really isolated; you have to reverse-engineer several module-crossing boundaries, extras classes and repetitive recursive functions. Ultimately, plugins should have access to the node graph, but not participate in configuration parsing. This way we ensure that plugins can't invent new methods of configuration parsing. Note: unit tests produce the same tree -> graph conversion as the old method. i.e. this is not intended to have a functional change. Change-Id: I8a5d62a076a5a50597f2f1df3a8615afba6dadb2
2017-05-17 05:54:02 +00:00
class TestConfigParsing(TestConfig):
Adding unit testing for configuration Add a range of unit-testing for configuration parsing, graph generation and mount-point generation. Unfortunately there's some global variable hacks, and some stubs, but it's a start. Change-Id: I9e4f950c2c2ea656fc0c1a14594059fb4c62fa35
2017-05-23 01:21:02 +00:00
"""Test parsing config file into a graph"""
Use networkx for digraph This switches the code to use networkx for the digraph implementation. Note that the old implementation specifically isn't removed in this change -- for review clarity. It will be replaced by a base class that defines things properly to the API described below. Plugins return a node object with three functions get_name() : return the unique name of this node get_nodes() : return a list of nodes for insertion into the graph. Usually this is just "self". Some special things like partitioning add extra nodes at this point, however. get_edges() : return a tuple of two lists; edges_from and edges_to As you would expect the first is a list of node names that points to us, and the second is a list of node names we point to. Usually this is only populated as ([self.base],[]) -- i.e. our "base" node points to us. Some plugins, such as mounting, create links both to and from themselves, however. Plugins have been updated, some test cases added (error cases specifically) Change-Id: Ic5a61365ef0132476b11bdbf1dd96885e91c3cb6
2017-05-23 23:57:32 +00:00
# test an entry in the config not being a valid plugin
Adding unit testing for configuration Add a range of unit-testing for configuration parsing, graph generation and mount-point generation. Unfortunately there's some global variable hacks, and some stubs, but it's a start. Change-Id: I9e4f950c2c2ea656fc0c1a14594059fb4c62fa35
2017-05-23 01:21:02 +00:00
def test_config_bad_plugin(self):
Add a more generic tree->graph parser This moves to a more generic config parser that doesn't have plugins parsing part of the tree. I understand why it ended up that way; we have "partitions" key which has special semantics compared to others keys and there was a desire to keep it isolated from core tree->graph code. But this isn't really isolated; you have to reverse-engineer several module-crossing boundaries, extras classes and repetitive recursive functions. Ultimately, plugins should have access to the node graph, but not participate in configuration parsing. This way we ensure that plugins can't invent new methods of configuration parsing. Note: unit tests produce the same tree -> graph conversion as the old method. i.e. this is not intended to have a functional change. Change-Id: I8a5d62a076a5a50597f2f1df3a8615afba6dadb2
2017-05-17 05:54:02 +00:00
config = self.load_config_file('bad_plugin.yaml')
self.assertRaises(BlockDeviceSetupException,
config_tree_to_graph,
config)
Adding unit testing for configuration Add a range of unit-testing for configuration parsing, graph generation and mount-point generation. Unfortunately there's some global variable hacks, and some stubs, but it's a start. Change-Id: I9e4f950c2c2ea656fc0c1a14594059fb4c62fa35
2017-05-23 01:21:02 +00:00
Use networkx for digraph This switches the code to use networkx for the digraph implementation. Note that the old implementation specifically isn't removed in this change -- for review clarity. It will be replaced by a base class that defines things properly to the API described below. Plugins return a node object with three functions get_name() : return the unique name of this node get_nodes() : return a list of nodes for insertion into the graph. Usually this is just "self". Some special things like partitioning add extra nodes at this point, however. get_edges() : return a tuple of two lists; edges_from and edges_to As you would expect the first is a list of node names that points to us, and the second is a list of node names we point to. Usually this is only populated as ([self.base],[]) -- i.e. our "base" node points to us. Some plugins, such as mounting, create links both to and from themselves, however. Plugins have been updated, some test cases added (error cases specifically) Change-Id: Ic5a61365ef0132476b11bdbf1dd96885e91c3cb6
2017-05-23 23:57:32 +00:00
# test a config that has multiple keys for a top-level entry
def test_config_multikey_node(self):
config = self.load_config_file('multi_key_node.yaml')
Replace assertRaisesRegexp with assertRaisesRegex assertRaisesRegexp was renamed to assertRaisesRegex in Py3.2 For more details, please check: https://docs.python.org/3/library/ unittest.html#unittest.TestCase.assertRaisesRegex Change-Id: I705c958c0dbf1daa409ed29ccbc038426298c306 Closes-Bug: #1436957
2017-06-03 05:57:24 +00:00
self.assertRaisesRegex(BlockDeviceSetupException,
"Config entry top-level should be a single "
"dict:",
config_tree_to_graph,
config)
Use networkx for digraph This switches the code to use networkx for the digraph implementation. Note that the old implementation specifically isn't removed in this change -- for review clarity. It will be replaced by a base class that defines things properly to the API described below. Plugins return a node object with three functions get_name() : return the unique name of this node get_nodes() : return a list of nodes for insertion into the graph. Usually this is just "self". Some special things like partitioning add extra nodes at this point, however. get_edges() : return a tuple of two lists; edges_from and edges_to As you would expect the first is a list of node names that points to us, and the second is a list of node names we point to. Usually this is only populated as ([self.base],[]) -- i.e. our "base" node points to us. Some plugins, such as mounting, create links both to and from themselves, however. Plugins have been updated, some test cases added (error cases specifically) Change-Id: Ic5a61365ef0132476b11bdbf1dd96885e91c3cb6
2017-05-23 23:57:32 +00:00
Adding unit testing for configuration Add a range of unit-testing for configuration parsing, graph generation and mount-point generation. Unfortunately there's some global variable hacks, and some stubs, but it's a start. Change-Id: I9e4f950c2c2ea656fc0c1a14594059fb4c62fa35
2017-05-23 01:21:02 +00:00
# a graph should remain the same
def test_graph(self):
graph = self.load_config_file('simple_graph.yaml')
Add a more generic tree->graph parser This moves to a more generic config parser that doesn't have plugins parsing part of the tree. I understand why it ended up that way; we have "partitions" key which has special semantics compared to others keys and there was a desire to keep it isolated from core tree->graph code. But this isn't really isolated; you have to reverse-engineer several module-crossing boundaries, extras classes and repetitive recursive functions. Ultimately, plugins should have access to the node graph, but not participate in configuration parsing. This way we ensure that plugins can't invent new methods of configuration parsing. Note: unit tests produce the same tree -> graph conversion as the old method. i.e. this is not intended to have a functional change. Change-Id: I8a5d62a076a5a50597f2f1df3a8615afba6dadb2
2017-05-17 05:54:02 +00:00
parsed_graph = config_tree_to_graph(graph)
Switch from unittest2 compat methods to Python 3.x methods With the removal of Python 2.x we can remove the unittest2 compat wrappers and switch to assertCountEqual instead of assertItemsEqual We have been able to use them since then, because testtools required unittest2, which still included it. With testtools removing Python 2.7 support [3][4], we will lose support for assertItemsEqual, so we should switch to use assertCountEqual. [1] - https://bugs.python.org/issue17866 [2] - https://hg.python.org/cpython/rev/d9921cb6e3cd [3] - testing-cabal/testtools#286 [4] - testing-cabal/testtools#277 Change-Id: I870286a2557e41099597c22dc9747743e1077615
2020-07-07 03:11:28 +00:00
self.assertCountEqual(parsed_graph, graph)
Adding unit testing for configuration Add a range of unit-testing for configuration parsing, graph generation and mount-point generation. Unfortunately there's some global variable hacks, and some stubs, but it's a start. Change-Id: I9e4f950c2c2ea656fc0c1a14594059fb4c62fa35
2017-05-23 01:21:02 +00:00
# equivalence of simple tree to graph
def test_simple_tree(self):
tree = self.load_config_file('simple_tree.yaml')
graph = self.load_config_file('simple_graph.yaml')
Add a more generic tree->graph parser This moves to a more generic config parser that doesn't have plugins parsing part of the tree. I understand why it ended up that way; we have "partitions" key which has special semantics compared to others keys and there was a desire to keep it isolated from core tree->graph code. But this isn't really isolated; you have to reverse-engineer several module-crossing boundaries, extras classes and repetitive recursive functions. Ultimately, plugins should have access to the node graph, but not participate in configuration parsing. This way we ensure that plugins can't invent new methods of configuration parsing. Note: unit tests produce the same tree -> graph conversion as the old method. i.e. this is not intended to have a functional change. Change-Id: I8a5d62a076a5a50597f2f1df3a8615afba6dadb2
2017-05-17 05:54:02 +00:00
parsed_graph = config_tree_to_graph(tree)
Switch from unittest2 compat methods to Python 3.x methods With the removal of Python 2.x we can remove the unittest2 compat wrappers and switch to assertCountEqual instead of assertItemsEqual We have been able to use them since then, because testtools required unittest2, which still included it. With testtools removing Python 2.7 support [3][4], we will lose support for assertItemsEqual, so we should switch to use assertCountEqual. [1] - https://bugs.python.org/issue17866 [2] - https://hg.python.org/cpython/rev/d9921cb6e3cd [3] - testing-cabal/testtools#286 [4] - testing-cabal/testtools#277 Change-Id: I870286a2557e41099597c22dc9747743e1077615
2020-07-07 03:11:28 +00:00
self.assertCountEqual(parsed_graph, graph)
Adding unit testing for configuration Add a range of unit-testing for configuration parsing, graph generation and mount-point generation. Unfortunately there's some global variable hacks, and some stubs, but it's a start. Change-Id: I9e4f950c2c2ea656fc0c1a14594059fb4c62fa35
2017-05-23 01:21:02 +00:00
# equivalence of a deeper tree to graph
def test_deep_tree(self):
tree = self.load_config_file('deep_tree.yaml')
graph = self.load_config_file('deep_graph.yaml')
Add a more generic tree->graph parser This moves to a more generic config parser that doesn't have plugins parsing part of the tree. I understand why it ended up that way; we have "partitions" key which has special semantics compared to others keys and there was a desire to keep it isolated from core tree->graph code. But this isn't really isolated; you have to reverse-engineer several module-crossing boundaries, extras classes and repetitive recursive functions. Ultimately, plugins should have access to the node graph, but not participate in configuration parsing. This way we ensure that plugins can't invent new methods of configuration parsing. Note: unit tests produce the same tree -> graph conversion as the old method. i.e. this is not intended to have a functional change. Change-Id: I8a5d62a076a5a50597f2f1df3a8615afba6dadb2
2017-05-17 05:54:02 +00:00
parsed_graph = config_tree_to_graph(tree)
Switch from unittest2 compat methods to Python 3.x methods With the removal of Python 2.x we can remove the unittest2 compat wrappers and switch to assertCountEqual instead of assertItemsEqual We have been able to use them since then, because testtools required unittest2, which still included it. With testtools removing Python 2.7 support [3][4], we will lose support for assertItemsEqual, so we should switch to use assertCountEqual. [1] - https://bugs.python.org/issue17866 [2] - https://hg.python.org/cpython/rev/d9921cb6e3cd [3] - testing-cabal/testtools#286 [4] - testing-cabal/testtools#277 Change-Id: I870286a2557e41099597c22dc9747743e1077615
2020-07-07 03:11:28 +00:00
self.assertCountEqual(parsed_graph, graph)
Adding unit testing for configuration Add a range of unit-testing for configuration parsing, graph generation and mount-point generation. Unfortunately there's some global variable hacks, and some stubs, but it's a start. Change-Id: I9e4f950c2c2ea656fc0c1a14594059fb4c62fa35
2017-05-23 01:21:02 +00:00
# equivalence of a complicated multi-partition tree to graph
def test_multipart_tree(self):
tree = self.load_config_file('multiple_partitions_tree.yaml')
graph = self.load_config_file('multiple_partitions_graph.yaml')
Add a more generic tree->graph parser This moves to a more generic config parser that doesn't have plugins parsing part of the tree. I understand why it ended up that way; we have "partitions" key which has special semantics compared to others keys and there was a desire to keep it isolated from core tree->graph code. But this isn't really isolated; you have to reverse-engineer several module-crossing boundaries, extras classes and repetitive recursive functions. Ultimately, plugins should have access to the node graph, but not participate in configuration parsing. This way we ensure that plugins can't invent new methods of configuration parsing. Note: unit tests produce the same tree -> graph conversion as the old method. i.e. this is not intended to have a functional change. Change-Id: I8a5d62a076a5a50597f2f1df3a8615afba6dadb2
2017-05-17 05:54:02 +00:00
parsed_graph = config_tree_to_graph(tree)
Adding unit testing for configuration Add a range of unit-testing for configuration parsing, graph generation and mount-point generation. Unfortunately there's some global variable hacks, and some stubs, but it's a start. Change-Id: I9e4f950c2c2ea656fc0c1a14594059fb4c62fa35
2017-05-23 01:21:02 +00:00
logger.debug(parsed_graph)
Switch from unittest2 compat methods to Python 3.x methods With the removal of Python 2.x we can remove the unittest2 compat wrappers and switch to assertCountEqual instead of assertItemsEqual We have been able to use them since then, because testtools required unittest2, which still included it. With testtools removing Python 2.7 support [3][4], we will lose support for assertItemsEqual, so we should switch to use assertCountEqual. [1] - https://bugs.python.org/issue17866 [2] - https://hg.python.org/cpython/rev/d9921cb6e3cd [3] - testing-cabal/testtools#286 [4] - testing-cabal/testtools#277 Change-Id: I870286a2557e41099597c22dc9747743e1077615
2020-07-07 03:11:28 +00:00
self.assertCountEqual(parsed_graph, graph)
Adding unit testing for configuration Add a range of unit-testing for configuration parsing, graph generation and mount-point generation. Unfortunately there's some global variable hacks, and some stubs, but it's a start. Change-Id: I9e4f950c2c2ea656fc0c1a14594059fb4c62fa35
2017-05-23 01:21:02 +00:00
class TestCreateGraph(TestGraphGeneration):
Use networkx for digraph This switches the code to use networkx for the digraph implementation. Note that the old implementation specifically isn't removed in this change -- for review clarity. It will be replaced by a base class that defines things properly to the API described below. Plugins return a node object with three functions get_name() : return the unique name of this node get_nodes() : return a list of nodes for insertion into the graph. Usually this is just "self". Some special things like partitioning add extra nodes at this point, however. get_edges() : return a tuple of two lists; edges_from and edges_to As you would expect the first is a list of node names that points to us, and the second is a list of node names we point to. Usually this is only populated as ([self.base],[]) -- i.e. our "base" node points to us. Some plugins, such as mounting, create links both to and from themselves, however. Plugins have been updated, some test cases added (error cases specifically) Change-Id: Ic5a61365ef0132476b11bdbf1dd96885e91c3cb6
2017-05-23 23:57:32 +00:00
# Test a graph with bad edge pointing to an invalid node
def test_invalid_missing(self):
config = self.load_config_file('bad_edge_graph.yaml')
Replace assertRaisesRegexp with assertRaisesRegex assertRaisesRegexp was renamed to assertRaisesRegex in Py3.2 For more details, please check: https://docs.python.org/3/library/ unittest.html#unittest.TestCase.assertRaisesRegex Change-Id: I705c958c0dbf1daa409ed29ccbc038426298c306 Closes-Bug: #1436957
2017-06-03 05:57:24 +00:00
self.assertRaisesRegex(BlockDeviceSetupException,
"Edge not defined: this_is_not_a_node",
create_graph,
Add state to NodeBase class Making the global state reference a defined part of the node makes some parts of the block device processing easier and removes the need for other global values. The state is passed to PluginNodeBase.__init__() and expected to be passed into all nodes as they are created. NodeBase.__init__() is updated with the new paramater 'state'. The parameter is removed from the create() call as nodes can simply reference it at any point as "self.state". This is similar to 1cdc8b20373c5d582ea928cfd7334469ff36dbce, except it is based on I68840594a34af28d41d9522addcfd830bd203b97 which loads the node-list from pickled state for later cmd_* calls. Thus we only build the state *once*, at cmd_create() time as we build the node list. Change-Id: I468dbf5134947629f125504513703d6f2cdace59
2017-06-01 04:31:49 +00:00
config, self.fake_default_config, {})
Use networkx for digraph This switches the code to use networkx for the digraph implementation. Note that the old implementation specifically isn't removed in this change -- for review clarity. It will be replaced by a base class that defines things properly to the API described below. Plugins return a node object with three functions get_name() : return the unique name of this node get_nodes() : return a list of nodes for insertion into the graph. Usually this is just "self". Some special things like partitioning add extra nodes at this point, however. get_edges() : return a tuple of two lists; edges_from and edges_to As you would expect the first is a list of node names that points to us, and the second is a list of node names we point to. Usually this is only populated as ([self.base],[]) -- i.e. our "base" node points to us. Some plugins, such as mounting, create links both to and from themselves, however. Plugins have been updated, some test cases added (error cases specifically) Change-Id: Ic5a61365ef0132476b11bdbf1dd96885e91c3cb6
2017-05-23 23:57:32 +00:00
# Test a graph with bad edge pointing to an invalid node
def test_duplicate_name(self):
config = self.load_config_file('duplicate_name.yaml')
Replace assertRaisesRegexp with assertRaisesRegex assertRaisesRegexp was renamed to assertRaisesRegex in Py3.2 For more details, please check: https://docs.python.org/3/library/ unittest.html#unittest.TestCase.assertRaisesRegex Change-Id: I705c958c0dbf1daa409ed29ccbc038426298c306 Closes-Bug: #1436957
2017-06-03 05:57:24 +00:00
self.assertRaisesRegex(BlockDeviceSetupException,
"Duplicate node name: "
"this_is_a_duplicate",
create_graph,
Add state to NodeBase class Making the global state reference a defined part of the node makes some parts of the block device processing easier and removes the need for other global values. The state is passed to PluginNodeBase.__init__() and expected to be passed into all nodes as they are created. NodeBase.__init__() is updated with the new paramater 'state'. The parameter is removed from the create() call as nodes can simply reference it at any point as "self.state". This is similar to 1cdc8b20373c5d582ea928cfd7334469ff36dbce, except it is based on I68840594a34af28d41d9522addcfd830bd203b97 which loads the node-list from pickled state for later cmd_* calls. Thus we only build the state *once*, at cmd_create() time as we build the node list. Change-Id: I468dbf5134947629f125504513703d6f2cdace59
2017-06-01 04:31:49 +00:00
config, self.fake_default_config, {})
Use networkx for digraph This switches the code to use networkx for the digraph implementation. Note that the old implementation specifically isn't removed in this change -- for review clarity. It will be replaced by a base class that defines things properly to the API described below. Plugins return a node object with three functions get_name() : return the unique name of this node get_nodes() : return a list of nodes for insertion into the graph. Usually this is just "self". Some special things like partitioning add extra nodes at this point, however. get_edges() : return a tuple of two lists; edges_from and edges_to As you would expect the first is a list of node names that points to us, and the second is a list of node names we point to. Usually this is only populated as ([self.base],[]) -- i.e. our "base" node points to us. Some plugins, such as mounting, create links both to and from themselves, however. Plugins have been updated, some test cases added (error cases specifically) Change-Id: Ic5a61365ef0132476b11bdbf1dd96885e91c3cb6
2017-05-23 23:57:32 +00:00
Adding unit testing for configuration Add a range of unit-testing for configuration parsing, graph generation and mount-point generation. Unfortunately there's some global variable hacks, and some stubs, but it's a start. Change-Id: I9e4f950c2c2ea656fc0c1a14594059fb4c62fa35
2017-05-23 01:21:02 +00:00
# Test digraph generation from deep_graph config file
def test_deep_graph_generator(self):
config = self.load_config_file('deep_graph.yaml')
Add state to NodeBase class Making the global state reference a defined part of the node makes some parts of the block device processing easier and removes the need for other global values. The state is passed to PluginNodeBase.__init__() and expected to be passed into all nodes as they are created. NodeBase.__init__() is updated with the new paramater 'state'. The parameter is removed from the create() call as nodes can simply reference it at any point as "self.state". This is similar to 1cdc8b20373c5d582ea928cfd7334469ff36dbce, except it is based on I68840594a34af28d41d9522addcfd830bd203b97 which loads the node-list from pickled state for later cmd_* calls. Thus we only build the state *once*, at cmd_create() time as we build the node list. Change-Id: I468dbf5134947629f125504513703d6f2cdace59
2017-06-01 04:31:49 +00:00
graph, call_order = create_graph(config, self.fake_default_config, {})
Adding unit testing for configuration Add a range of unit-testing for configuration parsing, graph generation and mount-point generation. Unfortunately there's some global variable hacks, and some stubs, but it's a start. Change-Id: I9e4f950c2c2ea656fc0c1a14594059fb4c62fa35
2017-05-23 01:21:02 +00:00
call_order_list = [n.name for n in call_order]
# manually created from deep_graph.yaml
# Note unlike below, the sort here is stable because the graph
# doesn't have multiple paths with only one partition
call_order_names = ['image0', 'root', 'mkfs_root',
'mount_mkfs_root',
'fstab_mount_mkfs_root']
self.assertListEqual(call_order_list, call_order_names)
Trivial fix typos Change-Id: Ib86aa9938fd852610ec0a6d8d868181f87bd2f24
2017-05-31 04:16:17 +00:00
# Test multiple partition digraph generation
Adding unit testing for configuration Add a range of unit-testing for configuration parsing, graph generation and mount-point generation. Unfortunately there's some global variable hacks, and some stubs, but it's a start. Change-Id: I9e4f950c2c2ea656fc0c1a14594059fb4c62fa35
2017-05-23 01:21:02 +00:00
def test_multiple_partitions_graph_generator(self):
config = self.load_config_file('multiple_partitions_graph.yaml')
Add state to NodeBase class Making the global state reference a defined part of the node makes some parts of the block device processing easier and removes the need for other global values. The state is passed to PluginNodeBase.__init__() and expected to be passed into all nodes as they are created. NodeBase.__init__() is updated with the new paramater 'state'. The parameter is removed from the create() call as nodes can simply reference it at any point as "self.state". This is similar to 1cdc8b20373c5d582ea928cfd7334469ff36dbce, except it is based on I68840594a34af28d41d9522addcfd830bd203b97 which loads the node-list from pickled state for later cmd_* calls. Thus we only build the state *once*, at cmd_create() time as we build the node list. Change-Id: I468dbf5134947629f125504513703d6f2cdace59
2017-06-01 04:31:49 +00:00
graph, call_order = create_graph(config, self.fake_default_config, {})
Adding unit testing for configuration Add a range of unit-testing for configuration parsing, graph generation and mount-point generation. Unfortunately there's some global variable hacks, and some stubs, but it's a start. Change-Id: I9e4f950c2c2ea656fc0c1a14594059fb4c62fa35
2017-05-23 01:21:02 +00:00
call_order_list = [n.name for n in call_order]
# The sort creating call_order_list is unstable.
# We want to ensure we see the "partitions" object in
# root->var->var_log order
root_pos = call_order_list.index('root')
var_pos = call_order_list.index('var')
var_log_pos = call_order_list.index('var_log')
self.assertGreater(var_pos, root_pos)
self.assertGreater(var_log_pos, var_pos)
# Ensure mkfs happens after partition
mkfs_root_pos = call_order_list.index('mkfs_root')
self.assertLess(root_pos, mkfs_root_pos)
mkfs_var_pos = call_order_list.index('mkfs_var')
self.assertLess(var_pos, mkfs_var_pos)
mkfs_var_log_pos = call_order_list.index('mkfs_var_log')
self.assertLess(var_log_pos, mkfs_var_log_pos)
Reference in New Issue Copy Permalink