Commit Graph

48 Commits

Author SHA1 Message Date
Doug Hellmann
28287942bb add lower-constraints job
Create a tox environment for running the unit tests against the lower
bounds of the dependencies.

Create a lower-constraints.txt to be used to enforce the lower bounds
in those tests.

Add openstack-tox-lower-constraints job to the zuul configuration.

See http://lists.openstack.org/pipermail/openstack-dev/2018-March/128352.html
for more details.

Change-Id: I911c66b2e9971a3e134c482a4b4ffa529d358a76
Depends-On: https://review.openstack.org/555034
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-04-06 01:23:10 -04:00
Andreas Jaeger
3c8e424150 Avoid tox_install.sh for constraints support
We do not need tox_install.sh, pip can handle constraints itself
and install the project correctly. Thus update tox.ini and remove
the now obsolete tools/tox_install.sh file.

This follows https://review.openstack.org/#/c/508061 to remove
tools/tox_install.sh.

Change-Id: Ieec1e7c7a4ae9a89fb413e4aa5288396690ed538
2017-12-02 17:07:13 +00:00
Tony Breeds
ab1ed1e59a Add Constraints support
Adding constraints support to libraries is slightly more complex than
services as the libraries themselves are listed in upper-constraints.txt
which leads to errors that you can't install a specific version and a
constrained version.

This change adds constraints support by also adding a helper script to
edit the constraints to remove diskimage-builder.

Change-Id: I795800589ea1491efd0e489a38b60e6adf33e7f3
2017-09-22 14:54:42 +10:00
Ian Wienand
d5c3863b87 Add env var to dump config graph
Make this a bit easier during debugging.  Add env var and some
developer instructions.

Change-Id: I34978ddb47d6642dfa22cae0f4c0543c0ba5475f
2017-06-08 05:04:58 +00:00
Ian Wienand
7101c52620 Add missing test requirements, fixup pylint env
Add some missing test requirements.  I noticed these because pylint
was unhappy about the imports if you look closely.

Also, pylint shouldn't be in deps as it comes from the parent's
"test-requirements.txt" install.  We don't need the VIRTUAL_ENV
setting either.

Change-Id: Ie082a058a9d3d51164448410a00d0719b0b37c4a
2017-06-05 12:22:52 +10:00
Andreas Florath
f314df12c3 Refactor: use lazy logging
As described in pep282 [1], the variable part of a log message
should be passed in via parameter.  In this case the parameters
are evaluated only when they need to be.

This patch fixes (unifies) this for DIB.

A check using pylint was added that this kind of passing parameters to
the logging subsystem is enforced in future.  As a blueprint a similar
(stripped-down) approach from cinder [2] was used.

[1] https://www.python.org/dev/peps/pep-0282/
[2] https://github.com/openstack/cinder/blob/master/tox.ini

Change-Id: I2d7bcc863e4e9583d82d204438b3c781ac99824e
Signed-off-by: Andreas Florath <andreas@florath.net>
2017-05-30 14:39:58 +10:00
Ian Wienand
543dc1baa6 Add pylint with indent check
This is an initial creation of pylint with a basic indent checker.
Small issues corrected.  Job added to gate with
Ib554a284e92583cc1d6a5c2219b3922852ca4c73

Change-Id: I7e24d8348db3aef79e1395d12692199a1f80161a
Co-Authored-By: Andreas Florath <andreas@florath.net>
2017-05-29 16:12:35 +10:00
Andreas Florath
fdd2c4b236 Semi-automatic doc generation of element dependency
Currently there is no description of dependencies in the generated
documentation of the elements: therefore a user of an element does not
know which other elements are automatically included and e.g. which
configuration options are available. In addition there are some
copy&pastes of parts of the README.rst scattered thought different
Ubuntu and Debian specific elements.

This patch adds a semi-automatic generation of dependency information
of all elements.  Nevertheless these are not automatically included.
The author of the element's README.rst can decide if and where the
dependency information should appear and can use the descriptor

.. element_deps::

for this.

This patch adds the dependency information for some Debian and
Ubuntu patches - and creates the base for later removing the
duplicated parts.

A call is added to element_dependencies._find_all_elements() to
populate reverse dependencies for Element objects.

(This is a reworking of I31d2b6050b6c46fefe37378698e9a330025db430 for
the feature/v2 branch)

Change-Id: Iebb83916fed71565071246baa550849eef40560b
2017-02-09 09:50:30 +11:00
Ian Wienand
3f8800832a Release 1.27.0
-----BEGIN PGP SIGNATURE-----
 
 iQEcBAABAgAGBQJYk8+9AAoJEBty/58O8cX8LdIH+wU/VrEVs0XYohiL6DUgabzs
 112U3UUihH5xMc/ca9Tarx+XwEvfMZkwYN2Qr0JoRJjmSt2AL6AezUhGSV+98vaY
 iQEccaFDFYlyDHm4V2r7N1xwS0B3mx87FPqVQQSUKlc3JsQxCy4o9RtD9aM8Gvqy
 +gAxMxL3p3O131K0Rvb0U5lC1FLgft9SuljCV8i5nU4/HdoryD6hedz2/ss8a9KG
 KKEdBKvPBKn73+nb8peQD/VXpej9C31r87q5VEjUsZkJ7gduY/qYLlGGgoBQqAXN
 WQ/ef1RkQKW5ba2jsjnk7fdOrA0+wYENxorR2WecuZbe2ieXw6fP3lYiD6VeWsM=
 =IUuh
 -----END PGP SIGNATURE-----

Merge tag '1.27.0' into merge-branch

Release 1.27.0

Change-Id: I9f6948636cae6d375d1d8315976504021f5a3bbb
2017-02-03 11:49:45 +11:00
Jenkins
03d1af7380 Merge "Don't run unit tests from run_functests.sh" into feature/v2 2017-02-02 22:51:04 +00:00
Ian Wienand
adf39c52cf Release 1.21.1
-----BEGIN PGP SIGNATURE-----
 
 iQEcBAABAgAGBQJYW2GoAAoJEBty/58O8cX8uSAH/15dJsglP6Zie7jSSJcR6k+e
 PJembHn9qrqrCjmJ5EwakojySaaLhwEJKvlP54OU9v7pmUXL9gJtK2OzW54LQ41g
 xBHIu0Pg4z7juyHm9+1P2Sr7Mzs1pVSEbsIYpDYUU19eghI1EAeIj3I1woKgajN7
 JlI61j3r67G6EAVtPOnmD1jvXS8CrtjiJ9wtWTH20pWfmksovg/GuXUCZrLkAAhO
 NcK35CdMii1Hkr7XOH424La/Ar+3qfUX18ZkbJY6yHzkq/ityTzzKOFjAaDl2Jg9
 WNc+SLCVYpPhPwgt7miTywamUNj3ZviA5/Hd8fuLXmtHSLQ23WOtBiaQMLtwXHs=
 =8dIl
 -----END PGP SIGNATURE-----

Merge tag '1.26.1' into merge-branch

Release 1.21.1

Change-Id: Ib9eb3dd1d384fc5b6a9846608216e056c57a173a
2017-02-02 20:36:23 +11:00
Ian Wienand
f1af76f228 Don't run unit tests from run_functests.sh
Change ec7f56c1b2 add added unit tests
in diskimage_builder/tests/functional (this is probably misnamed).

These are found and run by testr just as part of the normal "setup.py
test" run.  Don't run them as part of the functional tests (this
breaks "-h"/"-l" because it installs a virtualenv and runs tests, and
also is incorrect in the gate where it's creating a nested virtualenv
underneath the testing virtualenv).

Change-Id: I9908e080042d3026a198ba89eb653c6eff376d22
2017-02-02 15:20:24 +11:00
Ian Wienand
bfca36c772 Release 1.25.2
-----BEGIN PGP SIGNATURE-----
 
 iQEcBAABAgAGBQJYV1yqAAoJEBty/58O8cX8hLwIAKP66w6MdPN8PDgUOteui/Sx
 N0UFKJ9yR4GQOAP0NffPLjch5/g0iJLs3eFKOhtGC1LjbDjpVgjX8vW18ib8wBZK
 GemOZPF3uxg8FROrZF1vpoDy/cHgL1YV10hCnwdjN/r9rb8zOuSabqjW+Dennj2n
 fZ0SJfa8Owfudn3YxGuOymVb/wMtEloDmVGBEI1Y+h7osELCCDi3OXmwsA8qMsdl
 cTwbeugBs4PlOVbZUK/JKGuwIHKgPnDYzYu5KpXw77/MdjGT0fo5Tlq5AOBDI2sC
 9JOFEBDli4Ro05VwvI58ADMpvvOax+9EvOhLbB1dRPdZl21Iyb6gOdy2PUbFO0c=
 =aKxq
 -----END PGP SIGNATURE-----

Merge tag '1.25.2' into merge-branch

Release 1.25.2

Change-Id: I698bcf2e82117bd81649cd065a7af5cac85990c7
2017-02-02 11:20:00 +11:00
Andreas Florath
ec7f56c1b2 Refactor: block-device handling (partitioning)
During the creation of a disk image (e.g. for a VM), there is the need
to create, setup, configure and afterwards detach some kind of storage
where the newly installed OS can be copied to or directly installed
in.

This patch implements partitioning handling.

Change-Id: I0ca6a4ae3a2684d473b44e5f332ee4225ee30f8c
Signed-off-by: Andreas Florath <andreas@florath.net>
2017-01-24 19:59:10 +00:00
Ian Wienand
baf3ced0c3 Fix coverage report
There's a few things going on here

Firstly, we need to install coverage tool in test-requirements

Secondly, .testr.conf has to use PYTHON because the coverage report
works by resetting PYTHON='coverage run ...'

Thirdly, because we call ourselves diskimage-builder but the python
module is diskimage_builder that seems to confuse things.  We need to
use "setup.py test" (note, that is different to "setup.py testr"!) to
use the PBR testr wrapper.  That exposes a --coverage-package-name
argument that calls the coverage tool with the right argument.

With this I got a coverage report for our unit tests

Change-Id: I9012e18eb7d01bee035140e70afa76c47c27eb01
2017-01-18 16:14:01 +11:00
xhzhf
f6ecfb3b24 H803 hacking have been deprecated
H803 hacking have been removed.
https://github.com/openstack-dev/hacking/blob/master/setup.cfg
Closes-Bug: #1650741

Change-Id: Ic88b431d9260c6e31717b001ff657d10d76b9213
2016-12-21 22:19:59 +08:00
Gregory Haynes
54dd54d81d Switch py34 tests to py35
Py35 is now commonly available on things like -infra test nodes, lets
use that instead.

Change-Id: I554ec84d709a9937df5ec3c2fa7eab8f158cf64e
2016-12-19 23:22:24 +00:00
Gregory Haynes
5da7574aee Add get_elements; preserve backwards compat for expand_dependencies
expand_dependencies() was a public interface so we should try and
preserve backwards compat. However, since the interface is really
broken, add a new exported function "get_elements" that instack can
switch to.  This returns the canonical list of elements without
duplicates, and gives the path to each element too.

This highlighted that the unit tests were really a bit wrong.  They're
testing inner functions when we have an "API" in the get_elements()
function.  Convert all unit-tests to use this function instead.  Since
this is a library call, convert the sys.exit() calls to raised
exceptions.

Refactor the variable output into a separate function so we can do a
sanity check on it.

The added flake8 ignores are for the "over-indented for ... indent"
which happens a lot with these new longer lines.  Most other projects
ignore them.

This is an alternative proposal to
I15609389c18adf3017220fc94552514d195b323a

Change-Id: If97bcd21e45de1b5ed91023fdc441a4617051a6b
2016-12-20 08:18:59 +11:00
gengchc2
b2c3a3afdd Delete deprecated Hacking in tox.ini
Some hacking have been removed,so we can delete them.
More details:
    https://github.com/openstack-dev/hacking/blob/master/setup.cfg

Change-Id: I2ba6b00aa2a968ffb53d1061f895a593ea57a063
2016-12-04 13:04:50 +08:00
Ian Wienand
7d5afecfd9 Merge remote-tracking branch 'origin/master' into merge-branch
Change-Id: Ibab1bb95521292ae818bd91f7073c3749a2cc0cb
2016-11-18 13:53:56 +11:00
nizam
3b855faf5f Don't include openstack/common in flake8 exclude list
The directory openstack/common was used to keep codes from
oslo-incubator, we have retired oslo-incubator, so don't use
this directory any more

Change-Id: Icd13d32674b117626fbecdfdec2881463a87ad5d
2016-10-28 19:52:25 +05:30
Andreas Florath
3d48a528c1 Refactor: block-device handling (local loop)
Block device handling can be somewhat complex - especially
when taking things like md, lvm or encryption into account.

This patch factors out the creation and deletion of the local
loop image device handling into a python library.

The main propose of this patch is to implement the needed
infrastructure.  Based on this, more advanced functions can be added.
Example: (advanced) partitioning, LVM, handling different boot
scenarios (BIOS, UEFI, ...), possibility of handling multiple images
(local loop image, iSCSI, physical hard disk, ...), handling of
different filesystems for different partitions / LVs.

Change-Id: Ib626b36a00f8a5dc3dbde8df3e2619a2438eaaf1
Signed-off-by: Andreas Florath <andreas@florath.net>
2016-09-08 04:31:01 +00:00
Ian Wienand
634391185c Add releasenotes
Use reno to start at keeping release notes.  Add an initial log.

Change-Id: Iba3ebd3b01c15030ac2585dda82e43657e511310
2016-04-21 13:19:53 +10:00
janonymous
d2bc111d6a py26 is no longer supported by Infra's CI
Python 2.6 support would be dropped by
Infra team from mitaka,CI would no longer be testing it,
so projects should drop it too.

Change-Id: I916150ec5508a095fd850c9cb85dbeb464cdb826
2015-12-26 13:44:33 +05:30
Ondřej Nový
b35f3a6c4b Deprecated tox -downloadcache option removed
Caching is enabled by default from pip version 6.0

More info:
https://testrun.org/tox/latest/config.html#confval-downloadcache=path
https://pip.pypa.io/en/stable/reference/pip_install/#caching

Change-Id: If624cea7246e0a16355923af122b2f8586718d5e
2015-12-11 22:58:50 +01:00
Dmitry Tantsur
0e122e8e35 Add a tox target to run functional tests locally
Now 'tox -efunc' can be invoked to run all functional tests in
the 'venv' tox environment. Also `tox -efunc element-name` can be
used to run function tests for one element (e.g. ironic-agent).

Change-Id: Ia685d1b2a7deef2f8b98876ac09792134dd30f2f
2015-10-23 12:05:23 +02:00
THOMAS J. COCOZZELLO
d8a8f9b732 Activate pep8 check that _ is imported
Remove the specification in tox.ini that _ is a builtin so that
it will no longer assume that _ does not need to be imported.
This helps ensure that the _ from i18n is used.

Activating this check did not flag any violations.

Change-Id: Iac73937b60a6f8e7520123fbf59e748d296d4c7d
2015-09-03 15:41:31 -05:00
Ben Nemec
7f974d4b85 Reorder tox environments
If py34 is run after py2x, it results in a "db type could not be
determined" error.  The only current way to avoid that is to run
py34 first.

Change-Id: Id81e127e71ecd04a2ed16ab899d6fbf0d15bfee3
2015-03-30 20:57:34 +00:00
Gregory Haynes
4fbad49972 Add py34 to tox
We gate on py34, so we shoudl add it to our tox config.

Change-Id: I0d21cdd47841af8d05df7b36b2302f13bda3367b
2015-03-29 04:28:11 +00:00
James Polley
20b7c0b7b3 Add tox -edocs capability
This is much easier to remember than the full commandline - and keeps
this project in line with others

Change-Id: I2f631c25d8839272941b7c8d4bb571a853422083
2015-02-10 14:34:46 -08:00
Gregory Haynes
c4bbb6f3bc Create docs site containing element READMEs
We currently do not have the ability to create a docs site which
outlines all the elements.

Change-Id: I77ccf61e0c4b1509b3e7ce9b8f15ea5ccfd50d9b
2015-02-10 11:45:35 -08:00
Gregory Haynes
e5b8656141 Add new package-installs system
We currently support package-installs definitions which has some
limitations and oddities. This new format requires only one definition
which does not reside in our run-parts directories and follows a
consistent naming scheme (package-installs.yaml).

Change-Id: Ie51a7c4fdc15634ae8e069728e5e07cc1dc36095
2014-12-01 21:29:47 -08:00
Ben Nemec
f961d745e3 Add dib-lint script
This will provide a place to put checks that catch common errors
in elements.  To start, this just checks that files starting with
a shebang are chmod +x so they can actually be run.

Change-Id: I4116a8f38f7bdfc5866764354c459fad8ca18e92
2014-03-26 15:23:05 -05:00
Jeremy Stanley
606286f6e7 Remove tox locale overrides
* tox.ini: The LANG, LANGUAGE and LC_ALL environment overrides were
introduced originally during the testr migration in an attempt to be
conservative about the possibility that locale settings in the
calling environment could cause consistency problems for test runs.
In actuality, this should be unnecessary and any place where it does
cause issues ought to be considered an actual bug. Also, having
these in the configuration actively causes older pip to have
problems with non-ASCII content in some package metadata files under
Python 3, so drop it now.

Change-Id: I840f950d2e9f6e1e40af578900e92cedf8f14295
Closes-Bug: #1277495
2014-02-12 19:28:32 +00:00
JUN JIE NAN
7fd212c019 Move Babel and argparse from test-requirements.txt
to requirements.txt, to avoid disk-image-create failed without
dependencies installed.

Change-Id: I110809d298593890b2b050d6da2dcbd036db1ff3
2014-01-16 17:01:50 +08:00
James E. Blair
e6f31981f3 Add -U to pip install command in tox.ini
Ask pip to upgrade packages in the virtualenv so that a long-lived
virtualenv is still kept up to date as external dependencies change.

Change-Id: I5a57debd5c4bc03e6d00d3801be614d9b895c508
2013-12-11 09:38:14 -08:00
Sushil Kumar
31ff8235a3 Updates tox.ini to use new features
Reasons:
 - tox update v1.6

Changes:
 - tox 1.6 allows us to skip the sdist step, which is slow.
 - It also allows us to override the install line. In this case, it's
   important as it allows us to stop getting pre-release software we
   weren't asking for.

Original patch by Monty Taylor, talked about here:
http://lists.openstack.org/pipermail/openstack-dev/2013-September/015495.html

Change-Id: I7ea029bc75547ccd43566d3329037757d0388112
2013-12-09 14:24:01 +00:00
JUN JIE NAN
d6a241bd0c Using python to run testr instead
So tox will not report bash is not installed in testenv.

Fixes bug #1233024

Change-Id: Ifed717df51340b2834f2f0522125a842495dcdda
2013-10-08 15:00:24 +08:00
Monty Taylor
78ec39a711 Update from requirements
The specifically desired change is the one to setup.py which removes the
reference to d2to1. d2to1 is very undesirable to consume due to its
explicit dependency on distribute that is still unfixed. While we're
doing it, go ahead and do the full update.py from requirements.txt to
reduce later churn.

Change-Id: Ia862337b904ba021db8fc21cb21b886f948bd3fc
2013-09-28 23:16:05 -04:00
Robert Collins
bcad3737e8 Blacklist H803
H803 isn't supported by community consensus. Blacklist it to avoid
breaking folk's muscle memory.

Change-Id: I8eea9eb2562e12136018e762c8a5e23527fcf48f
2013-08-06 13:59:13 +12:00
Monty Taylor
6b74b65449 Fix hacking errors
As another step in aligning further with OpenStack practices, stop
ignoring the hacking style checks.

Change-Id: I16c9f0ca3be5790176467377303817249e7643ea
2013-07-27 12:17:02 -04:00
Monty Taylor
36a46649ed Fixed up test-requirements
As a first step to OpenStack alignment, get test-requirements.txt in
line with current OpenStack standard practices.

Fixes: Bug #1205546
Change-Id: I899c3dc167df191b84008250671c58520c436a30
2013-07-27 12:16:52 -04:00
Jenkins
a37f7ca2a4 Merge "Enable Flake8 F*** checks." 2013-06-21 13:44:53 +00:00
Jenkins
aa8afcf47d Merge "Switch from pep8 to flake8/hacking." 2013-06-21 13:33:58 +00:00
Joe Gordon
f95644470a Fix unit tests to have accurate return code.
Before 'tox -epy27' would not fail even if a unit test failed.

Change-Id: I3ed43fdd78d0d877b93b59e2eaaf5de289a17259
2013-06-20 14:21:40 -07:00
Joe Gordon
fcd1a2e7fd Enable Flake8 F*** checks.
Fix and enable F*** Checks by removing unused imports.

Change-Id: I39863a4664abff1349911546f16bfea055104e63
2013-06-19 22:42:13 -07:00
Joe Gordon
e39e3385b5 Switch from pep8 to flake8/hacking.
Use flake8/hacking, as is standard in other OpenStack projects, but
disable new checks -- they will be enabled in later patches.

Flake8 catches many useful things
http://flake8.readthedocs.org/en/2.0/warnings.html, and hacking enforces
the OpenStack Style guidelines, including things like one import per
line and alphabetical imports in order to make merging easier.

Change-Id: Ib43f7a6790bc17c0b4efdda0a5f891b5a90303c4
2013-06-19 22:32:55 -07:00
Clint Byrum
c3ee071d73 Make it possible for openstack-CI to run tests
Per http://wiki.openstack.org/ProjectTestingInterface we enable all
of the commands except build_sphinx because we do not have any sphinx
documentation as of yet. Includes babel support though there are no
properly internationalized strings just yet.

Change-Id: Iae6e6b3f9e605106f4575196fa5527d2187255df
2013-02-04 22:26:17 -08:00