testtools.ExpectedException(exc_class, regex) expects the exact
exc_class class, wheras the
test_setup_should_throw_exception_if_mount_nfs_command_fails() test
raises a PermissionError on Python 3.3+ instead of OSError. Replace
testtools.ExpectedException() with self.assertRaisesRegex() to port the
test to Python 3, this method accepts subclasses of exc_class.
Replace also testtools.ExpectedException() with self.assertRaisesRegex()
in other tests.
Blueprint https://blueprints.launchpad.net/cinder/+spec/cinder-python3
Change-Id: If5a74529e5ac68a2118afa8ecfd86bf24307b0b7
Adds --checkonly to tools/config/check_uptodate.sh.
This allows verifying that a configuration file was
generated.
We have had a number of issues introduced which
have caused config file generation to fail since we
removed the sample config file check in the gate.
This does not add back checks to ensure that the file
has been updated, it only checks to ensure that it is
still possible to update the sample file.
Change-Id: I2e0e376fbdec5a1bb584c6739231649e1d0f5d8e
This reverts commit e67adc385ce6997249fbf70e9402264f758fc6f4.
Adding optional dependency tracking would be nice,
but Cinder is not yet in a place to do this.
- Cinder supports tox 1.6 (minversion in tox.ini)
- This functionality does not work on 1.6, requires 1.7. Tox
fails to build a test environment if using tox 1.6.
- We have decided we can't move to tox 1.7 for now.
( https://review.openstack.org/#/c/211614/ )
This reverts commit 478e8e970e54aaf3a034dc6347605c1b18291d0d.
- Need to remove oslo.vmware from test-requirements.txt now, as
well.
Closes-Bug: #1484035
Change-Id: Icff52508ff1cd67e8c17964c840d3a5cdcfbbce3
Driver-specific requirements are not hard requirements, since the
choice of drivers is up to the operator. The oslo.vmware module is
herein moved out of requirements.txt and instead listed as an optional
dependency using the extras functionality in setup.cfg.
A check is added to the vmdk driver to gracefully handle import errors
if the vmdk driver is used and oslo.vmware is not found.
Change-Id: I9b00edc38f0700304a1a164f0679a734f8701ebe
Closes-Bug: #1475739
According to the PEP8(H405), multi-line docstring summaries
should be separated by an empty line.
Change-Id: I5cd8a9064dcefc504e85946ecdf1f56f10145d35
Closes-bug: #1407162
This change adds manage existing api task in flow. The task
is used in the volume api to provide full value task flow for
manage existing process. All errors occurred during manage
flow set volume to 'error' state.
Entry creating moved from volume api to EntryCreateTask. Also
added ManageCastTask to provide manage process to scheduler.
Related-Bug: #1364550
Change-Id: I12a4311953c1c86d584b5bf2fe2888e5b5127d43
This change adds a basic bandit config for Cinder. It can be invoked
by running the tox environment for bandit;
tox -e bandit
This is intended as a starting point for using bandit with Cinder
and it should be revisited to improve the testing as more is learned
about the specific needs of the Cinder code base.
Tox is configured to only show results for high and medium severity
results.
https://wiki.openstack.org/wiki/Security/Projects/Bandit
Change-Id: I0247e0ccaed6faacacb2b8d2f8b141a8edc704af
* Replace __builtin__ with six.moves.builtins.
* Replace tempfile.mkstemp() with tempfile.NamedTemporaryFile() to get a
text file instead of a binary file.
* tox.ini: add the following tests to Python 3.4
- cinder.tests.unit.test_hitachi_hnas_backend
- cinder.tests.unit.test_hitachi_hnas_iscsi
- cinder.tests.unit.test_hitachi_hnas_nfs
Blueprint cinder-python3
Change-Id: I9a26b4e67033a443271e8f13bcaea5e122ec865a
* When parsing share configuration: replace "\040" with a space
explicitly instead of using the Python unicode_escape encoding:
see the bug #1180984.
* Encode Unicode to UTF-8 before hash it with MD5.
* tox.ini: add cinder.tests.unit.test_glusterfs to Python 3.4
Blueprint cinder-python3
Change-Id: Iea943a1f2dfb050fe43ff41411e258adb3f3b6d0
It is useful to have a maintained (in source control) list
of drivers that exist in Cinder. It could be used in docs
and unit tests to check method impl on backend drivers.
This change add tool for generate list of drivers based on
existing BaseVD class in class hierarchy of volume drivers.
Output example:
Drivers: ['cinder.volume.drivers.lvm.LVMVolumeDriver',
'cinder.volume.drivers.rbd.RBDDriver',
...]
Implements: blueprint drivers-list-generator
Change-Id: I0e10906873e659e09a6e34531a0c932495d7c399
This patch attempts to refactor Huawei volume driver in liberty.
We add a base driver to implement the basic functions.
The sub-class will inherit from the base driver according to different
storages.
The following changes were made in this refactor:
1. Abstract a base class named HuaweiBaseDriver to make Huawei driver more
universal. You can find it in the huawei_driver.py.
2. Put all static variables into the constants.py.
3. Rename rest_common.py to rest_client.py. rest_client.py stores the
relevant methods implemented for Huawei driver.
4. Migrate some public methods from rest_client.py to huawei_utils.py,
such as parse_xml_file(), _get_volume_type() and so on.
5. This refactor only involves structural adjustment and does not involve
functional changes.
Change-Id: I768889e2577a4d975397218eb31e89b42e08e04f
Implements: blueprint refactor-huawei-volume-driver
It has been over a month since the CI maintainer was communicated via
third party mailing list, and the CI is still not reporting.
Change-Id: I25e125d76beb27da10a8ac617c70db357fcef57c
UpgradeImpact: OpenvStorage driver removed
* Replace filter() with a list-comprehension using if to get a list on
Python 3.
* Get the mock module from the stdlib unittest module on Python 3.3 and
newer, or fallback to the third-party mock module.
* Replace __builtin__ with six.moves.builtins.
* tox.ini: add the following tests for Python 3.4
- cinder.tests.unit.test_api
- cinder.tests.unit.test_cmd
Blueprint cinder-python3
Change-Id: Iea516ae598e8eebfc1087663a9b3e0a00d0633d3
* Fix usage of six.reraise(): new_exc is the exception value, not the
exception type
* Replace __builtin__ with six.moves.builtins.
* test_extracting_v2_boot_properties(): set config.glance_num_retries to
fix the test on Python 3 (comparison between mock and int now raises
a TypeError on Python 3).
* TestGlanceSerializer: use dictionaries with only one key to have a
reliable output even if the hash is randomized (hash randomization is
now enabled by default in Python 3).
* tox.ini: add cinder.tests.unit.image.test_glance to Python 3.4.
Blueprint cinder-python3
Change-Id: I33cd02e1b0666d7b5999b2fdaf469dc59fff1866
* Replace urllib2 with six.moves.urllib
* On Python 3, encode string to UTF-8 to hash it using MD5
* On Python 3, decode base64 from ASCII to get Unicode
* Replace "pattern in exc" with "pattern in exc.args"
* test_initialize_connection(): fix get_active_iscsi_target_portals()
mock. Use the return_value attribute to return a dictionary, instead
of returning a single IP address.
* tox.ini: add cinder.tests.unit.test_dothill to Python 3.4
Blueprint cinder-python3
Change-Id: Ib20bca813c2352eae447c374ded75c6dafb2e18d
* Replace StandardError with ZeroDivisionError: StandardError was
removed in Python 3.
* Replace __builtin__ with six.moves.builtins.
* Open text mode with "w" mode (instead of "wb").
* tox.ini: add targets unit tests to Python 3.4
Note: cinder.tests.unit.targets.test_iet_driver still fails on Python 3,
it will be fixed in a different change.
Blueprint cinder-python3
Change-Id: Ie0f5d2dfaf2ffdeab29fe40f692a2f09fb5a7aba
* Replace map() with a list-comprehension where a list is expected
* Replace apply(fn, args) with fn(*args)
* Use str.replace() to remove "{" and "}" characters in _clean_uuid(),
instead of using str.translate()
* Use literal syntax to create a new dictionary instead of using
dict() + dict.items()
* tox.ini: add cinder.tests.unit.test_drbdmanagedrv to Python 3.4
Blueprint cinder-python3
Change-Id: I4cb3ae422381442b778de024882e75f31eded5eb
* _dict_from_object(): on dictionaries, call the items() method instead
of iteritems() method. oslo.db objects have no items() method yet, so
check the object type to decide which method should be called.
* test_db_api: fix usage of db.quota_reserve(), quotas dictionary values
must be integers (hard limit of quotas), not Quota instances of
cinder.db.sqlalchemy.models.
* tox.ini: add the following tests to Python 3.4
- cinder.tests.unit.test_db_api
- cinder.tests.unit.test_quota
Blueprint cinder-python3
Change-Id: I93f5e0f2fe34c9a6c135f34d64ec068c5696032d
* On Python 3, shlex expects Unicode, not bytes: don't encode
* Replace map() with list-comprehension where a list is expected.
* tox.ini: add cinder.tests.unit.test_hitachi_hbsd_horcm_fc to Python 3.4
Blueprint cinder-python3
Change-Id: Id0db0dabef98f7dd8331a5149e88dee0622917b3
* On Python 3, encode text to UTF-8 before encoding it to base64
* On Python 3, decode encoded base64 from ASCII to get Unicode
* Fix JSON in unit test: COUNT must be an integer, not a string
* tox.ini: add cinder.tests.unit.test_huawei_18000 to Python 3.4
Blueprint cinder-python3
Change-Id: I8a2356e7faa98bf0a1a74fc15202866f5d5499a3
* Replace httplib import with six.moves.http_client
* Replace urllib imports with six.moves.urllib, update urllib.
For example, replace urllib.quote() with urllib.parse.quote().
* test_blockbridge: try get the mock module from unittest.mock, part of
the Python stdlib since Python 3.3. The third-party mock module has a
bug on Python 3.4.
* On Python 3, base64.encodestring() expects bytes and returns bytes.
Encode credentials to UTF-8 and decode base64 from ASCII to get the
final string as Unicode on Python 3.
* test_cfg_api_auth_scheme_password: parse the connection URL to compare
it. Comparing directly the URL fails on Python 3 because URL
parameters are rendered in a random order because of the hash
randomization.
* Replace pools.values()[0] with list(pools.values())[0]. On Python 3,
dict.values() returns an iterator.
* tox.ini: add cinder.tests.unit.test_blockbridge to Python 3.4.
Blueprint cinder-python3
Change-Id: I3c6b935680b5427d6ffdc1a00cd5221475209cdd
* On Python 3, encode/decode JSON to/from UTF-8
* Use byte strings for volume content
* Replace dict.keys()[0] and dict.value()[0] with list(dict.items())[0].
Get the key and the value at once to ensure that they are consistent.
On Python 3, items() returns an iterator: create a list to use the [0]
operator.
* SwiftBackupDriver: use a bytearray instead of a string because
bytes += bytes is inefficient on Python 3, whereas str += str is
optimized on Python 2. Replace also StringIO with BytesIO.
* BytesIO has no more len attribute on Python 3: get the length of the
content instead.
* Replace buffer(bytearray(128)) with b'\0' * 128 to create a string of
128 zeroed bytes
* Replace "rw" file mode with "w+" ("rw" raises an exception
on Python 3).
* tox.ini: add the following tests to Python 3.4
- cinder.tests.unit.test_backup_swift
- cinder.tests.unit.test_backup_tsm
Blueprint cinder-python3
Change-Id: I62d7ef8041847f31b5d06a92fa2edb65c6780497
* Add cinder.utils.convert_str() function: helper to convert a string to
a native string: convert to bytes on Python 2, convert to Unicode on
Python 3. Function needed by ceph and rbd drivers.
* Replace encodeutils.safe_encode() with utils.convert_str()
* test_backup_ceph: get the size of the volume file using os.fstat() to
fix a test. Before the volume size was a mock, but comparison between
integers and mocks raise a TypeError on Python 3.
* Fix RBDDriver._update_volume_stats(): replace a/b with a//b to get an integer
on Python 2 and Python 3 (a/b always returns a float on Python 3)
* Replace map() with a list-comprehension in RBDDriver because a list is
expected. On Python 3, map() returns an iterator.
* tox.ini: add to following tests to Python 3.4
- cinder.tests.unit.test_backup_ceph
- cinder.tests.unit.test_rbd
Blueprint cinder-python3
Change-Id: I7acb0d9eed0d351798133e0c6ef55408fed925dd
* Replace (int, long) with six.integer_types: the "long" type has been
removed in Python 3
* Replace str.encode("bas64") with base64.encodestring(str), base64
text codec has been removed in Python 3. Same change for decode
(base64.decodestring)
* On Python 3, encode JSON to UTF-8
* tox.ini: add the following tests to Python 3
- cinder.tests.unit.test_backup
- cinder.tests.unit.test_backup_driver_base
Blueprint cinder-python3
Change-Id: I86e04f8fbe9a3ce8849fd141dc3ee914e73c8796
Add a py34 test environment to tox.ini running a subset of tests which
pass on Python 3.4. With this change, the py34 check job should pass and
so it will become possible to make the job voting to avoid further
Python 3 regressions.
Blueprint cinder-python3
Change-Id: If8a8bafe0ff7e8d27baa3467f1790c3cb776457e
ostestr by default uses concurrency = ncpu.
This caused all sorts of things to blow up in Cinder
when one got over 6 cpus. Since the initial merge a
patch has landed that appears to fix the issues, and
we now run reliably at 8 and even 12 cpus.
This patch removes the hard coded concurrency=6 setting
from tox.ini
Change-Id: I483e53acaf1bb71deeba20d6c81184112cbcb046
When a development environment is under a proxy, tox fails even if
environment variables of the proxy are set.
This patch fixes this problem by passing (upper case and lower case)
http_proxy, https_proxy and no_proxy from environment to tox.
See: commit 2d02ebb775ec44a14e325c56d4a901c5705fbb94
Change-Id: Ib7964fd5ea4fb8398cdb4cd94a70fe7fae49f973
This patch dumps pretty_tox.sh and gets Cinder on board
with using ostestr. We're setting a concurrency value
of 6 (instances in the gate are 6 VCPU's), and things
seem to be stable at this value.
For folks running on 'smaller' system, it's still possible
to run in parallell; > ncpu's but if there are problems you
can still specify concurrency in your tox argument. Tox will
take the last occurence of the conncurrency argument; so for
example if you specify "tox -epy27 -- --concurrency=4" the
interpretted value for concurrency in the run will be 4.
Closes-Bug: #1459852
Change-Id: Iae12b42e915da22c863781015512df4f30aa95ca
Hacking 0.10 introduces new hacking rule H238 - old style classes
are deprecated (they are converted to new style classes - inherit
from `object`). In order to avoid any unwanted side effects all
classes should be declared using new style.
See reference on the differences:
https://www.python.org/download/releases/2.2.3/descrintro/
Change-Id: Iec5620e8f9da2d051a3754391eab4e75faf644e0
Release notes: http://lists.openstack.org/pipermail/openstack-dev/2015-January/054165.html
Ignore some new rules that have more then 10 failures, fix ones that
have fewer.
Removed deleted hacking rules from tox.ini
Note: One of the main benefits of the new hacking, is flake8 enables
concurrency by default now, so it should be much faster.
Change-Id: Ie650fa3dc91434967ad05e61edd3bcc1d833bfb1
H302 PEP8 check should be enabled to make core more clean
and readable and consistent with OpenStack Hacking rules.
Change-Id: Ie189f2418d12800a46664705eacfc127e7269f45
Partial-Bug: #1407162
This commit adds the pretty_tox.sh script and the usage of
subunit-trace to the cinder unit test jobs. This also updates the
run_tests.sh script to use subunit-trace so it'll have consistent
output formatting with the tox job.
Change-Id: I8ceafb476dc488b388c7f63501ef48128d8383d2
According to the PEP8(E265) there should be at least
one space before block comment.
Change-Id: Ic51f80210becc375b30f0f4e9eeb54995775c817
Partial-Bug: #1407162
