Modify assertRaisesAndMessageMatches() to tolerate subclasses. On
Python 3, a permission error now raises an exception PermissionError
which is a subclass of OSError, whereas the test expected exactly the
OSError type.
Partial-Implements: blueprint cinder-python3
Change-Id: Ia232c27de6a67eafad30b25301531e292fe349dc
BlkioCgroup._set_limits(): sort devices before iterating on them to
have a reliable behaviour.
The devs variable is a dictionary. On Python 3, the hash function is
now randomized, so iterating on a dictionary gives items in a random
order. Use sorted() to iterate on the list of sorted devices instead.
tox.ini: add cinder.tests.unit.test_volume_throttling to Python 3.
Blueprint cinder-python3
Partial-Bug: #1348818
Change-Id: Icf7141f772397c7ac08f0f1e21ad74cb86a06351
* don't use hasttr() to check if a lazy SQLAlchemy is loaded or not: use
the obj_attr_is_set() method instead. On Python 3, hasattr() pass
through SQLAlchemy exceptions which is unexpected.
* Replace a/b with a//b to use integer division, not float division.
* Replace filter() with list-comprehension using an if.
* Replace file() with open() and open /dev/null in binary mode (not text
mode, so Unicode on Python 3).
* test_volume require "import cinder.volume.targets.tgt" on Python 3
* Use a key function to sort a list of dictionaries. Dictionaries are no
more comparable on Python 3.
* tox.ini: add the following tests to Python 3.4.
- cinder.tests.unit.keymgr.test_mock_key_mgr
- cinder.tests.unit.test_volume
Note: test_list_availability_zones_enabled_service() of test_volume
was broken, the test checked that the result of .sort() is None...
Blueprint cinder-python3
Change-Id: If1a26acc0138db9bda7fde1cb1f40093d9b3c494
* Replace func.func_name with func.__name__: the func_name attribute
was removed in Python 3, whereas the __name__ attribute exists on
Python 2 and Python 3
* Create INITIATOR_TARGET_MAP using list(set()) to get the same order
than the tested code. The exact order is not reliable, it depends
on the hash function which is now randomized by default on
Python 3. The hash function is also different between Python 2.7
and 3.4.
* tox.ini: add test_pure to Python 3.4
Blueprint cinder-python3
Change-Id: I78601278259f1d34ad6ac3458d2dd0a3aca9d77a
This replicates the run_tests.sh -8 behavior, running
flake8 only on changes made in the last commit and
working tree.
Change-Id: Iebd5746c78e840225f0860843c77f4c638877c26
As it stands, the opts.py file that is passed into
oslo-config-generator isn't being generated dynamically
and the old way of generating the cinder.conf.sample is
dependent on oslo-incubator which Cinder is trying to
move away from. oslo-config-generator works differently
than oslo-incubator so a number of changes had to be made
in order to make this switch.
This patch adds the config directory to Cinder and in it
are two files:
-generate_cinder_opts.py that will take the
results of a grep command to create the opts.py
file to be passed into oslo-config-generator.
-cinder.conf which is the new configuration for
oslo-config-generator. The file is inside the config
directory to be consistent with other projects.
Some changes were made to the generate_sample.sh file in
order to give the base directories and target directories
to the generate_cinder_opts.py program.
tox.ini was edited to remove the checkonly option because
all that needs to happen in check_uptodate.sh is a check to
ensure that the cinder.conf.sample is actually being
generated with no issues.
All options were removed from the check_uptodate.sh
because they were unnecessary given the new, more simple
way of generating the cinder.conf.sample.
setup.cfg was also edited in order to add information
oslo-config-generator needs to run.
Co-Authored By: Jay Bryant <jsbryant@us.ibm.com>
Co-Authored By: Jacob Gregor <jgregor@us.ibm.com>
Change-Id: I643dbe5675ae9280e204f691781e617266f570d5
Closes-Bug: 1473768
Closes-Bug: 1437904
Closes-Bug: 1381563
Port cinder.image_utils.check_qemu_img_version() to Python 3.
Add the following tests to Python 3.4 in tox.ini:
* cinder.tests.unit.test_image_utils
* cinder.tests.unit.test_migrations
* cinder.tests.unit.test_misc
* cinder.tests.unit.test_service
Partially implements: blueprint cinder-python3
Change-Id: I69feda94c90206adc8771fbd5ddc6aae5bb6b64b
Encode Unicode to UTF-8 for salt and authentication key when computing
the crypt hash.
Partially implements: blueprint cinder-python3
Change-Id: I19d337cf0d40d91378e1c42061bc51b6009970a2
testtools.ExpectedException(exc_class, regex) expects the exact
exc_class class, wheras the
test_setup_should_throw_exception_if_mount_nfs_command_fails() test
raises a PermissionError on Python 3.3+ instead of OSError. Replace
testtools.ExpectedException() with self.assertRaisesRegex() to port the
test to Python 3, this method accepts subclasses of exc_class.
Replace also testtools.ExpectedException() with self.assertRaisesRegex()
in other tests.
Blueprint https://blueprints.launchpad.net/cinder/+spec/cinder-python3
Change-Id: If5a74529e5ac68a2118afa8ecfd86bf24307b0b7
Adds --checkonly to tools/config/check_uptodate.sh.
This allows verifying that a configuration file was
generated.
We have had a number of issues introduced which
have caused config file generation to fail since we
removed the sample config file check in the gate.
This does not add back checks to ensure that the file
has been updated, it only checks to ensure that it is
still possible to update the sample file.
Change-Id: I2e0e376fbdec5a1bb584c6739231649e1d0f5d8e
This reverts commit e67adc385ce6997249fbf70e9402264f758fc6f4.
Adding optional dependency tracking would be nice,
but Cinder is not yet in a place to do this.
- Cinder supports tox 1.6 (minversion in tox.ini)
- This functionality does not work on 1.6, requires 1.7. Tox
fails to build a test environment if using tox 1.6.
- We have decided we can't move to tox 1.7 for now.
( https://review.openstack.org/#/c/211614/ )
This reverts commit 478e8e970e54aaf3a034dc6347605c1b18291d0d.
- Need to remove oslo.vmware from test-requirements.txt now, as
well.
Closes-Bug: #1484035
Change-Id: Icff52508ff1cd67e8c17964c840d3a5cdcfbbce3
Driver-specific requirements are not hard requirements, since the
choice of drivers is up to the operator. The oslo.vmware module is
herein moved out of requirements.txt and instead listed as an optional
dependency using the extras functionality in setup.cfg.
A check is added to the vmdk driver to gracefully handle import errors
if the vmdk driver is used and oslo.vmware is not found.
Change-Id: I9b00edc38f0700304a1a164f0679a734f8701ebe
Closes-Bug: #1475739
According to the PEP8(H405), multi-line docstring summaries
should be separated by an empty line.
Change-Id: I5cd8a9064dcefc504e85946ecdf1f56f10145d35
Closes-bug: #1407162
This change adds manage existing api task in flow. The task
is used in the volume api to provide full value task flow for
manage existing process. All errors occurred during manage
flow set volume to 'error' state.
Entry creating moved from volume api to EntryCreateTask. Also
added ManageCastTask to provide manage process to scheduler.
Related-Bug: #1364550
Change-Id: I12a4311953c1c86d584b5bf2fe2888e5b5127d43
This change adds a basic bandit config for Cinder. It can be invoked
by running the tox environment for bandit;
tox -e bandit
This is intended as a starting point for using bandit with Cinder
and it should be revisited to improve the testing as more is learned
about the specific needs of the Cinder code base.
Tox is configured to only show results for high and medium severity
results.
https://wiki.openstack.org/wiki/Security/Projects/Bandit
Change-Id: I0247e0ccaed6faacacb2b8d2f8b141a8edc704af
* Replace __builtin__ with six.moves.builtins.
* Replace tempfile.mkstemp() with tempfile.NamedTemporaryFile() to get a
text file instead of a binary file.
* tox.ini: add the following tests to Python 3.4
- cinder.tests.unit.test_hitachi_hnas_backend
- cinder.tests.unit.test_hitachi_hnas_iscsi
- cinder.tests.unit.test_hitachi_hnas_nfs
Blueprint cinder-python3
Change-Id: I9a26b4e67033a443271e8f13bcaea5e122ec865a
* When parsing share configuration: replace "\040" with a space
explicitly instead of using the Python unicode_escape encoding:
see the bug #1180984.
* Encode Unicode to UTF-8 before hash it with MD5.
* tox.ini: add cinder.tests.unit.test_glusterfs to Python 3.4
Blueprint cinder-python3
Change-Id: Iea943a1f2dfb050fe43ff41411e258adb3f3b6d0
It is useful to have a maintained (in source control) list
of drivers that exist in Cinder. It could be used in docs
and unit tests to check method impl on backend drivers.
This change add tool for generate list of drivers based on
existing BaseVD class in class hierarchy of volume drivers.
Output example:
Drivers: ['cinder.volume.drivers.lvm.LVMVolumeDriver',
'cinder.volume.drivers.rbd.RBDDriver',
...]
Implements: blueprint drivers-list-generator
Change-Id: I0e10906873e659e09a6e34531a0c932495d7c399
This patch attempts to refactor Huawei volume driver in liberty.
We add a base driver to implement the basic functions.
The sub-class will inherit from the base driver according to different
storages.
The following changes were made in this refactor:
1. Abstract a base class named HuaweiBaseDriver to make Huawei driver more
universal. You can find it in the huawei_driver.py.
2. Put all static variables into the constants.py.
3. Rename rest_common.py to rest_client.py. rest_client.py stores the
relevant methods implemented for Huawei driver.
4. Migrate some public methods from rest_client.py to huawei_utils.py,
such as parse_xml_file(), _get_volume_type() and so on.
5. This refactor only involves structural adjustment and does not involve
functional changes.
Change-Id: I768889e2577a4d975397218eb31e89b42e08e04f
Implements: blueprint refactor-huawei-volume-driver
It has been over a month since the CI maintainer was communicated via
third party mailing list, and the CI is still not reporting.
Change-Id: I25e125d76beb27da10a8ac617c70db357fcef57c
UpgradeImpact: OpenvStorage driver removed
* Replace filter() with a list-comprehension using if to get a list on
Python 3.
* Get the mock module from the stdlib unittest module on Python 3.3 and
newer, or fallback to the third-party mock module.
* Replace __builtin__ with six.moves.builtins.
* tox.ini: add the following tests for Python 3.4
- cinder.tests.unit.test_api
- cinder.tests.unit.test_cmd
Blueprint cinder-python3
Change-Id: Iea516ae598e8eebfc1087663a9b3e0a00d0633d3
* Fix usage of six.reraise(): new_exc is the exception value, not the
exception type
* Replace __builtin__ with six.moves.builtins.
* test_extracting_v2_boot_properties(): set config.glance_num_retries to
fix the test on Python 3 (comparison between mock and int now raises
a TypeError on Python 3).
* TestGlanceSerializer: use dictionaries with only one key to have a
reliable output even if the hash is randomized (hash randomization is
now enabled by default in Python 3).
* tox.ini: add cinder.tests.unit.image.test_glance to Python 3.4.
Blueprint cinder-python3
Change-Id: I33cd02e1b0666d7b5999b2fdaf469dc59fff1866
* Replace urllib2 with six.moves.urllib
* On Python 3, encode string to UTF-8 to hash it using MD5
* On Python 3, decode base64 from ASCII to get Unicode
* Replace "pattern in exc" with "pattern in exc.args"
* test_initialize_connection(): fix get_active_iscsi_target_portals()
mock. Use the return_value attribute to return a dictionary, instead
of returning a single IP address.
* tox.ini: add cinder.tests.unit.test_dothill to Python 3.4
Blueprint cinder-python3
Change-Id: Ib20bca813c2352eae447c374ded75c6dafb2e18d
* Replace StandardError with ZeroDivisionError: StandardError was
removed in Python 3.
* Replace __builtin__ with six.moves.builtins.
* Open text mode with "w" mode (instead of "wb").
* tox.ini: add targets unit tests to Python 3.4
Note: cinder.tests.unit.targets.test_iet_driver still fails on Python 3,
it will be fixed in a different change.
Blueprint cinder-python3
Change-Id: Ie0f5d2dfaf2ffdeab29fe40f692a2f09fb5a7aba
* Replace map() with a list-comprehension where a list is expected
* Replace apply(fn, args) with fn(*args)
* Use str.replace() to remove "{" and "}" characters in _clean_uuid(),
instead of using str.translate()
* Use literal syntax to create a new dictionary instead of using
dict() + dict.items()
* tox.ini: add cinder.tests.unit.test_drbdmanagedrv to Python 3.4
Blueprint cinder-python3
Change-Id: I4cb3ae422381442b778de024882e75f31eded5eb
* _dict_from_object(): on dictionaries, call the items() method instead
of iteritems() method. oslo.db objects have no items() method yet, so
check the object type to decide which method should be called.
* test_db_api: fix usage of db.quota_reserve(), quotas dictionary values
must be integers (hard limit of quotas), not Quota instances of
cinder.db.sqlalchemy.models.
* tox.ini: add the following tests to Python 3.4
- cinder.tests.unit.test_db_api
- cinder.tests.unit.test_quota
Blueprint cinder-python3
Change-Id: I93f5e0f2fe34c9a6c135f34d64ec068c5696032d
* On Python 3, shlex expects Unicode, not bytes: don't encode
* Replace map() with list-comprehension where a list is expected.
* tox.ini: add cinder.tests.unit.test_hitachi_hbsd_horcm_fc to Python 3.4
Blueprint cinder-python3
Change-Id: Id0db0dabef98f7dd8331a5149e88dee0622917b3
* On Python 3, encode text to UTF-8 before encoding it to base64
* On Python 3, decode encoded base64 from ASCII to get Unicode
* Fix JSON in unit test: COUNT must be an integer, not a string
* tox.ini: add cinder.tests.unit.test_huawei_18000 to Python 3.4
Blueprint cinder-python3
Change-Id: I8a2356e7faa98bf0a1a74fc15202866f5d5499a3
* Replace httplib import with six.moves.http_client
* Replace urllib imports with six.moves.urllib, update urllib.
For example, replace urllib.quote() with urllib.parse.quote().
* test_blockbridge: try get the mock module from unittest.mock, part of
the Python stdlib since Python 3.3. The third-party mock module has a
bug on Python 3.4.
* On Python 3, base64.encodestring() expects bytes and returns bytes.
Encode credentials to UTF-8 and decode base64 from ASCII to get the
final string as Unicode on Python 3.
* test_cfg_api_auth_scheme_password: parse the connection URL to compare
it. Comparing directly the URL fails on Python 3 because URL
parameters are rendered in a random order because of the hash
randomization.
* Replace pools.values()[0] with list(pools.values())[0]. On Python 3,
dict.values() returns an iterator.
* tox.ini: add cinder.tests.unit.test_blockbridge to Python 3.4.
Blueprint cinder-python3
Change-Id: I3c6b935680b5427d6ffdc1a00cd5221475209cdd
* On Python 3, encode/decode JSON to/from UTF-8
* Use byte strings for volume content
* Replace dict.keys()[0] and dict.value()[0] with list(dict.items())[0].
Get the key and the value at once to ensure that they are consistent.
On Python 3, items() returns an iterator: create a list to use the [0]
operator.
* SwiftBackupDriver: use a bytearray instead of a string because
bytes += bytes is inefficient on Python 3, whereas str += str is
optimized on Python 2. Replace also StringIO with BytesIO.
* BytesIO has no more len attribute on Python 3: get the length of the
content instead.
* Replace buffer(bytearray(128)) with b'\0' * 128 to create a string of
128 zeroed bytes
* Replace "rw" file mode with "w+" ("rw" raises an exception
on Python 3).
* tox.ini: add the following tests to Python 3.4
- cinder.tests.unit.test_backup_swift
- cinder.tests.unit.test_backup_tsm
Blueprint cinder-python3
Change-Id: I62d7ef8041847f31b5d06a92fa2edb65c6780497
