When importing backup metadata only required fields were being imported,
and all other fields were being ignored.
This led to some serious problems when importing backups back, since
you would lose not only the original Volume ID, but also relevant parent
id information needed to restore incremental backups.
This patch fixes this by importing everything back and creating backup
records with the right ID.
Closes-Bug: #1455043
Closes-Bug: #1476416
Depends-On: Id7ab6e174c1fe85772477f03059c4f457c5c8b17
Change-Id: Ia42ba73e9078d993c63f8e16308151ad11721ea9
Add versionedobjects abstraction layer to services. Distinguish time
zone aware DateTime fields. The object derives from
CinderObjectDictCompat, so it supports both object (obj.prop) and dict
(obj['prop']) syntax to access properties. Complete move to object
notation will be made in a follow up clean up patch.
Co-Authored-By: Michal Dulko <michal.dulko@intel.com>
Change-Id: I09f593f9f9aa8befa40d989b731159b78a429071
Partial-Implements: blueprint cinder-objects
Adds --checkonly to tools/config/check_uptodate.sh.
This allows verifying that a configuration file was
generated.
We have had a number of issues introduced which
have caused config file generation to fail since we
removed the sample config file check in the gate.
This does not add back checks to ensure that the file
has been updated, it only checks to ensure that it is
still possible to update the sample file.
Change-Id: I2e0e376fbdec5a1bb584c6739231649e1d0f5d8e
This change adds a basic bandit config for Cinder. It can be invoked
by running the tox environment for bandit;
tox -e bandit
This is intended as a starting point for using bandit with Cinder
and it should be revisited to improve the testing as more is learned
about the specific needs of the Cinder code base.
Tox is configured to only show results for high and medium severity
results.
https://wiki.openstack.org/wiki/Security/Projects/Bandit
Change-Id: I0247e0ccaed6faacacb2b8d2f8b141a8edc704af
It is useful to have a maintained (in source control) list
of drivers that exist in Cinder. It could be used in docs
and unit tests to check method impl on backend drivers.
This change add tool for generate list of drivers based on
existing BaseVD class in class hierarchy of volume drivers.
Output example:
Drivers: ['cinder.volume.drivers.lvm.LVMVolumeDriver',
'cinder.volume.drivers.rbd.RBDDriver',
...]
Implements: blueprint drivers-list-generator
Change-Id: I0e10906873e659e09a6e34531a0c932495d7c399
SQLite doesn't support 'drop constraint' statament. So it notifies:
SAWarning: Table 'encryption' specifies columns 'volume_type_id'
as primary_key=True, not matching locally specified columns
'encryption_id'; setting the current primary key columns to
'encryption_id'. This warning may become an exception in a future
release.
Add new strategy for sqlite upgrade: table is renamed to temporary
name, new table is created followed by INSERT statements, and
renamed to original one.
Change-Id: I723fdadb946f6944dde03954409899a9ddc3ee0e
Closes-Bug: #1431374
Use oslo.service.service and periodic_task entry points to generate
a sample configuration file. oslo.service library doesn't have
an oslo.service entry point.
Change-Id: Ib3c320d73494258e2ef973a62a54b6afb6b95988
Closes-Bug: 1468955
oslo.service has graduated, so cinder should consume it.
Change-Id: I80487f559302b0e88953835ffb4a3bfe2a347501
Closes-Bug: #1466851
Depends-On: I305cf53bad6213c151395e93d656b53a8a28e1db
Noticed today while working on fixing issues
with the config generator that oslo_policy
had been added in the wrong location for
pulling the config options in to the config
generator.
This fix moves the items that had been put in
oslo.config.generator.rc but were missing in generate_sample.sh
over to the appropriate location. I also removed the option for
importing additional libraries in oslo.config.generator.rc
given that it will not work.
I am still working on a longer term solution, but
we really should get this fixed so we can generate sample
config files again.
Change-Id: I2c14c1acc49d68305dca5a3dedc686ab07ed2088
This patch adds VersionedObjects abstraction layer to volume backups.
The object derives from CinderObjectDictCompat, so it supports both
object (obj.prop) and dict (obj['prop']) syntax to access properties.
Complete move to object notation will be made in a follow up clean up
patch.
Co-Authored-By: Grzegorz Grasza <grzegorz.grasza@intel.com>
Change-Id: Icff37261b367463b71a1268be16f9c97f595bf0c
Partial-Implements: blueprint cinder-objects
Use graduated oslo.policy instead of sync with oslo-incubator code.
Co-Authored-By: Ivan Kolodyazhny <e0ne@e0ne.info>
Change-Id: Ibb295830ac6abde9317c2549f75c1780ac454125
Closes-Bug: #1421863
Partial-Bug: #1458945
The following patch switches direct db calls in
volume/api.py get_all_snapshots to use SnapshotList
instead.
Partial-Implements: blueprint cinder-objects
Change-Id: Ifdccc81a087f9797fcfbf098d3ecea3875ad7bd9
This patch dumps pretty_tox.sh and gets Cinder on board
with using ostestr. We're setting a concurrency value
of 6 (instances in the gate are 6 VCPU's), and things
seem to be stable at this value.
For folks running on 'smaller' system, it's still possible
to run in parallell; > ncpu's but if there are problems you
can still specify concurrency in your tox argument. Tox will
take the last occurence of the conncurrency argument; so for
example if you specify "tox -epy27 -- --concurrency=4" the
interpretted value for concurrency in the run will be 4.
Closes-Bug: #1459852
Change-Id: Iae12b42e915da22c863781015512df4f30aa95ca
Replace list(dict.iterkeys()) with list(dict). The call to iterkeys() is
useless and the iterkeys() method of dictionaries was removed in
Python 3
Change-Id: I16733643d4b92fe8134eec2549bbd679fe75fb9c
We just moved to using a hybrid approach that
uses the new oslo-config-generator for pulling out
library options to create the cinder.sample.conf file.
In that patch it was expected that we should use oslo_*
for the namespace. That, however, is not how
oslo-config-generator works.
This patch switches to using oslo.* for the namespace to get
the library options pulled in. This also fixes the fact that
we were putting multiple '[DEFAULT]' sections into the file.
Change-Id: Ie799d4d7e6deb2c486a93c5095fc3b766679fa95
Partial-bug: 1449235
After moving to oslo.config we still were using
incubator config generator. This was ok, but the
problem is we haven't been pulling config options
from the oslo libs.
This is a hack that just appends external lib calls
and appends those options to the sample file being built.
Change-Id: I2634b20ef4abd3bf7990f845d59ad3d208db234f
Closes-Bug: #1447380
Abstract volumes and volume snapshots into objects.
Get, create, and delete snapshot APIs were changed to
use new snapshot objects. A skeleton volume object was
created, but cinder internals were not changed to use
the volume object, although volume is referenced and
used by the snapshot object. The internals will be
changed to use volume object in a subsequent patch.
Change-Id: I387018e80c8539565e99454db65d976030002c0f
Implements: blueprint cinder-objects
This patch is to insert the objects code base, currently based
on nova.objects. It will be later based on
oslo_versionedobjects once it is available. This is done
to make progress on changing cinder internals to use objects,
since nova.objects and oslo.versionedobjects are similar.
Inserted objects code base into cinder and modified cinder services
to use object-aware serializer.
Change-Id: I42a6cc43da19ca29db4cfb33cbf794afed18f758
Implements: blueprint cinder-objects
This commit adds the pretty_tox.sh script and the usage of
subunit-trace to the cinder unit test jobs. This also updates the
run_tests.sh script to use subunit-trace so it'll have consistent
output formatting with the tox job.
Change-Id: I8ceafb476dc488b388c7f63501ef48128d8383d2
We used extra library oslo.concurrency in Cinder, and oslo.concurrency
provides configuration options, we need include these options in sample
config file. This commit handles that.
Change-Id: I534539b7e87a3f5dc36722395cbe241a10b2d75e
Change oslo.db to oslo_db
Change oslo.config to olso_config
Change oslo.rootwrap to oslo_rootwrap
The oslo libraries are moving away from namespace packages.
Partial-bug: 1409733
Change-Id: I3429282afa2db4c0b3aa630eb3a0eaece04405ed
The install_venv_common modules hasn't had a sync done
since early in the icehouse release. This sync brings
Cinder's version up to date with the latest code.
Current HEAD in OSLO:
---------------------
commit 36b0e8570b449129d6d474c03b02ceb62edb78df
Date: Thu Dec 11 11:27:08 2014 +0100
We shouldn't replace `oslo-incubator` in comments
Change being merged with this patch:
---------------------
fe3389e5 - Improve help strings
Change-Id: I66fd66650a7106b8d20e4fc483dabb91184922f9
The following pylint error was found:
cinder/backup/drivers/swift.py:486: [E1101, SwiftBackupDriver.backup]
Module 'hashlib' has no 'sha256' member
Affected patch:
https://review.openstack.org/#/c/110068/
A similar pylint error "Module 'hashlib' has no 'sha224' member"
was also generated.
However, both 'sha256' and 'sha224' are functions in 'hashlib'. So
these errors should be ignored.
Change-Id: I84690813b6c6a15e661e27acdf97149996951978
auth_token middleware in python-keystoneclient is deprecated and has
been moved to the keystonemiddleware repo.
Closes-Bug: #1342274
Change-Id: I1aadbe24db63eb2507b088cd53886d7f2e192cab
oslo.db first stable release has been cut and we can start using it
instead of openstack/common/db/* code which is now marked obsolete.
Change-Id: Ib350e8d1dcec3be707e3b1679c70202619256269
Cinder's generate sample script has fallen a
bit out of date and was no longer working properly
in local venv.
This patch updates it with a working version that
reads the rc file and makes some other updates.
Even better, we'll update the tox.ini file here so
that it actually works, and finally add some more
info to the error message when update check fails
to make is obvious how to run these appropriately.
Change-Id: I2389910d8ece6f737a9609ce4aa05263d9293e14
To keep Python 3.x compatibility, use six.moves.urllib.parse to
replace urlparse. tools/lintstack is changed to pass pylint test.
Partial-Bug: #1279611
Change-Id: I0ceaaccbf6c67b45609b4f2f5639c362b32d10ad
This patch adds oslo.messaging to the
CINDER_CONFIG_GENERATOR_EXTRA_LIBRARIES env variable, which adds
oslo.messaging config options to cinder.conf.sample
Implements blueprint: oslo-messaging
Change-Id: I8aed8a8fdbd5b4ef41e3a3a7dcdebc6f216ec75d
This sync pulls the latest Oslo config code over
to Cinder. This sync is needed to include a fix
to config generator which is needed as part of
the work to enabled i18n messages. Config generator
was failing when lazy message translation was enabled.
The following changes are included with this patch. Each
patch includes the file(s) the patch touches:
-> c178e56 Add basic Python 3 tests
--> cinder/openstack/common/__init__.py
-> 12bcdb7 Remove vim header
--> cinder/openstack/common/__init__.py
-> 547ab34 Fix Copyright Headers - Rename LLC to Foundation
--> cinder/openstack/common/__init__.py
-> 44b6ea3 Import oslo.config configuration file generator
--> cinder/openstack/common/config/__init__.py
-> dd9aa2b Remove unused variables
--> cinder/openstack/common/config/generator.py
-> 5dce17b Use entry points to discover options in libraries
--> tools/config/generate_sample.sh
--> cinder/openstack/common/config/generator.py
-> e8e636c generator: add an EXTRA_LIBRARIES env variable
--> cinder/openstack/common/config/generator.py
-> e3dddd7 generator: use EXTRA_* env vars in the bash script
--> tools/config/generate_sample.sh
--> cinder/openstack/common/config/generator.py
-> 6da13e8 generator: rename EXTRA_MODULES_FILE to RC_FILE
--> tools/config/generate_sample.sh
-> 763eedf Fix DictOpt support in config sample generator
--> cinder/openstack/common/config/generator.py
-> e839886 Config generator fails with lazy messages
--> cinder/openstack/common/config/generator.py
-> 343686b Add check_uptodate to tools/config
--> tools/config/check_uptodate.sh
Oslo version:
-> 0f24d82 Fix migration.db_version when no tables
-> Date: Sat, 22 Feb 2014 00:32:18 +0000
Change-Id: I26a95fe96b08d6340b0fce1b9e2949c8e661a946
Closes-Bug: 1280826
Related-bp: i18n-messages
to keep Python 3.x compatibility, use six.moves.cStringIO to
replace StringIO
Change-Id: Id89c6156d9bba0007e8b16cd2cc28413a7cc5fd3
Closes-Bug: #1280964
This fixes the sample configuration to contain proper
config groups alongway.
Regenerate sample config with oslo's config generator.
DocImpact: Removes the deprecated memcached_server
option from DEFAULT (now in keystone_authtoken group)
Closes-Bug: #1206650
Closes-Bug: #1260322
Change-Id: I170715bfe6ec7dad9f7317e7bb56cbb68034c4ec
Currently if --virtual-env-name is specified, the tests fail
because it still creates .venv/ but then looks in the specified
location to use the venv.
This makes --virtual-env-name work as expected.
Change-Id: I036cd08f7b7b9dc87e7012f8dd198a26375297a2
In case MultiStrOpt option's value is empty list, current extract_opts.py
logic will ignore it. This patch aims to update func _print_opt:
add the multistropt's option_name in conf even its value is empty list.
This bug also affects https://bugs.launchpad.net/cinder/+bug/1240783
Closes-bug: #1241348
Change-Id: Ica1da0cd20c97a587964cb0059836d609a440145
Updates our example config file so that it includes configuration
settings for the auth_token middleware.
A step towards using cinder.conf instead of the api-paste.ini config file
for authtoken configuration.
Once https://review.openstack.org/#/c/52259/ lands I'll push another
review which removes the settings from api-paste.ini (which takes priority).
Related-Bug #1240753
Change-Id: I6636d33ee522757145ac97fc354324a8b9379700
This fixes all triggers of hacking 0.6.x's warning:
H404 - multi line docstring should start with a summary.
and enables gating check on H404.
Change-Id: I034bd1f05da3f279d8d79aa14a7f6ce8bef5047c
This doesn't serve much purpose, and makes it much harder than it
should be to merge multiple patches that change the sample config.
Change-Id: I44895bfbf4a7c9af30040f84b17a97f8e2f4e5b6
See I62ce43a330d7ae94eda4c7498782a655e63747fa for the gorey details on
why this exists.
As of this fix:
https://github.com/eventlet/eventlet/pull/34
which was released in eventlet 0.13, we no longer need the patch.
This has now been removed from oslo-incubator, so this is really just
syncing that removal.
Change-Id: I84267f3c6726cb2e750f615e107c48b12c6ed353
