The oslo.messaging library takes the existing RPC code from oslo and
wraps it in a sane API with well defined semantics around which we can
make a commitment to retain compatibility in future.
The patch is large, but the changes can be summarized as:
* oslo.messaging>=1.3.0a4 is required; a proper 1.3.0 release will be
pushed before the icehouse release candidates.
* The new rpc module has init() and cleanup() methods which manage the
global oslo.messaging transport state. The TRANSPORT and NOTIFIER
globals are conceptually similar to the current RPCIMPL global,
except we're free to create and use alternate Transport objects
in e.g. the cells code.
* The rpc.get_{client,server,notifier}() methods are just helpers
which wrap the global messaging state, specifiy serializers and
specify the use of the eventlet executor.
* In oslo.messaging, a request context is expected to be a dict so
we add a RequestContextSerializer which can serialize to and from
dicts using RequestContext.{to,from}_dict()
* The allowed_rpc_exception_modules configuration option is replaced
by an allowed_remote_exmods get_transport() parameter. This is not
something that users ever need to configure, but it is something
each project using oslo.messaging needs to be able to customize.
* We maintain a global NOTIFIER object and create specializations of
it with specific publisher IDs in order to avoid notification driver
loading overhead.
* rpc.py contains transport aliases for backwards compatibility
purposes. setup.cfg also contains notification driver aliases for
backwards compat.
* The messaging options are moved about in cinder.conf.sample because
the options are advertised via a oslo.config.opts entry point and
picked up by the generator.
* We use messaging.ConfFixture in tests to override oslo.messaging
config options, rather than making assumptions about the options
registered by the library.
Implements blueprint: oslo-messaging
Change-Id: Ib912809428d92e788558439e2d85b51272ebefdd
This patch re-enables lazy translation to enable REST API
responses to be translated to the requested locale (I.E. setting
'Accept-Language: zh-CN'.)
This patch also removes the unnecessary manual import of
'cinder.openstack.common.gettextutils _' in Cinder code. Since _ is
still being imported in the cinder.openstack.common files from Oslo
we still need to set gettexttutils.enable_lazy(). enable_lazy will
be able to be removed once the manual imports of _ have been removed
from Oslo.
implements-bp: i18n-messages
Change-Id: I0b5a35a2cc28d97d12ceccccd58565b487fef428
Due to the limit of Python interpreter, API service of Cinder can't
really utilize underlying multi-core architecture even libraries
like eventlet has been used. To make API service much more scalable,
we'd adopt multi-process (worker) mode that has been used for long
in Glance/Swift/Nova.
The default behavior isn't changed with this patch, Cinder API
service will still run in one process (default value of
osapi_volume_workers is None).
Implementation wise, a good portion of cinder/service.py has been
removed because those content has been merged in Oslo version of
service module. cinder/wsgi.py is also updated to adopt the change
for multiple WSGI servers running in separate processes.
Implement bp: multi-process-api-service
DocImpact: 'New config option osapi_volume_workers is used to specify
number of API service workers (OS processes) to launch for Cinder
API service. Setting this config option to a proper value (e.g.
osapi_volume_workers = # of CPU cores/threads of the machine) can
greatly improve the total throughput of API service [# of API
requests can be handled per second].'
Also removed out-dated comments in bin/cinder-api due to the fact
that this bug [1] has been fixed in eventlet 0.9.13
[1] https://bitbucket.org/eventlet/eventlet/issue/92/eventletgreen-override-of-oswaitpid
Change-Id: I8361d0dc0d43040e48634ff1aee1324e5e0af466
Remove rootwrap code copied from oslo-incubator, make
cinder-rootwrap a console_script entrypoint pointing in
oslo.rootwrap instead.
Change-Id: I4e924bc9fccd7f5a8522ac38b7c6d29683990ea8
Implements: blueprint cinder-oslo-rootwrap
Backup service is missing in 'cinder-all' script, this patch simply
add it so that this all-in-one script could launch all Cinder
Services.
Change-Id: I29c7355baabddd18a14fe9f97b71f58d24a033dc
The script 'cinder-volume-usage-audit' is generally run by cron to
generate volume and snapshot exists audit notifications that can be
consumed by ceilometer or other auditing systems. In this mode it sends
notifications for volumes that existed in the 'last completed audit
period' as calculated by cinder.utils.last_completed_audit_period()
However, it may sometimes be necessary to be able to send notifications
for audit periods other than than the last completed audit period. For
example if the host running the audit script was down for a period.
This patch provides three optional command line parameters to the script
cinder-volume-usage-audit. If no parameters are provided then the
operation of the script is unchanged. If parameters are provided then
they have the following effects:
--start_time specifies a date and time that is used in place of the
start of the last completed audit period.
--end_time specifies a date and time that is used in place of the end of
the last completed audit period.
--send_actions As well as sending exits notifications the script will
send any create or delete notifications that would have been sent during
the specified audit period.
Closes-Bug: #1259147
Change-Id: I0cfe963ea59d811ba4bac0bd3de31382a088b8c8
Catch OSError(s) from os.getlogin() and fall back to looking at
environment variables to figure out the user's name. The OSError
is thrown where there is no tty for the python process when
os.getlogin() is called.
Related-Bug: #1221491
Change-Id: I2bd735c9669ba9d25da108da44ea602f358b2dcc
Many more output strings in cinder/bin/*.py don't support i18n,
which should be fixed as visible message.
Change-Id: I863c09a0a398b99360e5a080091e320b594d4024
Closes-Bug: #1257613
Update hacking version to match version specified in requirements repo.
Fixed the following issues, which the newer version checks for:
print "" -> print("")
self.assertEquals -> self.assertEqual
self.assertNotEquals -> self.assertNotEqual
Change-Id: Ic4b70fd8f565cda28e23fe6b1da0e278f949373c
Closes-Bug: #1256738
Closes-Bug: #1256737
Closes-Bug: #1257274
This reverts commit b66c3361be2095186e19b0d44e49a8b2c48f7235.
Dependencies of this code (rtslib-fb) are now Apache-licensed,
which resolves the previous issue with this code being in-tree.
This patch is a direct revert, other patches will follow to
line up changes that have occurred between b66c336 and now.
Conflicts:
setup.py
Blueprint: lio-rtslib
Change-Id: I14786bddf419d98ff7c468b185462f5387dbbba1
Late in the Havana cycle bug 1225099 was found in the lazy
translation code, and to be safe it was decided to disable lazy
translation for Havana. This change does that.
Change-Id: Ia934a7df9386baf6ae8eb9ff48c24386c47ecd23
Partial-bug: 1225099
In python 3 print statement is not supported, so we should use
only print() functions.
Fixes bug 1226943
Change-Id: I7b2e4d52fe9050f6a67c44e4cc1237a15ea90b23
The Windows service fails due to missing non-blocking IO features
in eventlet. This fix adds a conditional path on Windows to execute
the service accordingly.
Fixes bug: #1219896
Change-Id: I74f662e736e3a5fe58a383d172780a691a2b36c7
The new auto-pulled requirements files exposed
python 3 compat issues in a number of modules.
We added these to tox.ini ignore temporarily,
this change updates the print routines in bin/cinder
and removes the debug cruft that was in the unit tests.
Will update requirements and test-requirements
appropriately in a follow up patch.
Change-Id: I76ea24f4dc1f61c4f8a1d202b0554c90daf2c9cc
Add support for doing language resolution for a request, based on the
Accept-Language HTTP header.
Using the lazy gettext functionality from oslo gettextutils, it is now
possible to use the resolved language to translate an exception message
to the user requested language and return that translation from the API.
Partially implements bp user-locale-api
Change-Id: Ib2c8360372996d53b50542df54a52d92b07295ca
Removing leftover sm_backend manage code that was missed in the previous
code removal in 40cb16784f920387ef.
Fixes: bug #1207189
Change-Id: I52c44785924f489a85e4b5f947ed9b4adf679dd8
Unlike other cinder-manage 'list' commands the 'config list'
option doesn't allow users to specify a filter to limit
the output. This commit adds the ability to specify
the flag the user wishes to display.
If no flag is specified the default behavior is still to
display all the configured flags. If the flag requested
is not found, cinder-manage reports that the flag was not
found.
(fixes bug 1187137)
Change-Id: I698f4c06d7e93217d8f307a880e0ae40711151c2
Replace FLAGS with cfg.CONF
Rename modules fake_flags to conf_fixture, test_flags to test_conf, declare_flags to declare_conf, runtime_flags to runtime_conf
Renamed cinder.flags, because exactly the same was done in the glance and nova
Fixes: bug #1182037
Change-Id: I3424d0a401b3ef7a3254d3e913263554361a52ff
The 'migrate' option was only appropriate for use migrating
databases for volumes from Nova-Folsom-->Cinder-Folsom. The
code hasn't been tested since.
Given the above facts, this commit removes the migrate code from
cinder-manage. The man page for cinder-manage was already
updated to remove the option in a previous commit.
(fixes bug #1187134)
Change-Id: I7bf378cd4b0dd4f6d1bdf475b826a59a31112638
Per (proposed) H203 check, convert all uses of
the deprecated except x,y: construct to except
x as y:, which works with any Python version >= 2.6
Change-Id: I5528dc556f3ef8d356e01d59df04ba57f66c95b7
If a user runs something like "cinder-manage volume delete a1234",
a ValueError is thrown because it fails to cast to int.
Catch this and treat the parameter as an id which will result in a
later VolumeNotFound error rather than breaking this way.
Change-Id: I95a9b9d7628cebe4b6d855ea925b0ad3a5f1c4c4
Part of fixing bug #995287
Syncs these two commits from oslo-incubator:
Support overriding oslo localedir too
Add a gettextutils.install() helper function
to get a new gettextutils.install() function which allows the default
localedir to be overwritten via an environment variable.
Note that gettextutils.install() must be called before any other cinder
modules are imported since some modules attempt to translate strings
at import time (e.g. in cinder.flags). This is broken and inefficient,
but fixing it involves adding something like sphinx's l_() function and
would be very invaisve.
Change-Id: I86562b3a65d371673bb21f7179eecc7602bc0775
The gettext.install() function installs a builtin _() function which
translates a string in the translation domain supplied to the install()
function. If gettext.install() is called multiple times, it's the last
call to the function which wins and the last supplied translation domain
which is used e.g.
>>> import os
>>> os.environ['LANG'] = 'ja.UTF-8'
>>> import gettext
>>> gettext.install('keystone', unicode=1, localedir='/opt/stack/keystone/keystone/locale')
>>> print _('Invalid syslog facility')
n無効な syslog ファシリティ
>>> gettext.install('cinder', unicode=1, localedir='/opt/stack/nova/cinder/locale')
>>> print _('Invalid syslog facility')
Invalid syslog facility
Usually this function is called early on in a toplevel script and we
assume that no other code will call it and override the installed _().
However, in Cinder, we have taken a shortcut to avoid having to call it
explicitly from each script and instead call it from cinder/__init__.py.
This shortcut would be perfectly fine if we were absolutely sure that
nova modules would never be imported from another program. It's probably
quite incorrect for a program to use cinder code (indeed, if we wanted
to support this, Cinder code shouldn't use the default _() function) but
nevertheless there are some corner cases where it happens. For example,
the keystoneclient auth_token middleware tries to import cfg from
cinder.openstack and this in turn causes gettext.install('cinder') in
other projects like glance or quantum.
To avoid any doubt here, let's just rip out the shortcut and always
call gettext.install() from the top-level script.
However, there's a bit of an annoying detail here - by default,
nosetests starts in the current directly and tries to import all modules
it finds to look for tests. Without the _() builtin installed, importing
some modules like cinder.flags will fail.
Since it only ever makes sense to load tests from the cinder/tests dir,
we can ask nose to do that by using the --tests argument via setup.cfg.
Note, this means that if you previously did this:
$> tox -- cinder.tests.foo cinder.tests.bar
then you must now do this:
$> tox -- --tests cinder.tests.foo,cinder.tests.bar
Change-Id: If4125d6bcbde63df95de129ac5c83b4a6d6f130a
logging needs to be setup properly before we try to log
something. Fixed one spot from logging->LOG
Fixes LP# 1161038
Change-Id: I87743cff9062e47c1d5e989c2c0c05c3921b65c7
Add uuid support to param2id() to allow commands including
cinder-manage volume delete to work with uuid style volumd ids
Fixes bug #1158409
Change-Id: I2d197320b61f44d1104ed0342a6b5b70f673468a
This patch updates all the files from oslo under
stable/grizzly but policy.py. policy.py is a high risk
change at this date and it has lots of changes since the
last time we updated it.
Fixes bug #1157126
Change-Id: I399a1cd8474f718ed5196def90d2fea546fb01f5
rtslib is AGPL, so cinder-rtstool is defacto AGPL by importing it.
Remove cinder-rtstool from the OpenStack repository.
Fixes bug #1157091
Change-Id: I30c9ee19d43107fa7f2e5e65d69049b8d69d2bdf
* Added tests for db api call volume_get_active_by_window called by the volume
usage audit report.
* Added new db api called snapshot_get_active_by_window & test it. This is
similar to the volume_get_active_by_window method but returns the snapshots
active the supplied time window.
* Integrate the above into the cinder-volume-usage-audit script to report on
all the active snapshots as well as volumes in the specified time frame.
Fixes: bug #1157110
Change-Id: I7597da20c62150a8f6c74cbfeec159f68064d2a6
Update oslo rpc libraries to capture changes, primarly motivated
by secret=True flag on password config options. Skipping broken,
invalid test case while working on correct fix.
Change-Id: Ibb979189b4a6215f307cb49e4a17070ffc7f0f51
Currently, IQNs of remote nova compute nodes must be specified in
cinder.conf for them to be added to LIO's ACLs for LUNs.
This change will handle this at volume-attach time instead.
Change-Id: I278ce737042b15bd4d100d331564c1377bac0c55
Changes:
- Added cinder-backup to scripts in setup.py
- Added service_name to backup manager __init__
- Switched backup manager to using oslo.config
- Modified cinder-backup to use ProcessLauncher
Fixes bug: 1131147
Change-Id: I9b02d64def4ee892e9cfde4b4bcd50e5f9f9b707
