diff --git a/doc/source/development_middleware.rst b/doc/source/development_middleware.rst index 14bfcddb5b..b6dac83289 100644 --- a/doc/source/development_middleware.rst +++ b/doc/source/development_middleware.rst @@ -200,6 +200,8 @@ core swift features which predate sysmeta have added exceptions for custom non-user metadata headers (e.g. :ref:`acls`, :ref:`large-objects`) +.. _usermeta: + ^^^^^^^^^^^^^ User Metadata ^^^^^^^^^^^^^ @@ -209,7 +211,7 @@ User metadata takes the form of ``X--Meta-: ``, where and ```` and ```` are set by the client. User metadata should generally be reserved for use by the client or -client applications. An perfect example use-case for user metadata is +client applications. A perfect example use-case for user metadata is `python-swiftclient`_'s ``X-Object-Meta-Mtime`` which it stores on object it uploads to implement its ``--changed`` option which will only upload files that have changed since the last upload. @@ -223,6 +225,20 @@ borrows the user metadata namespace is :ref:`tempurl`. An example of middleware which uses custom non-user metadata to avoid the user metadata namespace is :ref:`slo-doc`. +User metadata that is stored by a PUT or POST request to a container or account +resource persists until it is explicitly removed by a subsequent PUT or POST +request that includes a header ``X--Meta-`` with no value or a +header ``X-Remove--Meta-: ``. In the latter case the +```` is not stored. All user metadata stored with an account or +container resource is deleted when the account or container is deleted. + +User metadata that is stored with an object resource has a different semantic; +object user metadata persists until any subsequent PUT or POST request is made +to the same object, at which point all user metadata stored with that object is +deleted en-masse and replaced with any user metadata included with the PUT or +POST request. As a result, it is not possible to update a subset of the user +metadata items stored with an object while leaving some items unchanged. + .. _sysmeta: ^^^^^^^^^^^^^^^ @@ -237,7 +253,7 @@ Swift WSGI Server. All headers on client requests in the form of ``X--Sysmeta-`` will be dropped from the request before being processed by any middleware. All headers on responses from back-end systems in the form -of ``X--Sysmeta-`` will be removed after all middleware has +of ``X--Sysmeta-`` will be removed after all middlewares have processed the response but before the response is sent to the client. See :ref:`gatekeeper` middleware for more information. @@ -249,3 +265,50 @@ modified directly by client requests, and the outgoing filter ensures that removing middleware that uses a specific system metadata key renders it benign. New middleware should take advantage of system metadata. + +System metadata may be set on accounts and containers by including headers with +a PUT or POST request. Where a header name matches the name of an existing item +of system metadata, the value of the existing item will be updated. Otherwise +existing items are preserved. A system metadata header with an empty value will +cause any existing item with the same name to be deleted. + +System metadata may be set on objects using only PUT requests. All items of +existing system metadata will be deleted and replaced en-masse by any system +metadata headers included with the PUT request. System metadata is neither +updated nor deleted by a POST request: updating individual items of system +metadata with a POST request is not yet supported in the same way that updating +individual items of user metadata is not supported. In cases where middleware +needs to store its own metadata with a POST request, it may use Object Transient +Sysmeta. + +^^^^^^^^^^^^^^^^^^^^^^^^ +Object Transient-Sysmeta +^^^^^^^^^^^^^^^^^^^^^^^^ + +If middleware needs to store object metadata with a POST request it may do so +using headers of the form ``X-Object-Transient-Sysmeta-: ``. + +All headers on client requests in the form of +``X-Object-Transient-Sysmeta-`` will be dropped from the request before +being processed by any middleware. All headers on responses from back-end +systems in the form of ``X-Object-Transient-Sysmeta-`` will be removed +after all middlewares have processed the response but before the response is +sent to the client. See :ref:`gatekeeper` middleware for more information. + +Transient-sysmeta updates on an object have the same semantic as user +metadata updates on an object (see :ref:`usermeta`) i.e. whenever any PUT or +POST request is made to an object, all existing items of transient-sysmeta are +deleted en-masse and replaced with any transient-sysmeta included with the PUT +or POST request. Transient-sysmeta set by a middleware is therefore prone to +deletion by a subsequent client-generated POST request unless the middleware is +careful to include its transient-sysmeta with every POST. Likewise, user +metadata set by a client is prone to deletion by a subsequent +middleware-generated POST request, and for that reason middleware should avoid +generating POST requests that are independent of any client request. + +Transient-sysmeta deliberately uses a different header prefix to user metadata +so that middlewares can avoid potential conflict with user metadata keys. + +Transient-sysmeta deliberately uses a different header prefix to system +metadata to emphasize the fact that the data is only persisted until a +subsequent POST. diff --git a/swift/common/middleware/copy.py b/swift/common/middleware/copy.py index a5fc44ca2d..1daadfe90c 100644 --- a/swift/common/middleware/copy.py +++ b/swift/common/middleware/copy.py @@ -145,7 +145,7 @@ from swift.common.http import HTTP_MULTIPLE_CHOICES, HTTP_CREATED, \ is_success, HTTP_OK from swift.common.constraints import check_account_format, MAX_FILE_SIZE from swift.common.request_helpers import copy_header_subset, remove_items, \ - is_sys_meta, is_sys_or_user_meta + is_sys_meta, is_sys_or_user_meta, is_object_transient_sysmeta from swift.common.wsgi import WSGIContext, make_subrequest @@ -206,16 +206,18 @@ def _check_destination_header(req): '/') -def _copy_headers_into(from_r, to_r): +def _copy_headers(src, dest): """ - Will copy desired headers from from_r to to_r - :params from_r: a swob Request or Response - :params to_r: a swob Request or Response + Will copy desired headers from src to dest. + + :params src: an instance of collections.Mapping + :params dest: an instance of collections.Mapping """ - pass_headers = ['x-delete-at'] - for k, v in from_r.headers.items(): - if is_sys_or_user_meta('object', k) or k.lower() in pass_headers: - to_r.headers[k] = v + for k, v in src.items(): + if (is_sys_or_user_meta('object', k) or + is_object_transient_sysmeta(k) or + k.lower() == 'x-delete-at'): + dest[k] = v class ServerSideCopyWebContext(WSGIContext): @@ -422,9 +424,7 @@ class ServerSideCopyMiddleware(object): source_resp.headers['last-modified'] # Existing sys and user meta of source object is added to response # headers in addition to the new ones. - for k, v in sink_req.headers.items(): - if is_sys_or_user_meta('object', k) or k.lower() == 'x-delete-at': - resp_headers[k] = v + _copy_headers(sink_req.headers, resp_headers) return resp_headers def handle_PUT(self, req, start_response): @@ -511,10 +511,10 @@ class ServerSideCopyMiddleware(object): remove_items(sink_req.headers, condition) copy_header_subset(source_resp, sink_req, condition) else: - # Copy/update existing sysmeta and user meta - _copy_headers_into(source_resp, sink_req) + # Copy/update existing sysmeta, transient-sysmeta and user meta + _copy_headers(source_resp.headers, sink_req.headers) # Copy/update new metadata provided in request if any - _copy_headers_into(req, sink_req) + _copy_headers(req.headers, sink_req.headers) # Create response headers for PUT response resp_headers = self._create_response_headers(source_path, diff --git a/swift/common/middleware/gatekeeper.py b/swift/common/middleware/gatekeeper.py index c5c1066505..e5df5bf44c 100644 --- a/swift/common/middleware/gatekeeper.py +++ b/swift/common/middleware/gatekeeper.py @@ -33,22 +33,25 @@ automatically inserted close to the start of the pipeline by the proxy server. from swift.common.swob import Request from swift.common.utils import get_logger, config_true_value -from swift.common.request_helpers import remove_items, get_sys_meta_prefix +from swift.common.request_helpers import ( + remove_items, get_sys_meta_prefix, OBJECT_TRANSIENT_SYSMETA_PREFIX +) import re #: A list of python regular expressions that will be used to #: match against inbound request headers. Matching headers will #: be removed from the request. # Exclude headers starting with a sysmeta prefix. +# Exclude headers starting with object transient system metadata prefix. +# Exclude headers starting with an internal backend header prefix. # If adding to this list, note that these are regex patterns, # so use a trailing $ to constrain to an exact header match # rather than prefix match. inbound_exclusions = [get_sys_meta_prefix('account'), get_sys_meta_prefix('container'), get_sys_meta_prefix('object'), + OBJECT_TRANSIENT_SYSMETA_PREFIX, 'x-backend'] -# 'x-object-sysmeta' is reserved in anticipation of future support -# for system metadata being applied to objects #: A list of python regular expressions that will be used to diff --git a/swift/common/request_helpers.py b/swift/common/request_helpers.py index 71a32106af..65f21bebce 100644 --- a/swift/common/request_helpers.py +++ b/swift/common/request_helpers.py @@ -44,6 +44,9 @@ from swift.common.utils import split_path, validate_device_partition, \ from swift.common.wsgi import make_subrequest +OBJECT_TRANSIENT_SYSMETA_PREFIX = 'x-object-transient-sysmeta-' + + def get_param(req, name, default=None): """ Get parameters from an HTTP request ensuring proper handling UTF-8 @@ -175,6 +178,19 @@ def is_sys_or_user_meta(server_type, key): return is_user_meta(server_type, key) or is_sys_meta(server_type, key) +def is_object_transient_sysmeta(key): + """ + Tests if a header key starts with and is longer than the prefix for object + transient system metadata. + + :param key: header key + :returns: True if the key satisfies the test, False otherwise + """ + if len(key) <= len(OBJECT_TRANSIENT_SYSMETA_PREFIX): + return False + return key.lower().startswith(OBJECT_TRANSIENT_SYSMETA_PREFIX) + + def strip_user_meta_prefix(server_type, key): """ Removes the user metadata prefix for a given server type from the start @@ -199,6 +215,17 @@ def strip_sys_meta_prefix(server_type, key): return key[len(get_sys_meta_prefix(server_type)):] +def strip_object_transient_sysmeta_prefix(key): + """ + Removes the object transient system metadata prefix from the start of a + header key. + + :param key: header key + :returns: stripped header key + """ + return key[len(OBJECT_TRANSIENT_SYSMETA_PREFIX):] + + def get_user_meta_prefix(server_type): """ Returns the prefix for user metadata headers for given server type. @@ -225,6 +252,20 @@ def get_sys_meta_prefix(server_type): return 'x-%s-%s-' % (server_type.lower(), 'sysmeta') +def get_object_transient_sysmeta(key): + """ + Returns the Object Transient System Metadata header for key. + The Object Transient System Metadata namespace will be persisted by + backend object servers. These headers are treated in the same way as + object user metadata i.e. all headers in this namespace will be + replaced on every POST request. + + :param key: metadata key + :returns: the entire object transient system metadata header for key + """ + return '%s%s' % (OBJECT_TRANSIENT_SYSMETA_PREFIX, key) + + def remove_items(headers, condition): """ Removes items from a dict whose keys satisfy diff --git a/swift/obj/server.py b/swift/obj/server.py index 7193b73e70..1edefb8cd4 100644 --- a/swift/obj/server.py +++ b/swift/obj/server.py @@ -46,7 +46,8 @@ from swift.common.http import is_success from swift.common.base_storage_server import BaseStorageServer from swift.common.header_key_dict import HeaderKeyDict from swift.common.request_helpers import get_name_and_placement, \ - is_user_meta, is_sys_or_user_meta, resolve_etag_is_at_header + is_user_meta, is_sys_or_user_meta, is_object_transient_sysmeta, \ + resolve_etag_is_at_header from swift.common.swob import HTTPAccepted, HTTPBadRequest, HTTPCreated, \ HTTPInternalServerError, HTTPNoContent, HTTPNotFound, \ HTTPPreconditionFailed, HTTPRequestTimeout, HTTPUnprocessableEntity, \ @@ -520,7 +521,8 @@ class ObjectController(BaseStorageServer): metadata = {'X-Timestamp': req_timestamp.internal} self._preserve_slo_manifest(metadata, orig_metadata) metadata.update(val for val in request.headers.items() - if is_user_meta('object', val[0])) + if (is_user_meta('object', val[0]) or + is_object_transient_sysmeta(val[0]))) headers_to_copy = ( request.headers.get( 'X-Backend-Replication-Headers', '').split() + @@ -767,9 +769,11 @@ class ObjectController(BaseStorageServer): 'Content-Length': str(upload_size), } metadata.update(val for val in request.headers.items() - if is_sys_or_user_meta('object', val[0])) + if (is_sys_or_user_meta('object', val[0]) or + is_object_transient_sysmeta(val[0]))) metadata.update(val for val in footer_meta.items() - if is_sys_or_user_meta('object', val[0])) + if (is_sys_or_user_meta('object', val[0]) or + is_object_transient_sysmeta(val[0]))) headers_to_copy = ( request.headers.get( 'X-Backend-Replication-Headers', '').split() + @@ -861,8 +865,9 @@ class ObjectController(BaseStorageServer): response.headers['Content-Type'] = metadata.get( 'Content-Type', 'application/octet-stream') for key, value in metadata.items(): - if is_sys_or_user_meta('object', key) or \ - key.lower() in self.allowed_headers: + if (is_sys_or_user_meta('object', key) or + is_object_transient_sysmeta(key) or + key.lower() in self.allowed_headers): response.headers[key] = value response.etag = metadata['ETag'] response.last_modified = math.ceil(float(file_x_ts)) @@ -913,8 +918,9 @@ class ObjectController(BaseStorageServer): response.headers['Content-Type'] = metadata.get( 'Content-Type', 'application/octet-stream') for key, value in metadata.items(): - if is_sys_or_user_meta('object', key) or \ - key.lower() in self.allowed_headers: + if (is_sys_or_user_meta('object', key) or + is_object_transient_sysmeta(key) or + key.lower() in self.allowed_headers): response.headers[key] = value response.etag = metadata['ETag'] ts = Timestamp(metadata['X-Timestamp']) diff --git a/swift/proxy/controllers/base.py b/swift/proxy/controllers/base.py index 407a7aed93..c1a909dad5 100644 --- a/swift/proxy/controllers/base.py +++ b/swift/proxy/controllers/base.py @@ -58,7 +58,8 @@ from swift.common.swob import Request, Response, Range, \ status_map from swift.common.request_helpers import strip_sys_meta_prefix, \ strip_user_meta_prefix, is_user_meta, is_sys_meta, is_sys_or_user_meta, \ - http_response_to_document_iters + http_response_to_document_iters, is_object_transient_sysmeta, \ + strip_object_transient_sysmeta_prefix from swift.common.storage_policy import POLICIES @@ -180,12 +181,18 @@ def headers_to_object_info(headers, status_int=HTTP_OK): Construct a cacheable dict of object info based on response headers. """ headers, meta, sysmeta = _prep_headers_to_info(headers, 'object') + transient_sysmeta = {} + for key, val in headers.iteritems(): + if is_object_transient_sysmeta(key): + key = strip_object_transient_sysmeta_prefix(key.lower()) + transient_sysmeta[key] = val info = {'status': status_int, 'length': headers.get('content-length'), 'type': headers.get('content-type'), 'etag': headers.get('etag'), 'meta': meta, 'sysmeta': sysmeta, + 'transient_sysmeta': transient_sysmeta } return info diff --git a/test/probe/test_object_metadata_replication.py b/test/probe/test_object_metadata_replication.py index 4759d5dfc3..57ef8e455e 100644 --- a/test/probe/test_object_metadata_replication.py +++ b/test/probe/test_object_metadata_replication.py @@ -339,6 +339,8 @@ class Test(ReplProbeTest): def test_sysmeta_after_replication_with_subsequent_post(self): sysmeta = {'x-object-sysmeta-foo': 'sysmeta-foo'} usermeta = {'x-object-meta-bar': 'meta-bar'} + transient_sysmeta = { + 'x-object-transient-sysmeta-bar': 'transient-sysmeta-bar'} self.brain.put_container(policy_index=int(self.policy)) # put object self._put_object() @@ -356,11 +358,13 @@ class Test(ReplProbeTest): # post some user meta to second server subset self.brain.stop_handoff_half() self.container_brain.stop_handoff_half() - self._post_object(usermeta) + user_and_transient_sysmeta = dict(usermeta) + user_and_transient_sysmeta.update(transient_sysmeta) + self._post_object(user_and_transient_sysmeta) metadata = self._get_object_metadata() - for key in usermeta: + for key in user_and_transient_sysmeta: self.assertTrue(key in metadata) - self.assertEqual(metadata[key], usermeta[key]) + self.assertEqual(metadata[key], user_and_transient_sysmeta[key]) for key in sysmeta: self.assertFalse(key in metadata) self.brain.start_handoff_half() @@ -376,6 +380,7 @@ class Test(ReplProbeTest): metadata = self._get_object_metadata() expected = dict(sysmeta) expected.update(usermeta) + expected.update(transient_sysmeta) for key in expected.keys(): self.assertTrue(key in metadata, key) self.assertEqual(metadata[key], expected[key]) @@ -399,6 +404,8 @@ class Test(ReplProbeTest): def test_sysmeta_after_replication_with_prior_post(self): sysmeta = {'x-object-sysmeta-foo': 'sysmeta-foo'} usermeta = {'x-object-meta-bar': 'meta-bar'} + transient_sysmeta = { + 'x-object-transient-sysmeta-bar': 'transient-sysmeta-bar'} self.brain.put_container(policy_index=int(self.policy)) # put object self._put_object() @@ -406,11 +413,13 @@ class Test(ReplProbeTest): # put user meta to first server subset self.brain.stop_handoff_half() self.container_brain.stop_handoff_half() - self._post_object(headers=usermeta) + user_and_transient_sysmeta = dict(usermeta) + user_and_transient_sysmeta.update(transient_sysmeta) + self._post_object(user_and_transient_sysmeta) metadata = self._get_object_metadata() - for key in usermeta: + for key in user_and_transient_sysmeta: self.assertTrue(key in metadata) - self.assertEqual(metadata[key], usermeta[key]) + self.assertEqual(metadata[key], user_and_transient_sysmeta[key]) self.brain.start_handoff_half() self.container_brain.start_handoff_half() @@ -436,7 +445,7 @@ class Test(ReplProbeTest): for key in sysmeta: self.assertTrue(key in metadata) self.assertEqual(metadata[key], sysmeta[key]) - for key in usermeta: + for key in user_and_transient_sysmeta: self.assertFalse(key in metadata) self.brain.start_primary_half() self.container_brain.start_primary_half() @@ -449,7 +458,7 @@ class Test(ReplProbeTest): for key in sysmeta: self.assertTrue(key in metadata) self.assertEqual(metadata[key], sysmeta[key]) - for key in usermeta: + for key in user_and_transient_sysmeta: self.assertFalse(key in metadata) self.brain.start_handoff_half() self.container_brain.start_handoff_half() diff --git a/test/unit/common/middleware/helpers.py b/test/unit/common/middleware/helpers.py index c295ee4768..1e31362f0d 100644 --- a/test/unit/common/middleware/helpers.py +++ b/test/unit/common/middleware/helpers.py @@ -19,6 +19,8 @@ from collections import defaultdict from hashlib import md5 from swift.common import swob from swift.common.header_key_dict import HeaderKeyDict +from swift.common.request_helpers import is_user_meta, \ + is_object_transient_sysmeta from swift.common.swob import HTTPNotImplemented from swift.common.utils import split_path @@ -87,7 +89,7 @@ class FakeSwift(object): if resp: return resp(env, start_response) - req_headers = swob.Request(env).headers + req = swob.Request(env) self.swift_sources.append(env.get('swift.source')) self.txn_ids.append(env.get('swift.trans_id')) @@ -114,26 +116,41 @@ class FakeSwift(object): # simulate object PUT if method == 'PUT' and obj: - input = ''.join(iter(env['wsgi.input'].read, '')) + put_body = ''.join(iter(env['wsgi.input'].read, '')) if 'swift.callback.update_footers' in env: footers = HeaderKeyDict() env['swift.callback.update_footers'](footers) - req_headers.update(footers) - etag = md5(input).hexdigest() + req.headers.update(footers) + etag = md5(put_body).hexdigest() headers.setdefault('Etag', etag) - headers.setdefault('Content-Length', len(input)) + headers.setdefault('Content-Length', len(put_body)) # keep it for subsequent GET requests later - self.uploaded[path] = (dict(req_headers), input) + self.uploaded[path] = (dict(req.headers), put_body) if "CONTENT_TYPE" in env: self.uploaded[path][0]['Content-Type'] = env["CONTENT_TYPE"] + # simulate object POST + elif method == 'POST' and obj: + metadata, data = self.uploaded.get(path, ({}, None)) + # select items to keep from existing... + new_metadata = dict( + (k, v) for k, v in metadata.items() + if (not is_user_meta('object', k) and not + is_object_transient_sysmeta(k))) + # apply from new + new_metadata.update( + dict((k, v) for k, v in req.headers.items() + if (is_user_meta('object', k) or + is_object_transient_sysmeta(k) or + k.lower == 'content-type'))) + self.uploaded[path] = new_metadata, data + # note: tests may assume this copy of req_headers is case insensitive # so we deliberately use a HeaderKeyDict - self._calls.append((method, path, HeaderKeyDict(req_headers))) + self._calls.append((method, path, HeaderKeyDict(req.headers))) # range requests ought to work, hence conditional_response=True - req = swob.Request(env) if isinstance(body, list): resp = resp_class( req=req, headers=headers, app_iter=body, diff --git a/test/unit/common/middleware/test_copy.py b/test/unit/common/middleware/test_copy.py index 3f024d4395..3a6663db00 100644 --- a/test/unit/common/middleware/test_copy.py +++ b/test/unit/common/middleware/test_copy.py @@ -689,9 +689,11 @@ class TestServerSideCopyMiddleware(unittest.TestCase): source_headers = { 'x-object-sysmeta-test1': 'copy me', 'x-object-meta-test2': 'copy me too', + 'x-object-transient-sysmeta-test3': 'ditto', 'x-object-sysmeta-container-update-override-etag': 'etag val', 'x-object-sysmeta-container-update-override-size': 'size val', - 'x-object-sysmeta-container-update-override-foo': 'bar'} + 'x-object-sysmeta-container-update-override-foo': 'bar', + 'x-delete-at': 'delete-at-time'} get_resp_headers = source_headers.copy() get_resp_headers['etag'] = 'source etag' @@ -713,20 +715,20 @@ class TestServerSideCopyMiddleware(unittest.TestCase): req = Request.blank('/v1/a/c/o', method='COPY', headers={'Content-Length': 0, 'Destination': 'c/o-copy0'}) - status, headers, body = self.call_ssc(req) + status, resp_headers, body = self.call_ssc(req) self.assertEqual('201 Created', status) - verify_headers(source_headers.copy(), [], headers) - method, path, headers = self.app.calls_with_headers[-1] + verify_headers(source_headers.copy(), [], resp_headers) + method, path, put_headers = self.app.calls_with_headers[-1] self.assertEqual('PUT', method) self.assertEqual('/v1/a/c/o-copy0', path) - verify_headers(source_headers.copy(), [], headers.items()) - self.assertIn('etag', headers) - self.assertEqual(headers['etag'], 'source etag') + verify_headers(source_headers.copy(), [], put_headers.items()) + self.assertIn('etag', put_headers) + self.assertEqual(put_headers['etag'], 'source etag') req = Request.blank('/v1/a/c/o-copy0', method='GET') - status, headers, body = self.call_ssc(req) + status, resp_headers, body = self.call_ssc(req) self.assertEqual('200 OK', status) - verify_headers(source_headers.copy(), [], headers) + verify_headers(source_headers.copy(), [], resp_headers) # use a COPY request with a Range header self.app.register('PUT', '/v1/a/c/o-copy1', swob.HTTPCreated, {}) @@ -734,7 +736,7 @@ class TestServerSideCopyMiddleware(unittest.TestCase): headers={'Content-Length': 0, 'Destination': 'c/o-copy1', 'Range': 'bytes=1-2'}) - status, headers, body = self.call_ssc(req) + status, resp_headers, body = self.call_ssc(req) expected_headers = source_headers.copy() unexpected_headers = ( 'x-object-sysmeta-container-update-override-etag', @@ -743,38 +745,54 @@ class TestServerSideCopyMiddleware(unittest.TestCase): for h in unexpected_headers: expected_headers.pop(h) self.assertEqual('201 Created', status) - verify_headers(expected_headers, unexpected_headers, headers) - method, path, headers = self.app.calls_with_headers[-1] + verify_headers(expected_headers, unexpected_headers, resp_headers) + method, path, put_headers = self.app.calls_with_headers[-1] self.assertEqual('PUT', method) self.assertEqual('/v1/a/c/o-copy1', path) - verify_headers(expected_headers, unexpected_headers, headers.items()) + verify_headers( + expected_headers, unexpected_headers, put_headers.items()) # etag should not be copied with a Range request - self.assertNotIn('etag', headers) + self.assertNotIn('etag', put_headers) req = Request.blank('/v1/a/c/o-copy1', method='GET') - status, headers, body = self.call_ssc(req) + status, resp_headers, body = self.call_ssc(req) self.assertEqual('200 OK', status) - verify_headers(expected_headers, unexpected_headers, headers) + verify_headers(expected_headers, unexpected_headers, resp_headers) # use a PUT with x-copy-from self.app.register('PUT', '/v1/a/c/o-copy2', swob.HTTPCreated, {}) req = Request.blank('/v1/a/c/o-copy2', method='PUT', headers={'Content-Length': 0, 'X-Copy-From': 'c/o'}) - status, headers, body = self.call_ssc(req) + status, resp_headers, body = self.call_ssc(req) self.assertEqual('201 Created', status) - verify_headers(source_headers.copy(), [], headers) - method, path, headers = self.app.calls_with_headers[-1] + verify_headers(source_headers.copy(), [], resp_headers) + method, path, put_headers = self.app.calls_with_headers[-1] self.assertEqual('PUT', method) self.assertEqual('/v1/a/c/o-copy2', path) - verify_headers(source_headers.copy(), [], headers.items()) - self.assertIn('etag', headers) - self.assertEqual(headers['etag'], 'source etag') + verify_headers(source_headers.copy(), [], put_headers.items()) + self.assertIn('etag', put_headers) + self.assertEqual(put_headers['etag'], 'source etag') req = Request.blank('/v1/a/c/o-copy2', method='GET') - status, headers, body = self.call_ssc(req) + status, resp_headers, body = self.call_ssc(req) self.assertEqual('200 OK', status) - verify_headers(source_headers.copy(), [], headers) + verify_headers(source_headers.copy(), [], resp_headers) + + # copy to same path as source + self.app.register('PUT', '/v1/a/c/o', swob.HTTPCreated, {}) + req = Request.blank('/v1/a/c/o', method='PUT', + headers={'Content-Length': 0, + 'X-Copy-From': 'c/o'}) + status, resp_headers, body = self.call_ssc(req) + self.assertEqual('201 Created', status) + verify_headers(source_headers.copy(), [], resp_headers) + method, path, put_headers = self.app.calls_with_headers[-1] + self.assertEqual('PUT', method) + self.assertEqual('/v1/a/c/o', path) + verify_headers(source_headers.copy(), [], put_headers.items()) + self.assertIn('etag', put_headers) + self.assertEqual(put_headers['etag'], 'source etag') def test_COPY_no_destination_header(self): req = Request.blank( diff --git a/test/unit/common/middleware/test_gatekeeper.py b/test/unit/common/middleware/test_gatekeeper.py index a01d45cbb1..5f4e87b5a2 100644 --- a/test/unit/common/middleware/test_gatekeeper.py +++ b/test/unit/common/middleware/test_gatekeeper.py @@ -74,12 +74,17 @@ class TestGatekeeper(unittest.TestCase): x_backend_headers = {'X-Backend-Replication': 'true', 'X-Backend-Replication-Headers': 'stuff'} + object_transient_sysmeta_headers = { + 'x-object-transient-sysmeta-': 'value', + 'x-object-transient-sysmeta-foo': 'value'} x_timestamp_headers = {'X-Timestamp': '1455952805.719739'} forbidden_headers_out = dict(sysmeta_headers.items() + - x_backend_headers.items()) + x_backend_headers.items() + + object_transient_sysmeta_headers.items()) forbidden_headers_in = dict(sysmeta_headers.items() + - x_backend_headers.items()) + x_backend_headers.items() + + object_transient_sysmeta_headers.items()) shunted_headers_in = dict(x_timestamp_headers.items()) def _assertHeadersEqual(self, expected, actual): diff --git a/test/unit/common/test_request_helpers.py b/test/unit/common/test_request_helpers.py index 1c39e9f0af..e451174516 100644 --- a/test/unit/common/test_request_helpers.py +++ b/test/unit/common/test_request_helpers.py @@ -21,8 +21,8 @@ from swift.common.storage_policy import POLICIES, EC_POLICY, REPL_POLICY from swift.common.request_helpers import is_sys_meta, is_user_meta, \ is_sys_or_user_meta, strip_sys_meta_prefix, strip_user_meta_prefix, \ remove_items, copy_header_subset, get_name_and_placement, \ - http_response_to_document_iters, update_etag_is_at_header, \ - resolve_etag_is_at_header + http_response_to_document_iters, is_object_transient_sysmeta, \ + update_etag_is_at_header, resolve_etag_is_at_header from test.unit import patch_policies from test.unit.common.test_utils import FakeResponse @@ -69,6 +69,14 @@ class TestRequestHelpers(unittest.TestCase): self.assertEqual(strip_user_meta_prefix(st, 'x-%s-%s-a' % (st, mt)), 'a') + def test_is_object_transient_sysmeta(self): + self.assertTrue(is_object_transient_sysmeta( + 'x-object-transient-sysmeta-foo')) + self.assertFalse(is_object_transient_sysmeta( + 'x-object-transient-sysmeta-')) + self.assertFalse(is_object_transient_sysmeta( + 'x-object-meatmeta-foo')) + def test_remove_items(self): src = {'a': 'b', 'c': 'd'} diff --git a/test/unit/obj/test_diskfile.py b/test/unit/obj/test_diskfile.py index 2a18478087..0a92f184f2 100644 --- a/test/unit/obj/test_diskfile.py +++ b/test/unit/obj/test_diskfile.py @@ -2374,6 +2374,7 @@ class DiskFileMixin(BaseDiskFileTestMixin): def test_disk_file_default_disallowed_metadata(self): # build an object with some meta (at t0+1s) orig_metadata = {'X-Object-Meta-Key1': 'Value1', + 'X-Object-Transient-Sysmeta-KeyA': 'ValueA', 'Content-Type': 'text/garbage'} df = self._get_open_disk_file(ts=self.ts().internal, extra_metadata=orig_metadata) @@ -2382,6 +2383,7 @@ class DiskFileMixin(BaseDiskFileTestMixin): # write some new metadata (fast POST, don't send orig meta, at t0+1) df = self._simple_get_diskfile() df.write_metadata({'X-Timestamp': self.ts().internal, + 'X-Object-Transient-Sysmeta-KeyB': 'ValueB', 'X-Object-Meta-Key2': 'Value2'}) df = self._simple_get_diskfile() with df.open(): @@ -2389,8 +2391,11 @@ class DiskFileMixin(BaseDiskFileTestMixin): self.assertEqual('text/garbage', df._metadata['Content-Type']) # original fast-post updateable keys are removed self.assertNotIn('X-Object-Meta-Key1', df._metadata) + self.assertNotIn('X-Object-Transient-Sysmeta-KeyA', df._metadata) # new fast-post updateable keys are added self.assertEqual('Value2', df._metadata['X-Object-Meta-Key2']) + self.assertEqual('ValueB', + df._metadata['X-Object-Transient-Sysmeta-KeyB']) def test_disk_file_preserves_sysmeta(self): # build an object with some meta (at t0) diff --git a/test/unit/obj/test_server.py b/test/unit/obj/test_server.py index a40d75c5a2..79fc1b32f4 100755 --- a/test/unit/obj/test_server.py +++ b/test/unit/obj/test_server.py @@ -1683,7 +1683,8 @@ class TestObjectController(unittest.TestCase): 'ETag': '1000d172764c9dbc3a5798a67ec5bb76', 'X-Object-Meta-1': 'One', 'X-Object-Sysmeta-1': 'One', - 'X-Object-Sysmeta-Two': 'Two'}) + 'X-Object-Sysmeta-Two': 'Two', + 'X-Object-Transient-Sysmeta-Foo': 'Bar'}) req.body = 'VERIFY SYSMETA' resp = req.get_response(self.object_controller) self.assertEqual(resp.status_int, 201) @@ -1702,7 +1703,8 @@ class TestObjectController(unittest.TestCase): 'name': '/a/c/o', 'X-Object-Meta-1': 'One', 'X-Object-Sysmeta-1': 'One', - 'X-Object-Sysmeta-Two': 'Two'}) + 'X-Object-Sysmeta-Two': 'Two', + 'X-Object-Transient-Sysmeta-Foo': 'Bar'}) def test_PUT_succeeds_with_later_POST(self): ts_iter = make_timestamp_iter() @@ -1875,6 +1877,62 @@ class TestObjectController(unittest.TestCase): resp = req.get_response(self.object_controller) check_response(resp) + def test_POST_transient_sysmeta(self): + # check that diskfile transient system meta is changed by a POST + timestamp1 = normalize_timestamp(time()) + req = Request.blank( + '/sda1/p/a/c/o', environ={'REQUEST_METHOD': 'PUT'}, + headers={'X-Timestamp': timestamp1, + 'Content-Type': 'text/plain', + 'ETag': '1000d172764c9dbc3a5798a67ec5bb76', + 'X-Object-Meta-1': 'One', + 'X-Object-Sysmeta-1': 'One', + 'X-Object-Transient-Sysmeta-Foo': 'Bar'}) + req.body = 'VERIFY SYSMETA' + resp = req.get_response(self.object_controller) + self.assertEqual(resp.status_int, 201) + + timestamp2 = normalize_timestamp(time()) + req = Request.blank( + '/sda1/p/a/c/o', environ={'REQUEST_METHOD': 'POST'}, + headers={'X-Timestamp': timestamp2, + 'X-Object-Meta-1': 'Not One', + 'X-Object-Sysmeta-1': 'Not One', + 'X-Object-Transient-Sysmeta-Foo': 'Not Bar'}) + resp = req.get_response(self.object_controller) + self.assertEqual(resp.status_int, 202) + + # original .data file metadata should be unchanged + objfile = os.path.join( + self.testdir, 'sda1', + storage_directory(diskfile.get_data_dir(0), 'p', + hash_path('a', 'c', 'o')), + timestamp1 + '.data') + self.assertTrue(os.path.isfile(objfile)) + self.assertEqual(open(objfile).read(), 'VERIFY SYSMETA') + self.assertDictEqual(diskfile.read_metadata(objfile), + {'X-Timestamp': timestamp1, + 'Content-Length': '14', + 'Content-Type': 'text/plain', + 'ETag': '1000d172764c9dbc3a5798a67ec5bb76', + 'name': '/a/c/o', + 'X-Object-Meta-1': 'One', + 'X-Object-Sysmeta-1': 'One', + 'X-Object-Transient-Sysmeta-Foo': 'Bar'}) + + # .meta file metadata should have only user meta items + metafile = os.path.join( + self.testdir, 'sda1', + storage_directory(diskfile.get_data_dir(0), 'p', + hash_path('a', 'c', 'o')), + timestamp2 + '.meta') + self.assertTrue(os.path.isfile(metafile)) + self.assertDictEqual(diskfile.read_metadata(metafile), + {'X-Timestamp': timestamp2, + 'name': '/a/c/o', + 'X-Object-Meta-1': 'Not One', + 'X-Object-Transient-Sysmeta-Foo': 'Not Bar'}) + def test_PUT_then_fetch_system_metadata(self): timestamp = normalize_timestamp(time()) req = Request.blank( @@ -1884,7 +1942,8 @@ class TestObjectController(unittest.TestCase): 'ETag': '1000d172764c9dbc3a5798a67ec5bb76', 'X-Object-Meta-1': 'One', 'X-Object-Sysmeta-1': 'One', - 'X-Object-Sysmeta-Two': 'Two'}) + 'X-Object-Sysmeta-Two': 'Two', + 'X-Object-Transient-Sysmeta-Foo': 'Bar'}) req.body = 'VERIFY SYSMETA' resp = req.get_response(self.object_controller) self.assertEqual(resp.status_int, 201) @@ -1903,6 +1962,8 @@ class TestObjectController(unittest.TestCase): self.assertEqual(resp.headers['x-object-meta-1'], 'One') self.assertEqual(resp.headers['x-object-sysmeta-1'], 'One') self.assertEqual(resp.headers['x-object-sysmeta-two'], 'Two') + self.assertEqual(resp.headers['x-object-transient-sysmeta-foo'], + 'Bar') req = Request.blank('/sda1/p/a/c/o', environ={'REQUEST_METHOD': 'HEAD'}) @@ -1921,9 +1982,13 @@ class TestObjectController(unittest.TestCase): headers={'X-Timestamp': timestamp, 'Content-Type': 'text/plain', 'ETag': '1000d172764c9dbc3a5798a67ec5bb76', + 'X-Object-Meta-0': 'deleted by post', + 'X-Object-Sysmeta-0': 'Zero', + 'X-Object-Transient-Sysmeta-0': 'deleted by post', 'X-Object-Meta-1': 'One', 'X-Object-Sysmeta-1': 'One', - 'X-Object-Sysmeta-Two': 'Two'}) + 'X-Object-Sysmeta-Two': 'Two', + 'X-Object-Transient-Sysmeta-Foo': 'Bar'}) req.body = 'VERIFY SYSMETA' resp = req.get_response(self.object_controller) self.assertEqual(resp.status_int, 201) @@ -1934,7 +1999,8 @@ class TestObjectController(unittest.TestCase): headers={'X-Timestamp': timestamp2, 'X-Object-Meta-1': 'Not One', 'X-Object-Sysmeta-1': 'Not One', - 'X-Object-Sysmeta-Two': 'Not Two'}) + 'X-Object-Sysmeta-Two': 'Not Two', + 'X-Object-Transient-Sysmeta-Foo': 'Not Bar'}) resp = req.get_response(self.object_controller) self.assertEqual(resp.status_int, 202) @@ -1951,8 +2017,13 @@ class TestObjectController(unittest.TestCase): self.assertEqual(resp.headers['etag'], '"1000d172764c9dbc3a5798a67ec5bb76"') self.assertEqual(resp.headers['x-object-meta-1'], 'Not One') + self.assertEqual(resp.headers['x-object-sysmeta-0'], 'Zero') self.assertEqual(resp.headers['x-object-sysmeta-1'], 'One') self.assertEqual(resp.headers['x-object-sysmeta-two'], 'Two') + self.assertEqual(resp.headers['x-object-transient-sysmeta-foo'], + 'Not Bar') + self.assertNotIn('x-object-meta-0', resp.headers) + self.assertNotIn('x-object-transient-sysmeta-0', resp.headers) req = Request.blank('/sda1/p/a/c/o', environ={'REQUEST_METHOD': 'HEAD'}) diff --git a/test/unit/proxy/controllers/test_base.py b/test/unit/proxy/controllers/test_base.py index 689c6c88a8..55214f6d03 100644 --- a/test/unit/proxy/controllers/test_base.py +++ b/test/unit/proxy/controllers/test_base.py @@ -29,7 +29,9 @@ from swift.common.http import is_success from swift.common.storage_policy import StoragePolicy from test.unit import fake_http_connect, FakeRing, FakeMemcache from swift.proxy import server as proxy_server -from swift.common.request_helpers import get_sys_meta_prefix +from swift.common.request_helpers import ( + get_sys_meta_prefix, get_object_transient_sysmeta +) from test.unit import patch_policies @@ -537,6 +539,14 @@ class TestFuncs(unittest.TestCase): self.assertEqual(resp['sysmeta']['whatevs'], 14) self.assertEqual(resp['sysmeta']['somethingelse'], 0) + def test_headers_to_object_info_transient_sysmeta(self): + headers = {get_object_transient_sysmeta('Whatevs'): 14, + get_object_transient_sysmeta('somethingelse'): 0} + resp = headers_to_object_info(headers.items(), 200) + self.assertEqual(len(resp['transient_sysmeta']), 2) + self.assertEqual(resp['transient_sysmeta']['whatevs'], 14) + self.assertEqual(resp['transient_sysmeta']['somethingelse'], 0) + def test_headers_to_object_info_values(self): headers = { 'content-length': '1024', diff --git a/test/unit/proxy/test_server.py b/test/unit/proxy/test_server.py index f43ca5778e..6452fb5b0c 100644 --- a/test/unit/proxy/test_server.py +++ b/test/unit/proxy/test_server.py @@ -53,7 +53,7 @@ from swift.common.utils import hash_path, storage_directory, \ iter_multipart_mime_documents, public from test.unit import ( - connect_tcp, readuntil2crlfs, FakeLogger, FakeRing, fake_http_connect, + connect_tcp, readuntil2crlfs, FakeLogger, fake_http_connect, FakeRing, FakeMemcache, debug_logger, patch_policies, write_fake_ring, mocked_http_conn, DEFAULT_TEST_EC_TYPE) from swift.proxy import server as proxy_server diff --git a/test/unit/proxy/test_sysmeta.py b/test/unit/proxy/test_sysmeta.py index 1a7f82334e..eb58523e39 100644 --- a/test/unit/proxy/test_sysmeta.py +++ b/test/unit/proxy/test_sysmeta.py @@ -28,6 +28,7 @@ from swift.common.wsgi import monkey_patch_mimetools, WSGIContext from swift.obj import server as object_server from swift.proxy import server as proxy import swift.proxy.controllers +from swift.proxy.controllers.base import get_object_info from test.unit import FakeMemcache, debug_logger, FakeRing, \ fake_http_connect, patch_policies @@ -172,6 +173,17 @@ class TestObjectSysmeta(unittest.TestCase): 'x-object-meta-test1': 'meta1 changed'} new_meta_headers = {'x-object-meta-test3': 'meta3'} bad_headers = {'x-account-sysmeta-test1': 'bad1'} + # these transient_sysmeta headers get changed... + original_transient_sysmeta_headers_1 = \ + {'x-object-transient-sysmeta-testA': 'A'} + # these transient_sysmeta headers get deleted... + original_transient_sysmeta_headers_2 = \ + {'x-object-transient-sysmeta-testB': 'B'} + # these are replacement transient_sysmeta headers + changed_transient_sysmeta_headers = \ + {'x-object-transient-sysmeta-testA': 'changed_A'} + new_transient_sysmeta_headers_1 = {'x-object-transient-sysmeta-testC': 'C'} + new_transient_sysmeta_headers_2 = {'x-object-transient-sysmeta-testD': 'D'} def test_PUT_sysmeta_then_GET(self): path = '/v1/a/c/o' @@ -180,6 +192,7 @@ class TestObjectSysmeta(unittest.TestCase): hdrs = dict(self.original_sysmeta_headers_1) hdrs.update(self.original_meta_headers_1) hdrs.update(self.bad_headers) + hdrs.update(self.original_transient_sysmeta_headers_1) req = Request.blank(path, environ=env, headers=hdrs, body='x') resp = req.get_response(self.app) self._assertStatus(resp, 201) @@ -189,6 +202,7 @@ class TestObjectSysmeta(unittest.TestCase): self._assertStatus(resp, 200) self._assertInHeaders(resp, self.original_sysmeta_headers_1) self._assertInHeaders(resp, self.original_meta_headers_1) + self._assertInHeaders(resp, self.original_transient_sysmeta_headers_1) self._assertNotInHeaders(resp, self.bad_headers) def test_PUT_sysmeta_then_HEAD(self): @@ -198,6 +212,7 @@ class TestObjectSysmeta(unittest.TestCase): hdrs = dict(self.original_sysmeta_headers_1) hdrs.update(self.original_meta_headers_1) hdrs.update(self.bad_headers) + hdrs.update(self.original_transient_sysmeta_headers_1) req = Request.blank(path, environ=env, headers=hdrs, body='x') resp = req.get_response(self.app) self._assertStatus(resp, 201) @@ -208,6 +223,7 @@ class TestObjectSysmeta(unittest.TestCase): self._assertStatus(resp, 200) self._assertInHeaders(resp, self.original_sysmeta_headers_1) self._assertInHeaders(resp, self.original_meta_headers_1) + self._assertInHeaders(resp, self.original_transient_sysmeta_headers_1) self._assertNotInHeaders(resp, self.bad_headers) def test_sysmeta_replaced_by_PUT(self): @@ -306,6 +322,8 @@ class TestObjectSysmeta(unittest.TestCase): hdrs.update(self.original_sysmeta_headers_2) hdrs.update(self.original_meta_headers_1) hdrs.update(self.original_meta_headers_2) + hdrs.update(self.original_transient_sysmeta_headers_1) + hdrs.update(self.original_transient_sysmeta_headers_2) req = Request.blank(path, environ=env, headers=hdrs, body='x') resp = req.get_response(self.copy_app) self._assertStatus(resp, 201) @@ -315,6 +333,8 @@ class TestObjectSysmeta(unittest.TestCase): hdrs.update(self.new_sysmeta_headers) hdrs.update(self.changed_meta_headers) hdrs.update(self.new_meta_headers) + hdrs.update(self.changed_transient_sysmeta_headers) + hdrs.update(self.new_transient_sysmeta_headers_1) hdrs.update(self.bad_headers) hdrs.update({'Destination': dest}) req = Request.blank(path, environ=env, headers=hdrs) @@ -326,6 +346,9 @@ class TestObjectSysmeta(unittest.TestCase): self._assertInHeaders(resp, self.changed_meta_headers) self._assertInHeaders(resp, self.new_meta_headers) self._assertInHeaders(resp, self.original_meta_headers_2) + self._assertInHeaders(resp, self.changed_transient_sysmeta_headers) + self._assertInHeaders(resp, self.new_transient_sysmeta_headers_1) + self._assertInHeaders(resp, self.original_transient_sysmeta_headers_2) self._assertNotInHeaders(resp, self.bad_headers) req = Request.blank('/v1/a/c/o2', environ={}) @@ -337,6 +360,9 @@ class TestObjectSysmeta(unittest.TestCase): self._assertInHeaders(resp, self.changed_meta_headers) self._assertInHeaders(resp, self.new_meta_headers) self._assertInHeaders(resp, self.original_meta_headers_2) + self._assertInHeaders(resp, self.changed_transient_sysmeta_headers) + self._assertInHeaders(resp, self.new_transient_sysmeta_headers_1) + self._assertInHeaders(resp, self.original_transient_sysmeta_headers_2) self._assertNotInHeaders(resp, self.bad_headers) def test_sysmeta_updated_by_COPY_from(self): @@ -380,3 +406,84 @@ class TestObjectSysmeta(unittest.TestCase): self._assertInHeaders(resp, self.new_meta_headers) self._assertInHeaders(resp, self.original_meta_headers_2) self._assertNotInHeaders(resp, self.bad_headers) + + def _test_transient_sysmeta_replaced_by_PUT_or_POST(self, app): + # check transient_sysmeta is replaced en-masse by a POST + path = '/v1/a/c/o' + + env = {'REQUEST_METHOD': 'PUT'} + hdrs = dict(self.original_transient_sysmeta_headers_1) + hdrs.update(self.original_transient_sysmeta_headers_2) + hdrs.update(self.original_meta_headers_1) + req = Request.blank(path, environ=env, headers=hdrs, body='x') + resp = req.get_response(app) + self._assertStatus(resp, 201) + + req = Request.blank(path, environ={}) + resp = req.get_response(app) + self._assertStatus(resp, 200) + self._assertInHeaders(resp, self.original_transient_sysmeta_headers_1) + self._assertInHeaders(resp, self.original_transient_sysmeta_headers_2) + self._assertInHeaders(resp, self.original_meta_headers_1) + + info = get_object_info(req.environ, app) + self.assertEqual(2, len(info.get('transient_sysmeta', ()))) + self.assertEqual({'testa': 'A', 'testb': 'B'}, + info['transient_sysmeta']) + + # POST will replace all existing transient_sysmeta and usermeta values + env = {'REQUEST_METHOD': 'POST'} + hdrs = dict(self.changed_transient_sysmeta_headers) + hdrs.update(self.new_transient_sysmeta_headers_1) + req = Request.blank(path, environ=env, headers=hdrs) + resp = req.get_response(app) + self._assertStatus(resp, 202) + + req = Request.blank(path, environ={}) + resp = req.get_response(app) + self._assertStatus(resp, 200) + self._assertInHeaders(resp, self.changed_transient_sysmeta_headers) + self._assertInHeaders(resp, self.new_transient_sysmeta_headers_1) + self._assertNotInHeaders(resp, self.original_meta_headers_1) + self._assertNotInHeaders(resp, + self.original_transient_sysmeta_headers_2) + + info = get_object_info(req.environ, app) + self.assertEqual(2, len(info.get('transient_sysmeta', ()))) + self.assertEqual({'testa': 'changed_A', 'testc': 'C'}, + info['transient_sysmeta']) + + # subsequent PUT replaces all transient_sysmeta and usermeta values + env = {'REQUEST_METHOD': 'PUT'} + hdrs = dict(self.new_transient_sysmeta_headers_2) + hdrs.update(self.original_meta_headers_2) + req = Request.blank(path, environ=env, headers=hdrs, body='x') + resp = req.get_response(app) + self._assertStatus(resp, 201) + + req = Request.blank(path, environ={}) + resp = req.get_response(app) + self._assertStatus(resp, 200) + self._assertInHeaders(resp, self.original_meta_headers_2) + self._assertInHeaders(resp, self.new_transient_sysmeta_headers_2) + # meta from previous POST should have gone away... + self._assertNotInHeaders(resp, self.changed_transient_sysmeta_headers) + self._assertNotInHeaders(resp, self.new_transient_sysmeta_headers_1) + # sanity check that meta from first PUT did not re-appear... + self._assertNotInHeaders(resp, self.original_meta_headers_1) + self._assertNotInHeaders(resp, + self.original_transient_sysmeta_headers_1) + self._assertNotInHeaders(resp, + self.original_transient_sysmeta_headers_2) + + info = get_object_info(req.environ, app) + self.assertEqual(1, len(info.get('transient_sysmeta', ()))) + self.assertEqual({'testd': 'D'}, info['transient_sysmeta']) + + def test_transient_sysmeta_replaced_by_PUT_or_POST(self): + self._test_transient_sysmeta_replaced_by_PUT_or_POST(self.app) + + def test_transient_sysmeta_replaced_by_PUT_or_POST_as_copy(self): + # test post-as-copy by issuing requests to the copy middleware app + self.copy_app.object_post_as_copy = True + self._test_transient_sysmeta_replaced_by_PUT_or_POST(self.copy_app)