Project

General

Profile

Statistics
| Revision:

metacat / src / edu / ucsb / nceas / metacat / replication @ 7370

# Date Author Comment
7370 09/04/2012 03:43 PM ben leinfelder

use SchemaLocationResolver to fetch remote entries for the xml_catalog -- we want to be able to fetch included xsd files as well as use any error handling it provides for checking the schemas.

7358 08/23/2012 09:45 PM ben leinfelder

create docid-guid mapping during replication if it does not exist. we were [incorrectly] assuming that there would be SM coming with the document info that would fill this information in, but for traditional non-MN Metacat deployments there is no SM to provide a mapping. In this case we use the docid as the guid.

7356 08/17/2012 12:42 PM ben leinfelder

stream the replication "update" response rather than building up a complete list in a stringbuffer. prompted by findings on t he CN: https://redmine.dataone.org/issues/3141

7348 08/06/2012 11:08 AM ben leinfelder

remove unused "dataonelogger"

7338 08/01/2012 07:00 PM ben leinfelder

remove possibility for infinite loop in case data replication is not configured for the server and a data file is encountered (yikes!)

7337 08/01/2012 05:33 PM ben leinfelder

added logging debug statements to see where the replication timeout might be occurring.

7317 07/17/2012 12:06 PM ben leinfelder

only look up the client timeout property once, not every time we make a call
https://redmine.dataone.org/issues/3078

7313 07/17/2012 10:19 AM ben leinfelder

configurable replication client timeout
https://redmine.dataone.org/issues/3078

7196 05/29/2012 09:33 AM ben leinfelder

stack trace the HZ put exception during CN-CN replication

7195 05/29/2012 07:37 AM ben leinfelder

additional debugging statements for CONCURRENT_MAP_PUT error during CN-CN replication.

7175 05/18/2012 12:09 PM ben leinfelder

instead of generating SM and ORE maps during dataone configuration/MN registration, moved this all to the replication admin screen where we can target generation for specific nodes. That way it's more controlled as to when and where we generate DataONE required content....

7170 05/14/2012 02:36 PM ben leinfelder

add "Generate System Metadata" button to the replication server list display. When clicked, we generate SM for records belonging to that source server. This is only enabled when DataONE has been configured.
https://redmine.dataone.org/issues/2762

7150 04/30/2012 04:03 PM ben leinfelder

optionally remove the document/data file from the filesystem completely when 'deleting' it.
https://redmine.dataone.org/issues/2677

7128 04/09/2012 03:18 PM ben leinfelder

add a parameter for optionally writing EML-embedded access control rules to the Metacat DB.
https://redmine.dataone.org/issues/2584
https://redmine.dataone.org/issues/2583

7120 04/05/2012 04:28 PM ben leinfelder

band-aid for CN-CN replication permOrder issue when access control is embedded in EML and the system metadata is replicated before the EML. we just log the inconsistency and allow the insert to succeed
https://redmine.dataone.org/issues/2583

7045 03/07/2012 02:30 PM ben leinfelder

check whether mapping (

6968 02/01/2012 09:44 AM ben leinfelder

process systemMetadata from the docInfo string before writing to the database so that we guarantee guid-docid mapping exists before attempting to look it up.

6814 12/21/2011 01:00 PM ben leinfelder

remove flag for independent system metadata replication -- these entries are replicated along with the data/metadata objects or via hazelcast when the actual object is not on the server.

6783 12/13/2011 05:06 PM ben leinfelder

only create guid->docid mapping during metadata replication if it does not already exist
http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5520

6782 12/13/2011 04:50 PM ben leinfelder

do not treat access change as an update -- it should not attempt to retrieve the contents of the object
http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5520

6780 12/13/2011 04:18 PM ben leinfelder

only create guid->docid mapping during data replication if it does not already exist
http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5520

6775 12/13/2011 11:34 AM ben leinfelder

process system metadata before access rules (access control is now driven by GUID so the mapping needs to be there)

6708 11/29/2011 01:05 PM ben leinfelder

use shared method for looking up "docInfo" map -- both in Metacat replication and in D1 system metadata generation

6672 11/17/2011 02:47 PM ben leinfelder

replication control panel now fully implemented as an admin configuration screen
http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5528

6669 11/17/2011 12:19 PM ben leinfelder

move replication configuration actions to the admin servlet and out of the replication servlet
http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5528

6666 11/17/2011 11:19 AM ben leinfelder

save SystemMetadata when replicating data and metadata -- this way if/when the node decides to be a DataONE MN it already has the information needed for each object

6628 11/09/2011 04:26 PM ben leinfelder

get server param only when it is expected

6627 11/09/2011 03:34 PM ben leinfelder

check replication table (not keystore) for trusted server host name match

6620 11/08/2011 11:31 AM ben leinfelder

started replication unit test

6616 11/07/2011 03:53 PM ben leinfelder

add note about alternative methods for getting cert/key

6614 11/07/2011 02:31 PM ben leinfelder

use DateTimeMarshaller for all replication date transfers

6613 11/07/2011 12:31 PM ben leinfelder

print the stacktrace when there is an error -- debuggin!

6611 11/07/2011 10:55 AM ben leinfelder

use SSL to get content from stream

6595 11/02/2011 08:40 PM ben leinfelder

http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5527

6587 11/01/2011 07:35 PM ben leinfelder

skip verification -- remaining TODO

6586 11/01/2011 07:08 PM ben leinfelder

verify certificate

6585 11/01/2011 06:41 PM ben leinfelder

inspect keystore entries for matching client certificate

6584 11/01/2011 05:27 PM ben leinfelder

lookup the correct property for keystore file

6582 11/01/2011 03:23 PM ben leinfelder

use HttpClient to set up SSL connection when doing replication calls -- this will use the server's configured certificate as the client certificate on the request. The server it is calling can then inspect that certificate and decide whether or not it trusts the caller.

6579 11/01/2011 01:33 PM ben leinfelder

check client-provided certificate when servicing ReplicationServlet requests.

6542 10/20/2011 02:03 PM ben leinfelder

add User-Agent logging to support D1 requirements

6531 10/12/2011 11:08 AM ben leinfelder

include SystemMetadata when replicating data and metadata documents -- this allows us to establish the guid-to-docid mapping that is crucial for being able to read the replicated document by guid (d1 api)

6448 09/16/2011 03:17 PM ben leinfelder

do not send <systemMetadata> with the <docInfo> replication information - this is handled by the Hazelcast shared map

6447 09/16/2011 03:07 PM ben leinfelder

rely on Hazelcast to store the SystemMetadata locally for the node. Entry event listeners store the shared system metadata on their local node when alerted. TODO: remove old replication code that included system metadata xml when replicating scimeta and data

6367 07/28/2011 10:10 AM ben leinfelder

remove ServiceTypeUtil - replace with TypeMarshaller

6366 07/27/2011 04:25 PM ben leinfelder

use new "v1" types from DataONE

6335 07/11/2011 12:19 PM ben leinfelder

use correct log name for the class

6135 06/08/2011 05:08 PM ben leinfelder

add option for replicating system metadata (dataone)
https://redmine.dataone.org/issues/1626

6125 06/07/2011 10:27 AM ben leinfelder

force replication for newly-registered system metadata

6124 06/07/2011 09:53 AM Chris Jones

Merged in the D1_0_6_2_BRANCH changes that include the transition from ObjectFormat calls to ObjectFormatCache calls.

6119 06/02/2011 04:40 PM ben leinfelder

include System Metadata forced replication - just need to figure out when to call it!

6118 06/01/2011 01:45 PM ben leinfelder

handle timed replication of system metadata. there are still a few outstanding issues:
-track server location of system metadata-only entries
-replication policy flag for system metadata-only entries?
-locking for replicated entries?
-forced replication of entries

6102 05/25/2011 03:53 PM ben leinfelder

transfer full System Metadata (as XML) during document and data replication

6099 05/25/2011 11:59 AM ben leinfelder

-remove system metadata guid -> local id mapping (there is no document for system metadata now)
-include system metadata elements when replicating data objects (TODO: transfer all system metadata structures with the docinfo request).
TODO: remove docid+rev from the systemMetadata table definition

6097 05/24/2011 04:18 PM ben leinfelder

do not use XML files for storing SystemMetadata - use DB tables only.

6079 05/05/2011 03:14 PM ben leinfelder

use update method to update the mapping between local and guid (d1) when we get a force replication request that is an "update

6057 05/03/2011 04:04 PM ben leinfelder

use "object_format" element consistently so that it is replicated across instances
https://redmine.dataone.org/issues/1514

6015 03/22/2011 05:19 PM ben leinfelder

insert/update documents with null user and null group to circumvent access control restrictions then update the user_owner and user_updated values to reflect what exists on the originating server (pisco)

6014 03/22/2011 01:24 PM ben leinfelder

use 'user_updated' field when writing the replicated document - allows most recent ownership/permissions to be used (in case LDAP groups have shifted) and is more accurate for both updates and initial inserts (hopefully addresses the replication issue we are having with pisco)

6001 03/02/2011 02:12 PM Chris Jones

DocumentImpl.delete() now throws finer grained exceptions (not a general exception). Consequently, the classes that call it have been updated to handle the thrown exceptions, including CrudService, ReplicationHandler, and ReplicationService.

5950 02/11/2011 01:48 PM berkley

adding more debuggin and fixing bug with systemmetadata

5944 02/10/2011 04:17 PM berkley

fixed replication bug where systemmetadata was not getting procssed correctly

5930 02/08/2011 12:05 PM berkley

fixed typo that prevented replication

5917 02/04/2011 12:39 PM berkley

fixed bugs in listObjects

5895 02/03/2011 03:25 PM berkley

added code to do database query for listObjects

5755 12/23/2010 09:10 AM ben leinfelder

only call response.getWriter() when we are about to send text/xml to the client, otherwise we end of calling both getWriter() and getOutputStream() - resulting in an illegal state.

5752 12/21/2010 02:26 PM ben leinfelder

use detected XML encoding when reading/writing files
use UTF-8 as default when performing queries in the DB (assume DB is using UTF-8)
remove as many PrintWriters (uses system default character encoding only) as possible and construct OutputStreamWriters where explicit encoding can be given....

5709 12/08/2010 04:59 PM ben leinfelder

add support for EML 2.1.1

5459 07/28/2010 10:00 AM berkley

reformatting logs for robert

5458 07/27/2010 11:25 AM berkley

added another logging statement

5457 07/27/2010 11:04 AM berkley

added replicate log statements with the guid and localId

5455 07/26/2010 08:21 PM berkley

hopefully fixed bug with systemmetadata replication

5454 07/26/2010 08:04 PM berkley

debug statements in dbsaxhandler

5451 07/26/2010 02:05 PM berkley

fixing problems with replication and systemmetadata

5441 07/22/2010 03:30 PM berkley

added functionality to set access permissions to system metadata the same as the document that it describes

5440 07/22/2010 12:32 PM berkley

fixed major bug in replication where the document info was being truncated due to a poorly implemented sax parser

5392 06/15/2010 04:13 PM berkley

added a DataOneLogger for event notifications on the CN. The logger is called DataOneLogger and can be managed in the log4j.properties file

5374 06/08/2010 12:34 PM berkley

refactored the sessionService to use a correct singleton initialization scheme. Added true authentication to ResourceHandler.

5324 04/26/2010 04:11 PM berkley

replication of guids now works. tested this for both forced replication and update/insert/delete triggered replication

5323 04/26/2010 02:25 PM berkley

fixed bug where guid end tag wasn't getting printed

5322 04/26/2010 12:29 PM berkley

added a method in IdentifierManager to get a guid from a docid and rev. added fields in the documentinfo replication document to pass the guid. now need to handle the guid and insert it into the table if its found

5319 04/19/2010 04:51 PM Matt Jones

Modifications to support the DataONE service API version 0.1.0. For DataONE, the get() and
create() services are partially complete. Several more functions and checks need to be added to
create() before it is viable. This DataONE support is not complete, and the current support breaks the MetacatRestClientTest for the time being (this client will eventually be removed).

5195 01/19/2010 10:25 AM daigle

Pass the doc xml as a string to docImpl.write and writeRepication. This is so a reader can be create for the parsing and for the write to disk. Also created a db access class for xml query result deletion.

5175 01/04/2010 09:01 AM daigle

Log doc and rev query counts and times. Fix mis-spellings.

5174 01/04/2010 09:00 AM daigle

Log doc and rev query counts and times. Fix mis-spellings.

5151 12/02/2009 04:00 PM daigle

Change add sql to use a prepared statement. Only try to download a cert if a url was provided.

5098 10/30/2009 03:39 PM daigle

change AccessControlForSingleFile to only be instantiated for one file. move ACL methods to AccessControlForSingleFile. Change format of access sections returned to EML 2.1.0.

5089 10/16/2009 11:10 AM daigle

Move access control source to it's own directory.

5070 10/02/2009 12:59 PM daigle

Add user name to SessionData object

5030 08/24/2009 02:34 PM daigle

Change location of PropertyService to properties directory

5027 08/14/2009 02:26 PM daigle

Change MetaCatVersion to MetacatVersion

5014 08/04/2009 02:30 PM daigle

Create replication directory. Move replication code there. Use log4j for replication logging (rollingfileappender). Beef up replication logging and error control.