Metacat

use SSL to get content from stream

http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5527

use HttpClient to set up SSL connection when doing replication calls -- this will use the server's configured certificate as the client certificate on the request. The server it is calling can then inspect that certificate and decide whether or not it trusts the caller.

add User-Agent logging to support D1 requirements

include SystemMetadata when replicating data and metadata documents -- this allows us to establish the guid-to-docid mapping that is crucial for being able to read the replicated document by guid (d1 api)

do not send <systemMetadata> with the <docInfo> replication information - this is handled by the Hazelcast shared map

rely on Hazelcast to store the SystemMetadata locally for the node. Entry event listeners store the shared system metadata on their local node when alerted. TODO: remove old replication code that included system metadata xml when replicating scimeta and data

remove ServiceTypeUtil - replace with TypeMarshaller

use new "v1" types from DataONE

add option for replicating system metadata (dataone)
https://redmine.dataone.org/issues/1626

Merged in the D1_0_6_2_BRANCH changes that include the transition from ObjectFormat calls to ObjectFormatCache calls.

include System Metadata forced replication - just need to figure out when to call it!

handle timed replication of system metadata. there are still a few outstanding issues:
-track server location of system metadata-only entries
-replication policy flag for system metadata-only entries?
-locking for replicated entries?
-forced replication of entries

transfer full System Metadata (as XML) during document and data replication

-remove system metadata guid -> local id mapping (there is no document for system metadata now)
-include system metadata elements when replicating data objects (TODO: transfer all system metadata structures with the docinfo request).
TODO: remove docid+rev from the systemMetadata table definition

do not use XML files for storing SystemMetadata - use DB tables only.

use update method to update the mapping between local and guid (d1) when we get a force replication request that is an "update

use "object_format" element consistently so that it is replicated across instances
https://redmine.dataone.org/issues/1514

insert/update documents with null user and null group to circumvent access control restrictions then update the user_owner and user_updated values to reflect what exists on the originating server (pisco)

use 'user_updated' field when writing the replicated document - allows most recent ownership/permissions to be used (in case LDAP groups have shifted) and is more accurate for both updates and initial inserts (hopefully addresses the replication issue we are having with pisco)

DocumentImpl.delete() now throws finer grained exceptions (not a general exception). Consequently, the classes that call it have been updated to handle the thrown exceptions, including CrudService, ReplicationHandler, and ReplicationService.

adding more debuggin and fixing bug with systemmetadata

fixed replication bug where systemmetadata was not getting procssed correctly

added code to do database query for listObjects

only call response.getWriter() when we are about to send text/xml to the client, otherwise we end of calling both getWriter() and getOutputStream() - resulting in an illegal state.

use detected XML encoding when reading/writing files
use UTF-8 as default when performing queries in the DB (assume DB is using UTF-8)
remove as many PrintWriters (uses system default character encoding only) as possible and construct OutputStreamWriters where explicit encoding can be given....

add support for EML 2.1.1

reformatting logs for robert

added another logging statement

hopefully fixed bug with systemmetadata replication

debug statements in dbsaxhandler

fixing problems with replication and systemmetadata

added functionality to set access permissions to system metadata the same as the document that it describes

fixed major bug in replication where the document info was being truncated due to a poorly implemented sax parser

replication of guids now works. tested this for both forced replication and update/insert/delete triggered replication

fixed bug where guid end tag wasn't getting printed

added a method in IdentifierManager to get a guid from a docid and rev. added fields in the documentinfo replication document to pass the guid. now need to handle the guid and insert it into the table if its found

Modifications to support the DataONE service API version 0.1.0. For DataONE, the get() and
create() services are partially complete. Several more functions and checks need to be added to
create() before it is viable. This DataONE support is not complete, and the current support breaks the MetacatRestClientTest for the time being (this client will eventually be removed).

Pass the doc xml as a string to docImpl.write and writeRepication. This is so a reader can be create for the parsing and for the write to disk. Also created a db access class for xml query result deletion.

Log doc and rev query counts and times. Fix mis-spellings.

Change add sql to use a prepared statement. Only try to download a cert if a url was provided.

change AccessControlForSingleFile to only be instantiated for one file. move ACL methods to AccessControlForSingleFile. Change format of access sections returned to EML 2.1.0.

Move access control source to it's own directory.

Change location of PropertyService to properties directory

Change MetaCatVersion to MetacatVersion

Create replication directory. Move replication code there. Use log4j for replication logging (rollingfileappender). Beef up replication logging and error control.

Roll back replication user changes. Fix code that converts access levels to integer and to text.

Introduce replication user. Use the fileutil writer methods instead of writing directly.

Beef up exception handling from file utilities. Move UtilException to MetacatUtilException to eliminate conflict with similar exception in utility package.

Replace System.out.println statements with logMetacat statements.

Update replication documentation and fix code so that replication log is available.

Renamed MetaCatUtil to MetacatUtil

Rename LDAPUtil to AuthUtil

Add some generic typing

check for existing access control rows before inserting (we do not need duplicate rows specifying the same access control)

in cases when only access control is being modified, we can temporarily ignore the exception thrown my the write method, insert the updated access control rules, then deal with the write exception. this occurs when the docid remains the same (no revision increase) but the permissions have been altered.

use correct log directory/file for replication errors

replicate xml_access entries across servers.
see: http://bugzilla.ecoinformatics.org/show_bug.cgi?id=3464
note: still needs to be tested...but needed to be checked in for other metacat installations to be updated....

Move the DBAdaptor accessor into a DatabaseService class

qualify xml and eml properties with an xml. prefix

Fully qualify spatial and replication properties

Use qualified properties

Use SystemUtil.getContextDir() instead of constructing directory path.

Differentiate between tomcat directory and deployment directory.

Append context url onto system id instead of server url.

Add addProperty and addPropertyNoPersist for new properties

Merge 1.9 changes into Head

Add even log in replication.

Somehow the change went to the head rather than branch. So i rollback the change in head.

This commit is for branch. In this commit the correct ip and user name will be stored in access_log table in replication event. However, it only for Forcereplication. And the test isn't completed yet. This commit is only for future use.

Removed the release ant token from all files in 'src'.

Using join query for more efficiency.

Fixed bug to get docid list.

Add more info transfer.

Replacing MetaCatUtil.debugMessage or MetaCatUtil.logMetacat call with logMetacat (private Logger object) call

Add rev in get data doc info.

Add date info.

Fixed the bug in xml segment is not correct to express revision document list.

Add code to handle deleted document replication.

Add code to get revision table info

Add code to check the if the user has permission to start or stop timed replication.

Use a util function to get replication server.

Persistant timed replication parameter.

Add new feature that delete can be broadcasted by force replication.

Add a code to load eml201 parser to fix the bug that couldn't generate access rule for eml201 doc.

Made initial changes for accepting EML version 2.1.0 in Metacat

Change eml2 to eml200

Fixed bug in replication if doctype is null.

Fixed a bug that replication didn't do inline data.

Change the year from YY to YYYY

Revise initial value for server registry.

Change format from yy-mm-dd to mm/dd/yy.

Get rid of to_date function.

Revise getURLContent method and give up httpclient.

Using httpclient as http and https protocol handler.

Revise update method. Every write action will check out a db connection. So it can handle a huge mount of write action.

Add code to handle initialize eml2 parser.

Change debugMessage

Add code to handle text code was splited.

Merge cvs branch replication to cvs head.