During the replication, the remote content will be saved without alteration.
add support for v2 DataONE API.
Index the document after document is written to the db.
recursively submit obsoleted objects for indexing when instructed. https://projects.ecoinformatics.org/ecoinfo/issues/6424
Unify solr indexing with an IndexTask that is added to the queue -- allows us to send more than just the systemMetadata to the indexer. Initially this is for READ event counts for each document. https://projects.ecoinformatics.org/ecoinfo/issues/6346
use an independent ISet<SystemMetadata> structure to communicate objects that should be indexed by metacat-index. https://projects.ecoinformatics.org/ecoinfo/issues/5943
move DocInfo parsing into utilities project so that it can be used by Morpho as well as Metacat.http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5737
use SchemaLocationResolver to fetch remote entries for the xml_catalog -- we want to be able to fetch included xsd files as well as use any error handling it provides for checking the schemas.
create docid-guid mapping during replication if it does not exist. we were [incorrectly] assuming that there would be SM coming with the document info that would fill this information in, but for traditional non-MN Metacat deployments there is no SM to provide a mapping. In this case we use the docid as the guid.
stream the replication "update" response rather than building up a complete list in a stringbuffer. prompted by findings on t he CN: https://redmine.dataone.org/issues/3141
remove unused "dataonelogger"
stack trace the HZ put exception during CN-CN replication
additional debugging statements for CONCURRENT_MAP_PUT error during CN-CN replication.
optionally remove the document/data file from the filesystem completely when 'deleting' it.https://redmine.dataone.org/issues/2677
add a parameter for optionally writing EML-embedded access control rules to the Metacat DB.https://redmine.dataone.org/issues/2584https://redmine.dataone.org/issues/2583
check whether mapping (
process systemMetadata from the docInfo string before writing to the database so that we guarantee guid-docid mapping exists before attempting to look it up.
save SystemMetadata when replicating data and metadata -- this way if/when the node decides to be a DataONE MN it already has the information needed for each object
use DateTimeMarshaller for all replication date transfers
use SSL to get content from stream
http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5527
add User-Agent logging to support D1 requirements
include SystemMetadata when replicating data and metadata documents -- this allows us to establish the guid-to-docid mapping that is crucial for being able to read the replicated document by guid (d1 api)
do not send <systemMetadata> with the <docInfo> replication information - this is handled by the Hazelcast shared map
rely on Hazelcast to store the SystemMetadata locally for the node. Entry event listeners store the shared system metadata on their local node when alerted. TODO: remove old replication code that included system metadata xml when replicating scimeta and data
remove ServiceTypeUtil - replace with TypeMarshaller
use new "v1" types from DataONE
Merged in the D1_0_6_2_BRANCH changes that include the transition from ObjectFormat calls to ObjectFormatCache calls.
handle timed replication of system metadata. there are still a few outstanding issues: -track server location of system metadata-only entries-replication policy flag for system metadata-only entries?-locking for replicated entries?-forced replication of entries
transfer full System Metadata (as XML) during document and data replication
-remove system metadata guid -> local id mapping (there is no document for system metadata now)-include system metadata elements when replicating data objects (TODO: transfer all system metadata structures with the docinfo request).TODO: remove docid+rev from the systemMetadata table definition
do not use XML files for storing SystemMetadata - use DB tables only.
use update method to update the mapping between local and guid (d1) when we get a force replication request that is an "update
use "object_format" element consistently so that it is replicated across instanceshttps://redmine.dataone.org/issues/1514
insert/update documents with null user and null group to circumvent access control restrictions then update the user_owner and user_updated values to reflect what exists on the originating server (pisco)
use 'user_updated' field when writing the replicated document - allows most recent ownership/permissions to be used (in case LDAP groups have shifted) and is more accurate for both updates and initial inserts (hopefully addresses the replication issue we are having with pisco)
DocumentImpl.delete() now throws finer grained exceptions (not a general exception). Consequently, the classes that call it have been updated to handle the thrown exceptions, including CrudService, ReplicationHandler, and ReplicationService.
adding more debuggin and fixing bug with systemmetadata
fixed replication bug where systemmetadata was not getting procssed correctly
fixed typo that prevented replication
fixed bugs in listObjects
added code to do database query for listObjects
add support for EML 2.1.1
reformatting logs for robert
added replicate log statements with the guid and localId
fixing problems with replication and systemmetadata
fixed major bug in replication where the document info was being truncated due to a poorly implemented sax parser
added a DataOneLogger for event notifications on the CN. The logger is called DataOneLogger and can be managed in the log4j.properties file
replication of guids now works. tested this for both forced replication and update/insert/delete triggered replication
Modifications to support the DataONE service API version 0.1.0. For DataONE, the get() andcreate() services are partially complete. Several more functions and checks need to be added tocreate() before it is viable. This DataONE support is not complete, and the current support breaks the MetacatRestClientTest for the time being (this client will eventually be removed).
Pass the doc xml as a string to docImpl.write and writeRepication. This is so a reader can be create for the parsing and for the write to disk. Also created a db access class for xml query result deletion.
Log doc and rev query counts and times. Fix mis-spellings.
change AccessControlForSingleFile to only be instantiated for one file. move ACL methods to AccessControlForSingleFile. Change format of access sections returned to EML 2.1.0.
Move access control source to it's own directory.
Change location of PropertyService to properties directory
Create replication directory. Move replication code there. Use log4j for replication logging (rollingfileappender). Beef up replication logging and error control.
Roll back replication user changes. Fix code that converts access levels to integer and to text.
Introduce replication user. Use the fileutil writer methods instead of writing directly.
Renamed MetaCatUtil to MetacatUtil
check for existing access control rows before inserting (we do not need duplicate rows specifying the same access control)
replicate xml_access entries across servers.see: http://bugzilla.ecoinformatics.org/show_bug.cgi?id=3464note: still needs to be tested...but needed to be checked in for other metacat installations to be updated....
Move the DBAdaptor accessor into a DatabaseService class
qualify xml and eml properties with an xml. prefix
Continue to qualify property names
Use qualified properties
Merge 1.9 changes into Head
Check into cvs head. In handle replication list method, parser will be intialized every time when it handles a list from another server.
Add event log for timed replication.
Somehow the change went to the head rather than branch. So i rollback the change in head.
This commit is for branch. In this commit the correct ip and user name will be stored in access_log table in replication event. However, it only for Forcereplication. And the test isn't completed yet. This commit is only for future use.
Removed the release ant token from all files in 'src'.
release
Fixed a bug in log
Fixed log bug when xml_revision updated failed.
Add more debug info.
Add code to handle replication document in xml_document table.
Replacing MetaCatUtil.debugMessage or MetaCatUtil.logMetacat call with logMetacat (private Logger object) call
Remove some debug message.
Change the code to make sure DocumentImpl class's constructor should have rev attached.
Add date info.
Add code to handle deleted document replication.
Add code to trim server name white space.
Remove output stream from the constructor.
Add some method to setting replication start time.
Add new feature that delete can be broadcasted by force replication.
Add a code to load eml201 parser to fix the bug that couldn't generate access rule for eml201 doc.
Made initial changes for accepting EML version 2.1.0 in Metacat
Change eml2 to eml200
Revise inital value for insert a new server. Change some debug level.
Change the date format from yy-mm-dd to mm/dd/yy.
Get rid of to_date function.
Fixed a bug to checking error.
Add code to get rid of white space in url.
Add a method to write error message into log file.
Add code to handle eml2 replciation.
Add code to handle text node was splited.
Merge cvs branch replication to cvs head.
Merge DBConnection branch to head.
Add the code to handle error in the respone for a update request.
Change code to handle the error message.
In update method, add code to handle if a user couldn't pass a trusting test.