recursively submit obsoleted objects for indexing when instructed. https://projects.ecoinformatics.org/ecoinfo/issues/6424
sync pids of <distribution><online> data objects with CN when their access rules change in EML 2.0.* <additionalMetadata>
Sync access policy between MN -> CN when access rules are updated in EML 2.1+ for data object
Unify solr indexing with an IndexTask that is added to the queue -- allows us to send more than just the systemMetadata to the indexer. Initially this is for READ event counts for each document. https://projects.ecoinformatics.org/ecoinfo/issues/6346
On changes to system metadata in CNodeService and DocumentImpl, increment the serialVersion.
Refactor to use IOUtils.closeQuietly() which handles nulls and streams that are already closed.
Added close() to finally block for readFromFileSystem() call.
Closing FileOutputStream handles so that the OS limits on filehandles are not exceeded.
If the pathquery engine is disabled, the xml path index queue will be disabled as well.
support a "force replication delete all action" during replication. This is used when we want Metacat to remove the content from the other target replicas because the DataONE delete() action was called (more powerful than just "archive").
do not use tmp file to return an inputstream on read() operations - just read from the file we already have. https://projects.ecoinformatics.org/ecoinfo/issues/6009
use an independent ISet<SystemMetadata> structure to communicate objects that should be indexed by metacat-index. https://projects.ecoinformatics.org/ecoinfo/issues/5943
remove indexing task from the queue when we are updating the document
move DocInfo parsing into utilities project so that it can be used by Morpho as well as Metacat.http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5737
rollback the delete() when there is an error performing part of it -- don't want to end up with partial delete.
use Identifier object not String when retrieving SM from the HZ map to set archived during delete()
remove document from the indexing queue when delete is called. http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5750
mark documents as archived=true when they are deleted using the Metacat API.https://redmine.dataone.org/issues/3406
correct the number of prepared statement parameters when inserting to xml_revisions table. Errors like the following were showing in the replication log file:knb 20120831-19:42:38: [ERROR]: DocumentImpl.writeReplication - Failed to create access rule for package: john.15950.1 because The column index is out of range: 12, number of columns: 11. [ReplicationLogging]
when updating a document on a remote server, we still need to use the previous docid to check that the user has permissions to do so (rather than the new id that is obsoleting the old id). This was discovered by M Servilla at LTER.
[optionally] do not archive the xml_documents and xml_nodes to *_revisions when 'deleting' a document. This will effectively guarantee that the document/data cannot be retrieved after delete.NOTE: D1 system metadata will persist (for now) so that the ID cannot be reused with the DataONE API but Metacat calls may allow the ID to be reused -- may need to reconsider this behavior....
optionally remove the document/data file from the filesystem completely when 'deleting' it.https://redmine.dataone.org/issues/2677
add a parameter for optionally writing EML-embedded access control rules to the Metacat DB.https://redmine.dataone.org/issues/2584https://redmine.dataone.org/issues/2583
check previous revision for permissions to update (includes data described by EML)
refactor Metacat access handling to be on a per-revision basis so that it more closely aligns with the DataONE approachhttp://bugzilla.ecoinformatics.org/show_bug.cgi?id=5560
uses prepared statement instead of plain old statement.deprecated the DBConnection.createStatement() method to discourage direct parameter value use in favor of parameter binding.http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5527
http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5527
include clearer error message when UPDATE action is requested on a replicated document and we fail to successfully get a lock from the source Metacat serverhttp://bugzilla.ecoinformatics.org/show_bug.cgi?id=4907
do not delete the access rules when we "archive" the document on "delete" (commented out for now)
remove System.out statements in favor of logging
use the jaxb date parser for ISO 8601 formats. the numeric and date node values are now calculated after the document has been successfully inserted in the db so any sql exceptions do not prevent the raw node data from being saved.http://bugzilla.ecoinformatics.org/show_bug.cgi?id=2084
add support for temporal element query in pathqueryhttp://bugzilla.ecoinformatics.org/show_bug.cgi?id=2084
DocumentImpl.delete() now throws finer grained exceptions (not a general exception). Consequently, the classes that call it have been updated to handle the thrown exceptions, including CrudService, ReplicationHandler, and ReplicationService.
use detected XML encoding when reading/writing filesuse UTF-8 as default when performing queries in the DB (assume DB is using UTF-8)remove as many PrintWriters (uses system default character encoding only) as possible and construct OutputStreamWriters where explicit encoding can be given....
add support for EML 2.1.1
made delete serialize the identifier
fixed a bunch of small errors, did some reformatting, and fixed a bug that I thought was fixed last week
Fixed spelling error.
Correct log warning message - not "Account" but "Accession" number
removed system.outs
amost have update working. still need to get unit test squared away.
refactored XMLSchemaService to not have static methods. made the CrudServiceTest more robust.
fixed schema location bug. the dataone schemas are now correclty found
removed CrudService dependency on servlet params. CrudService is now a singleton. I'm getting an error from metacat saying it can't find teh systemmetadata schema, even though it is, in fact, registered with metacat. need to identify why this is happening.
Modifications to support the DataONE service API version 0.1.0. For DataONE, the get() andcreate() services are partially complete. Several more functions and checks need to be added tocreate() before it is viable. This DataONE support is not complete, and the current support breaks the MetacatRestClientTest for the time being (this client will eventually be removed).
Merge 1.9.2 changes back into the trunk
Modified readFromMetacat() to pass most exceptions up the call stack, which allows creation of new entry points for calling reads. Still need tocontinue factoring out the HTTPServletResponse that is passed in in order tomake entrypoints that are not servlet based possible. Problem now is in...
Beef up log messages
do a quick retry if building path index fails with a SQL error
Make sure buildIndex throws an exception if it has a sqlexception. That way the indexing object will be added to the indexing queue and reprocessed.
Pass the doc xml as a string to docImpl.write and writeRepication. This is so a reader can be create for the parsing and for the write to disk. Also created a db access class for xml query result deletion.
Fix the bug of http://bugzilla.ecoinformatics.org/show_bug.cgi?id=4645. handleGetRevisionAndDocTypeAction will search both xml_documents and xml_revisions table.It also changed some constrain in AccessionNumber when user update a document, of which all previous versions are in xml_revisions table.
Change log levels where appropriate and add class/method name to output.
beef up error log messages
special handling for RDF namespace documents (semtools project)
Move access control source to it's own directory.
Change location of PropertyService to properties directory
Move document specific utilities to DocumentUtil from MetacatUtil. This makes it easier to define a layer between the core metacat services and the rest of the code.
Create database and shared directories for database management code and shared code respectively.
Add archival read funtionality (jar/kar/war files)
Introduce replication user. Use the fileutil writer methods instead of writing directly.
Beef up exception handling from file utilities. Move UtilException to MetacatUtilException to eliminate conflict with similar exception in utility package.
Do not read or write zero length documents to/from disk
Format indexPaths in metacat.properties. Remove from build.properties and build.xml. Move indexPath list getter from MetacatUtil to SystemUtil.
Fix typos in error messages.
Renamed MetaCatUtil to MetacatUtil
move readerToString function to StringUtil class in utilities module.
Add debug statements
Rename LDAPUtil to AuthUtil
Move the code to write metadata to disk into documentImpl
Add some generics typing. Separate the code that strips inline data from document files to have a different strategy for 2.0.X versus 2.1.X documents.
Look for schemaLocations in the document while initializing parser. If full schema validation is turned on in metacat.properties, and at least one schema is not registered locally, then turn on full schema validation in the parser.
Fix workgroup description
add closing </distribution> tag to regex for inline data.
Created a method in DocumentImpl to read a document file from disk. Created a support method to strip inline data for users that don't have read access.
Move the DBAdaptor accessor into a DatabaseService class
qualify xml and eml properties with an xml. prefix
Continue to qualify property names
Fully qualify spatial and replication properties
Use qualified properties
Add sql debug statements
Append context url onto system id instead of server url.
Merge 1.9 changes into Head
fixed bug 3403
Remove an obsolete private method getPartNodeRecordList.
Use dev=1 replace dev like '1'. since postgres 8.3 doesn't support it.
Add debug info for special charater.
Modified a sql command from "like" to "=". It dramatically improves the peformance of build index.
Add debug information.
Add code to expire cached query result when metacat have a insert, update or delete action.
Fix a bug that move node from xml_nodes to xml_nodes_revisions.
Decreased the debug message priority to 'debug' level for messages when reading an XML document.
Somehow the change went to the head rather than branch. So i rollback the change in head.
This commit is for branch. In this commit the correct ip and user name will be stored in access_log table in replication event. However, it only for Forcereplication. And the test isn't completed yet. This commit is only for future use.
Change some log info.
One more patch for bug 2469:Although the correct parentid values were being indexed in xml_path_indexfor leaf node xpaths, they were still incorrect for relative and absolutepaths. This patch modifies traverseParents() and changes the parent node idto be indexed to that of the leaf node, no matter if the path is a leaf,...
As a continued fix for http://bugzilla.ecoinformatics.org/show_bug.cgi?id=2469,I've fixed the indexing implementation in both buildIndex() andtraverseParents(). Duane pointed out that the incorrect parent node idswere being indexed in xml_path_index, causing some stylesheets to render...
Fixed the implementation of the buildIndex function which was notworking for new document insertions. A previous fix in updatePathIndexfor ATTRIBUTE data inadvertantly caused a foreign key duplicationexception for insertions of ELEMENT nodes when multiple relative paths...
I'm fixing a compile problem under jdk 1.4.2, where the get() method inHashMap needs an Object as a parameter, not a primitive data type. I changedthe long to a Long as the lookup key.