Metacat

Reviewed code for all uses of FileInputStream, checking to see if the method should be closing the stream, and if so, closing it in the method as well as in the finally clause to ensure we don't leak file descriptors.

Refer to metacat.war deployments since those are now the default. https://projects.ecoinformatics.org/ecoinfo/issues/6082

check both previous and current data revisions when updating packages. https://projects.ecoinformatics.org/ecoinfo/issues/5647

do not use a URL that could resolve when testing the Metacat inaccessible exception.

use shared metacatContextDir variable for calls to Harvester.loadProperties(metacatContextDir);

add Metacat servlet action to force the reindexing of one or more or all pids in the system. https://projects.ecoinformatics.org/ecoinfo/issues/5945

move DocInfo parsing into utilities project so that it can be used by Morpho as well as Metacat.
http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5737

include identifier.guid in the test SQL clause.

simplify the xml_access query, and instead use guid to check for permission. Now the docid/rev join (to get most recent version for search results) happens "higher up" in the query.
http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5696

test for running concurrent Metacat queries to mimic Kepler data search.
http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5518

no not record EML access rules that use the "denyFirst" permOrder.
https://redmine.dataone.org/issues/2614

defer to super class member variables

adjust after refactoring tests that use EML queries

query for deleted metadata when testing that replication communicated the deletion. to check data, we try to update the data object on the target node (which should fail)

prepared statement toString() does not include quotes (') around the inserted values -- now the test reflects this.

Metacat now keeps track of permissions on a per-version basis -- the test reflects this change in behavior.
http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5560

refactor Metacat access handling to be on a per-revision basis so that it more closely aligns with the DataONE approach
http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5560

update test to reflect newer change:
handle queries with predicates correctly.
when docids are used in the return field look up, we need to make sure they are included in the values in the correct order for their corresponding parameter place holders (?)

Add new test method to test getAccessControl.

use IOUtils for testing read/write of XML strings. This goes along with the utilities project change to use the same apache commons IO library. Be sure to clean the utilities checkout in order to catch the right utilities svn tag when building!

uses prepared statement parameter binding for queries
http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5527

add User-Agent logging to support D1 requirements

remove CrudServiceTest reference in main Metacat test (about to remove from svn)

merge changes from 1.9.5 branch that read site docs from SVN not a defunct LTER server

do not use pisco for referral testing (merge from 1.9.5)

merge 1.9.5 changes to trunk (delete now preserves access rules and docs can be read after deletion)

Get username and password from a property file.

handle timed replication of system metadata. there are still a few outstanding issues:
-track server location of system metadata-only entries
-replication policy flag for system metadata-only entries?
-locking for replicated entries?
-forced replication of entries

-remove system metadata guid -> local id mapping (there is no document for system metadata now)
-include system metadata elements when replicating data objects (TODO: transfer all system metadata structures with the docinfo request).
TODO: remove docid+rev from the systemMetadata table definition

Commenting out these MetacatRestClient tests because the Metacat RestService is no longer registered as a servlet and needs to be removed, along with the associated MetacatRestClient and tests.

Fixed MetaCatServletTest test failures by removing the hardcoded username/passwords used in the tests and replacing those with data from metacat.properties.

Fix AuthLdapTest to use accounts that are more reliable when searching for existing accounts in the getPrincipals output.

removed newline addition to test doc now that the newline problem is fixed in utilities project. Small adjustments to comments and debug statements to aid in debugging. Reordered tests to remove possible insertion between read and getLastDocid test.

Make this class be the subclass of the MCTestCase. The super class will initialize the property service for it.

commenting out the pisco user tests, since they only work in production.

put in more explanatory exception methods in the private helper functions that throw exceptions, and some debug statements in the onlineData201CasesTest_1 test.

Due to changes in MetacatHandler that creates system metadata from uploaded EML documents, OnlineDataAccessTest was failing certain tests because of errors in how the test documents were originally uploaded. MetacatHandler.createSystemMetadata() was throwing exceptions because of these errors, specifically when a data document is uploaded with a docid that is difeerent than the docid statement in the EML distribution URL. Most errors involved stated docids with .2 versions, and the metadata had .1 versions....

Fixed the testNCEASLogin() password that I mistakenly changed.

Now that MetacatHandler creates GUID identifier mappings and system metadata, many of the tests in IdentifierManagerTest were failing that were manually creating id mappings and system metadata.
Modified testCreateMapping() and testCreateSystemMetadataMapping() to use docids that haven't been inserted to just exercise the methods....

Added a new test method testDeleteDocumentByGUID() to ensure MetacatHandler can delete based on GUID, but fall back to docid if the GUID isn't present. Modified CrudServiceTest and made some private methods public to leverage the D1 REST calls in that library (i.e. create() a doc with a GUID).

Add a method to test download a schema with included schemas.

Add a test file for XMLSchemaServcieTest.

Add a junit test file for testing the XMLSchemaParser class.

use more unique scope (document id prefix) so that getlastdocid runs successfully. use superclass generateDocumentId() method

adding fields for additional system metadata info

create another test for various i18n features including eml 2.1.1
http://bugzilla.ecoinformatics.org/show_bug.cgi?id=2495

uncomment the other test cases and make sure we do the clean up when the tests are complete (delete the uploaded documents)

use detected XML encoding when reading/writing files
use UTF-8 as default when performing queries in the DB (assume DB is using UTF-8)
remove as many PrintWriters (uses system default character encoding only) as possible and construct OutputStreamWriters where explicit encoding can be given....

fix for paths with semi-colons or other 'reserved' characters in them for D1 rest services

correct the prolog

add XmlStreamReader test for determining encoding

found a failure when using an InputStreamReader without an encoding

a test to explore character encoding with utf8 chars

support EML 2.1.1

include anonymous parameter when getting report

add support for EML 2.1.1

merging changes from branch

merging test changes into the trunk

fixed identifier manager problem with getting guid for systememtadaa doc

fixed bug in access control where the sessionid was not correctly passed along so changes of privileges were not being made

working on a function to create dataone system metadata docs for legacy metacat objects that do not have SM.

removed CrudService dependency on servlet params. CrudService is now a singleton. I'm getting an error from metacat saying it can't find teh systemmetadata schema, even though it is, in fact, registered with metacat. need to identify why this is happening.

added a test for CrudService and added tests for IdentifierManager for each method

Add a debug statement to show existing doc id

Refactored putObject method to separate the create() and update() portions in order to match the method signatures needed for DataONE.

Modified ResourceHandler.deleteObject() to take a globally unique id (guid)
as input rather than a metacat-conformant docid. The guid is translated
into a metacat docid which is then used in the handleDelete() call. So now
Metacat supports use of arbitrary identifiers for delete operations, but...

Added support to ResourceHandler to allow the putObject method to use
arbitrary guid strings as input. These strings are examined, and if they
match the Metacat docid format (scope.id.rev), they are used directly. If
the id is a string in another format, a new localId is generated based on...

Updated MetacatRestClientTest so that the tests are actually calling fail()
when errors occur, and fixed all sixteen tests so that they run successfully
and can be run from any client (i.e., they don't depend on particular
documents being present in Metacat). The 'restfiles' directory can probably ...

Added initial support for arbitrary string identifiers in metacat (referred to as a guid). The new identifier table is used to map arbitrary string identifiers to Metacat's current docid format (referred to as the localId). Added a new IdentifierManager class to manage this table, adding new mappings as objects are added to the system. Modified the MetacatTest service to utilize this mapping table to look up a localId from a guid. IdentiferManagerTest is working with these guids now, as is the MetacatTest service get() method, but other parts of the system are unaware of them (e.g., create, update, delete operations are unaware, and no mapping is created when new documents are created). As a consequence, the MetacatRestClientTest is not working (although it also had lots of hardcoded dependencies that need to be fixed as well).

Merging in REST interface implementation that was created by Serhan Akin. Main change is
a refactored MetaCatServlet.java class, in which all of the handle* methods were moved
into a separate MetacatHandler.java class. This allows both the standard MetaCatServlet...

add delay in insertSpatialDocs test to avoid buildIndex conflict

Fix errors in workflow scheduler unit test

Beef up comments

Add access api test and supporting files

handle ben's change to the metacat client read api because it returns inputstream now

Change location of PropertyService to properties directory

Change MetaCatVersion to MetacatVersion

Fix references to db classes that were moved and refactoring missed.

Remove unneeded exception handling

Implement fix for Bug #4245: Harvester command line scripts don't execute.

Handle UtilException

Remove initialization of MetacatUtil.buildIndexPaths in test files. This is taken care of in the background now using SystemUtil.getIndexPaths()

The test was failing because it was reading the sitemap file into a buffer and the limit was being reached. Used the FileUtil string reader to get the file.

Create a propertyService.getTestInstance method to facilitate unit tests

Renamed MetaCatUtil to MetacatUtil

Renamed MetaCatUtil to MetacatUtil

Added EML 2.1.0 test cases.

Change Inline to be inline in the test eml. Added debug statements.

Insert documents with different permissions and make sure only the publically readable ones get pulled into the sitemap file.

Change getAccessBlock from private to protected so it can override the MCTestCase version

Stripped out common eml document creation elements and put into the MCTestCase base class

Add debug statements at the beginning of every test module.

Add EML 2.1.0 access scenarios

Add test cases for EML 2.1.0 documents. Add some generics typing.

Add test cases for EML 2.1.0 documents. Add some generics typing.

New test case to test xml schema registration.

Delete test document after test is through.

Convert remainder of tests that use PropertyService to extend the MCTestCase class

Convert remainder of tests that use PropertyService to extend the MCTestCase class

Add debug statements