Metacat

Merged in the D1_0_6_2_BRANCH changes that include the transition from ObjectFormat calls to ObjectFormatCache calls.

include GUID column for xml_access and related methods for storing/retrieving access rules

handle timed replication of system metadata. there are still a few outstanding issues:
-track server location of system metadata-only entries
-replication policy flag for system metadata-only entries?
-locking for replicated entries?
-forced replication of entries

-remove system metadata guid -> local id mapping (there is no document for system metadata now)
-include system metadata elements when replicating data objects (TODO: transfer all system metadata structures with the docinfo request).
TODO: remove docid+rev from the systemMetadata table definition

do not use XML files for storing SystemMetadata - use DB tables only.

Commenting out these MetacatRestClient tests because the Metacat RestService is no longer registered as a servlet and needs to be removed, along with the associated MetacatRestClient and tests.

Fixed MetaCatServletTest test failures by removing the hardcoded username/passwords used in the tests and replacing those with data from metacat.properties.

Fix AuthLdapTest to use accounts that are more reliable when searching for existing accounts in the getPrincipals output.

Modified Metacat to build against the D1_SCHEMA_0_6_1 branch of the dataone schemas by incorporating the 0.6.1-SNAPSHOT version of d1_common and d1_libclient libraries, and refactoring Metacat code references to the d1 schema changed types.

updating CrudServiceTest with better exception handling (more specific) to match refactored CrudService.java class

removed newline addition to test doc now that the newline problem is fixed in utilities project. Small adjustments to comments and debug statements to aid in debugging. Reordered tests to remove possible insertion between read and getLastDocid test.

Make this class be the subclass of the MCTestCase. The super class will initialize the property service for it.

commenting out the pisco user tests, since they only work in production.

put in more explanatory exception methods in the private helper functions that throw exceptions, and some debug statements in the onlineData201CasesTest_1 test.

Due to changes in MetacatHandler that creates system metadata from uploaded EML documents, OnlineDataAccessTest was failing certain tests because of errors in how the test documents were originally uploaded. MetacatHandler.createSystemMetadata() was throwing exceptions because of these errors, specifically when a data document is uploaded with a docid that is difeerent than the docid statement in the EML distribution URL. Most errors involved stated docids with .2 versions, and the metadata had .1 versions....

Cleaned up imports.

Updated D1 libraries to head to eliminate problems with date parsing -- added
Joda jar to support date parsing. Removed hardcoded DN from test file. Now
CrudServiceTest is passing again.

Remove hardcoded LDAP usernames from testing so that MetaCatServletNetTest will work even using alternate
LDAP servers.

Fixed the testNCEASLogin() password that I mistakenly changed.

Now that MetacatHandler creates GUID identifier mappings and system metadata, many of the tests in IdentifierManagerTest were failing that were manually creating id mappings and system metadata.
Modified testCreateMapping() and testCreateSystemMetadataMapping() to use docids that haven't been inserted to just exercise the methods....

Added a new test method testDeleteDocumentByGUID() to ensure MetacatHandler can delete based on GUID, but fall back to docid if the GUID isn't present. Modified CrudServiceTest and made some private methods public to leverage the D1 REST calls in that library (i.e. create() a doc with a GUID).

Add a method to test download a schema with included schemas.

Add a test file for XMLSchemaServcieTest.

Add a junit test file for testing the XMLSchemaParser class.

removed comments from crudservicetest

fixed bugs in listObjects

remove httpclient 3.1 and custom-built httpclient.jar
rework MetacatClient (and other classes) to use httpclient 4
updated build to not create httpclient.jar
encoding tests now pass.

Fix the MetadataTypeRegisterTest because it had the wrong URI for METS.

added code to do database query for listObjects

remove the empty dir.

This is redundant to the WorkflowSchedulerCleintTest in wsfscheduler svn module.

use more unique scope (document id prefix) so that getlastdocid runs successfully. use superclass generateDocumentId() method

adding fields for additional system metadata info

new paths for metacat populator

updated the populator

fixed compile errors

uncommented tests

fixes for creating SM for legacy docs

uncommenting tests

use detected document encoding or Metacat's default encoding (UTF-8)

include shared method in superclass

create another test for various i18n features including eml 2.1.1
http://bugzilla.ecoinformatics.org/show_bug.cgi?id=2495

uncomment the other test cases and make sure we do the clean up when the tests are complete (delete the uploaded documents)

use detected XML encoding when reading/writing files
use UTF-8 as default when performing queries in the DB (assume DB is using UTF-8)
remove as many PrintWriters (uses system default character encoding only) as possible and construct OutputStreamWriters where explicit encoding can be given....

fix for paths with semi-colons or other 'reserved' characters in them for D1 rest services

correct the prolog

changed test to use actual semi-colons instead of colons

add XmlStreamReader test for determining encoding

added semi-colon id test

found a failure when using an InputStreamReader without an encoding

a test to explore character encoding with utf8 chars

support EML 2.1.1

include anonymous parameter when getting report

add support for EML 2.1.1

organized code for inputstream handling

removed dependency on InputStreamToOutputStream so that exceptions will be passed correctly.

changes for new d1 schemas

uncommenting tests

implemented crud.delete

implemented crud.describe

got the streaming mime multipart working

still working on mmp stream parsing

working on integrating streaming mmp into resourceHandler

more work on streaming mmp support

getting closer to a non-memory bound solution for mime multipart de/encoding

fixed a couple bugs with dates and fixed a major bug where metacat was reading the entire document from the database everytime a DocumentImpl instance was created even though it didn't need to

fixed a bunch of small errors, did some reformatting, and fixed a bug that I thought was fixed last week

minor changes to crudservicetest

adding a test that I forgot to check in a while back

test for the profiler

added notes on packages to upload with knb-mn

updated the metacatpopulator to use the d1 eml parser to create system metadata in a smarter fashion.

print the workflow server name

merging changes from branch

merging test changes into the trunk

fixed error where insert/update errors were not getting caught as exceptions

fixed bug with logger where localid didn't get appended when inserting a data file

fixed update bug I introduced yesterday while fixing replication

fixed identifier manager problem with getting guid for systememtadaa doc

updated for date format that supports time zones correctly.

changed date comparison to compare to gmt only

fixed bug in access control where the sessionid was not correctly passed along so changes of privileges were not being made

add back all tests

fixing date formatting and timezone issues

fixed some bugs in time handling for listObjects and getLogRecords

fixed bug in getLogRecords where the wrong id was getting set as the identifier

fixed listObject bugs, added params to the listObjects rest interface, consolidated the date format passed in params

fixed query for listObjects so that paging works as advertised.

fixed error with listObjects and get.

got the crud test to pass on knb-mn, now need to figure out why the rest service still isn't working

added a failure test to the testPublicAccess test. This test should allow public gets and listObjects but not public creates

changed getSessionData so that it will respond property to public tokens

test to check for public access to get and listObjects

MetacatPopulator is now working. It allows you to do a query from one metacat instance, then insert any returned docs into another metacat instance

tracked down streaming bug. now need to figure out a better way to fix it.

adding getlogrecords to the rest interface.

changed getLogRecords to use new classes for log events

fixed bug in metacat that allowed attribute data to be appended to element data in returnfields

implementing getLogRecords