Metacat

removing non-junit file from the test area

load hazelcast test files from the classpath rather than from rob's personal directory.

SystemMetadataManager's functionality is handled by IdentifierManager. Removing it and it's test.

MetadataTypeRegister is now replaced by ObjectFormatService. Removing it and it's test.

move the DataONE 1.0.0-SNAPSHOT

add User-Agent logging to support D1 requirements

remove old RestServlet handler -- not used now

including newer d1 libclient that uses Foresite (and Jena) to construct/parse ORE resource maps for DataONE

update D1 jars to include recent SubjectList -> SubjectInfo refactoring and the SUBJECT_PUBLIC constant

use an actual data object (bytes of "test") when generating system metadata so that the checksum is a valid checksum

test relationship assertion using obsoletes

move bulk of the Hazelcast code into HazelcastService from CNodeService so that it is centrall located - easier to manage and configure

cleaned up mock tests on hzObjectPathMap. split out code for mocking a datastore into MockObjectPathMap.

initialize Hazelcast from the custom configuration when initializing the Metacat service.

exercise the shared hazelecast system metadata map and custom loader/store by inserting normally and also directly into the map. both mechanisms allow up to read the same system metadata (only tested on single local instance)

test hzSystemMetadata Map and custom loader

copied default hazelcast configuration from the jar into hazelcast.test.properties.xml, to get instances to talk to each other.

configuring hazelcast tests

further refactoring and start of unit tests for hazelcast elements

return the test Identifier so other tests can piggyback

Update test for getReplicationStatus to reflect API change.

changes for schema update (d1_common)

Update tests to use the DataONE 0.6.4 schema and types.

latest D1 jars - changes include:
updateSystemMetadata() impl for CN
new identifier methods (generate is its own method)
removal of the resourceMap pointer from system metadata

refactor Constants

refactor Constants and ServiceTypeUtil

use new "v1" types from DataONE

remove CrudService -- replaced by MNodeService and CNodeService

reset object input stream (testGet)
check for reserveIdentifier NotImplemented (expected)

Return the new pid (not the obsoleted pid) on update(), and set the correct system metadata.

include Mock version of looking up the base url for our own node - for testReplicate().
still expecting a failure during MN.replicate() since we are trying to add the replica to the same MN from which it came.

allow alternative CNode implementations (subclasses) to be returned when D1Client.getCN() is called. The alternative is given in the Settings configuration via the D1Client.cnClassName property.
MockCNode in Metacat is an example of such an alternative and is used when Metacat requires methods of a CN be present for unit testing.

Test that the obsoletes and derivedFrom system metadata fields were set correctly.

use objectFormatIdentifier for listObjects()
remove provisional system metadata indicator - Metacat will not implement reserveIdentifier()

add test for synchronization failed -- TODO: verify that the log record made it in the logs

provisional replicate() implementation -- does not check if the session's subject is "allowed" to do this.
the test also requires 2 servers -- right now it attempts to replicate with itself which will fail because of duplicate IDs

provisional version of getOperationStatistics() -- not clear if we are meant to aggregate by hour or by day

include test for replicate() method -- still fails as not implemented

allow null objects (use default checksum)

wire-up the getCapabilities() and getOperationStatistics() methods for when they are actually implemented (tests fail now)

test listObjects()

include MNAuthorization tests

refactor to use common superclass D1NodeServiceTest for shared methods

test for getLogRecords -- with some semi-reasonable expectations on the return that is expected

tests for isAuthorized and setAccessPolicy

include getLogRecords() test, though it is commented out at the moment

test for CN.getSystemMetadata()

test CN.get()

Uncommented MNStorage tests.

Send the correct Date format in testGetLogRecords(). I've commented out certain tests in the suite until isAuthorized() functionality is working. It is returning a NotFound exception even though the object is clearly in Metacat.

Add MNodeServiceTest tests for MNCore, MNRead, MNStorage, MNAuthorization implementations. MNReplication remains to be tested, as does MNCore.getCapabilities() and getOperationStatistics().

add the d1_common formats if they do not exist on the server

remove resolve() test -- not implemented in Metacat

add methods for testing CNodeService impl. Still some failing tests, but whittling them down

refactor to test the entire CN service impl

CNCoreImpl is replaced by CNodeService

Updated tests to use ObjectFormatCache.getInstance(), and added tests for CNCoreImpl.listFormats and getFormat().

When calling SystemMetadata.getObjectFormat(), return the string value of the ObjectFormatIdentifier rather than ObjectFormat.toString() (which no longer returns the fmtid string).

organize imports so that it is clearer what dependencies exist on the D1 jars

Merged in the D1_0_6_2_BRANCH changes that include the transition from ObjectFormat calls to ObjectFormatCache calls.

include GUID column for xml_access and related methods for storing/retrieving access rules

do not use XML files for storing SystemMetadata - use DB tables only.

Modified Metacat to build against the D1_SCHEMA_0_6_1 branch of the dataone schemas by incorporating the 0.6.1-SNAPSHOT version of d1_common and d1_libclient libraries, and refactoring Metacat code references to the d1 schema changed types.

updating CrudServiceTest with better exception handling (more specific) to match refactored CrudService.java class

Cleaned up imports.

Updated D1 libraries to head to eliminate problems with date parsing -- added
Joda jar to support date parsing. Removed hardcoded DN from test file. Now
CrudServiceTest is passing again.

Added a new test method testDeleteDocumentByGUID() to ensure MetacatHandler can delete based on GUID, but fall back to docid if the GUID isn't present. Modified CrudServiceTest and made some private methods public to leverage the D1 REST calls in that library (i.e. create() a doc with a GUID).

removed comments from crudservicetest

fixed bugs in listObjects

Fix the MetadataTypeRegisterTest because it had the wrong URI for METS.

added code to do database query for listObjects

new paths for metacat populator

updated the populator

fixed compile errors

uncommented tests

fixes for creating SM for legacy docs

uncommenting tests

use detected document encoding or Metacat's default encoding (UTF-8)

changed test to use actual semi-colons instead of colons

added semi-colon id test

organized code for inputstream handling

removed dependency on InputStreamToOutputStream so that exceptions will be passed correctly.

changes for new d1 schemas

uncommenting tests

implemented crud.delete

implemented crud.describe

got the streaming mime multipart working

still working on mmp stream parsing

working on integrating streaming mmp into resourceHandler

more work on streaming mmp support

getting closer to a non-memory bound solution for mime multipart de/encoding

fixed a couple bugs with dates and fixed a major bug where metacat was reading the entire document from the database everytime a DocumentImpl instance was created even though it didn't need to

fixed a bunch of small errors, did some reformatting, and fixed a bug that I thought was fixed last week

minor changes to crudservicetest

adding a test that I forgot to check in a while back

test for the profiler

added notes on packages to upload with knb-mn

updated the metacatpopulator to use the d1 eml parser to create system metadata in a smarter fashion.

fixed error where insert/update errors were not getting caught as exceptions