Metacat

MN.getPackage() - test with ORE that includes 2 data files and a "metadata" file (still should be using EML for that test). https://projects.ecoinformatics.org/ecoinfo/issues/6026

First pass at MN.getPackage() implementation using Bagit library from LOC. https://projects.ecoinformatics.org/ecoinfo/issues/6026

add method for publishing existing object (usually assumed to be scimeta) with a DOI. https://projects.ecoinformatics.org/ecoinfo/issues/6014

add Metacat servlet action to force the reindexing of one or more or all pids in the system. https://projects.ecoinformatics.org/ecoinfo/issues/5945

add simple test for the IndexEventDAO class -- adding, removing, listing events in the DB table. https://projects.ecoinformatics.org/ecoinfo/issues/5944

correct regex for whitespace in D1 identifier.

Add code to test title for the query result.

Rewrite some methods, so the query result can be processed many times.

Removed the commented out text cases and add a test for archvied document.

Call setting certificate location to be the test one after getting the MN baseurl.
The method to getting MN baseurl somehow calls CN and it set the certificate location to be /var/metacat/certs/METACAT1.pem.

Use the MNode to query the server when we use certificates to set up the session.

Add the code to test the user with a distrusted certificate.

Add tests to test group and rightsholder.

Change the delete to archive.

Add a test for testing access control for the solr query.

use maven to manage most jar dependencies in Metacat.
Exceptions include: LSID, Datamamager (EML),

lookup the title for EML files when registering DOIs.
lookup the creator from DataONE CN (if available).
add EML-based test. http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5513

include the create test in the suite

Correctly mint and register DOIs in teh MN API implementation. Add tests to exercise minting and creating. http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5513

remove older lucene library and include ORE test to make sure that change does not prevent us from generating OREs. http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5874

move DocInfo parsing into utilities project so that it can be used by Morpho as well as Metacat.
http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5737

return from test when we encounter the NotImplemented exception for CN.search()

include identifier.guid in the test SQL clause.

CN.search() id not implemented by metacat -- making that explicit and also testing for it.

In migrating to Hazelcast 2.4.x, replace deprecated methods.

simplify the xml_access query, and instead use guid to check for permission. Now the docid/rev join (to get most recent version for search results) happens "higher up" in the query.
http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5696

remove use of HttpMessage (in morpho.jar) in favor of standard httpclient methods for calling the servlet in tests

use CDATA for docname field in docInfo so that XML parser ignores the content that can contain characters like "&

sleep before updating and deleting test documents - otherwise their index entries may not be fully written and this causes errors (update and delete first attempt to remove index references, but if they are not in the DB yet then they are not removed but then they do get added and the FK constraints make the delete fail). Since we know indexing occurs in a separate thread with a configured delay, we just use this same delay in our testing.

Use a final static string to replace the hard code.
Search document title rather than id in testReplicateEML_AtoB method.

use CN session when testing getLogRecords() and getOperationStatistics() becuase they are now protecting "sensitive" information

test for running concurrent Metacat queries to mimic Kepler data search.
http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5518

include testSynchronizationFailed() and call as the CN subject so that it is authorized.

use MN (self) as the Session.subject so that the MN.delete() call is successful.

comment out testDelete because it requires acting as the MN
comment out testSynchronizationFailed because it requires acting as the CN

uncomment the MN tests (I bet this was an oversight during local testing)

no not record EML access rules that use the "denyFirst" permOrder.
https://redmine.dataone.org/issues/2614

change ordering of getLogRecords() parameter -- pidFilter is in the middle now

upgrade to latest RC in libclient and common jars -- includes updated getLogRecords and new mn.generateIdentifier method

Add testIsEquivIdentityAuthorized() to ensure that [MN|CN].isAuthorized() is authorizing equivalent identities correctly. Note: Using TypeMarshaller.marshalTypeToOutputStream(type, System.out) to serialize an object seems to jack up output to stdout - not sure why.

refactor D1-specific upgrade utilities into their own package

test harness for running system metadata generation outside of the upgrade process

include comment about KNB estimated time to run during upgrade:
Total time: 20 minutes 58 seconds

use "test" to exercise upgrade code on staging DB.

Update the MNodeServiceTest to test the validity of the node document returned by getCapabilities() by parsing it with the TypeMarshaller.

use RC-1 Dataone jars

remove method: assertRelation
https://redmine.dataone.org/issues/2158

Use a Date with resolution to milliseconds.

include the EML and data tests in the suite

debugging data locking test

cannot check for deleted data since it is forever available (archived)

defer to super class member variables

do not include the "v1" in the base url for the target MN

update tests to comply with these chenages:
new jars with many changes -- including new CN methods: ping, describe, listChecksumAlgorithm. Removed MN.setAccessPolicy. Refactored CN.setOwner() to CN.setRightsHolder().

for test to compile, provide BaseException param for setReplicationStatus. I used a NotAuthorized instance.

adjust after refactoring tests that use EML queries

query for deleted metadata when testing that replication communicated the deletion. to check data, we try to update the data object on the target node (which should fail)

add test for data locking

delete data and eml on the home Metacat and check that the change propagates to the target

actually include the test in the suite
http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5520

EML replication test with insert, update and set access
http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5520

test update and set access during replication from A->B
http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5520

a minute for replication?

comment out B->A test
http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5520

use correct URLs for replication/Metacat client

simple tests for 2-way data replication
http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5520

fix tests:
-insert default objectFormatList from d1_common (includes schemaLocation now)
-calculate checksum for the system metadata using the ChecksumUtil in d1_common

prepared statement toString() does not include quotes (') around the inserted values -- now the test reflects this.

Metacat now keeps track of permissions on a per-version basis -- the test reflects this change in behavior.
http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5560

refactor Metacat access handling to be on a per-revision basis so that it more closely aligns with the DataONE approach
http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5560

update test to reflect newer change:
handle queries with predicates correctly.
when docids are used in the return field look up, we need to make sure they are included in the values in the correct order for their corresponding parameter place holders (?)

Add new test method to test getAccessControl.

catch all possible exceptions

act as a registered CN when calling CN methods -- create(), for example, is restricted to only be callable by a node of type CN.

use IOUtils for testing read/write of XML strings. This goes along with the utilities project change to use the same apache commons IO library. Be sure to clean the utilities checkout in order to catch the right utilities svn tag when building!

when comparing D1 Subject objects, use the equals() method not direct string comparison
https://redmine.dataone.org/issues/2050

upgrade to 1.0.1-SNAPSHOT DataONE jars

started replication unit test

MockCNode.getSystemMetadata() no longer throws InvalidRequest.

remove catch blocks for unthrown exceptions

uses prepared statement instead of plain old statement.
deprecated the DBConnection.createStatement() method to discourage direct parameter value use in favor of parameter binding.
http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5527

Add a test for systemMetadataChanged. This should be fleshed out more so that the test uses a CN certificate.

uses prepared statement parameter binding for queries
http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5527

check object format id values since they are not Comparable objects

make MNodeServiceTest pass JUnit testing

Update CNodeService test to include serialVersion in it's calls as appropriate.

restore -- apparently this was used with reflection in a test...

MN.setAccessPolicy() is deprecated, commenting out the test.

removing non-junit file from the test area

load hazelcast test files from the classpath rather than from rob's personal directory.

SystemMetadataManager's functionality is handled by IdentifierManager. Removing it and it's test.

MetadataTypeRegister is now replaced by ObjectFormatService. Removing it and it's test.

move the DataONE 1.0.0-SNAPSHOT

add User-Agent logging to support D1 requirements

remove old RestServlet handler -- not used now

including newer d1 libclient that uses Foresite (and Jena) to construct/parse ORE resource maps for DataONE

update D1 jars to include recent SubjectList -> SubjectInfo refactoring and the SUBJECT_PUBLIC constant

use an actual data object (bytes of "test") when generating system metadata so that the checksum is a valid checksum

test relationship assertion using obsoletes

move bulk of the Hazelcast code into HazelcastService from CNodeService so that it is centrall located - easier to manage and configure