Project

General

Profile

Statistics
| Revision:

# Date Author Comment
6826 01/03/2012 10:56 AM ben leinfelder

upgrade routine to purge empty replicated data files so that they can be re-replicated
http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5536

6822 12/22/2011 11:51 AM Chris Jones

Make sure the local id isn't null when we try to get the object from the local instance.

6821 12/22/2011 11:16 AM Chris Jones

Simplify the error handling, and throw the exception once the CN is updated with the new status.

6820 12/22/2011 11:13 AM Chris Jones

Set the replica status to failed (not invalidated) when we get exceptions trying to read the object bytes. Not much of a difference, but only the CN, in theory, is supposed to be able to set the invalidated status.

6819 12/22/2011 11:07 AM Chris Jones

Set the replication status to invalidated when we have a localId, but getting the object bytes fails for any reason.

6818 12/22/2011 10:51 AM Chris Jones

Only call super.create() if there's no localId found on the MN (ie a replica is there from an out of band process).

6817 12/22/2011 10:40 AM Chris Jones

Get the object inputstream from the local metacat instance using MetacatHandler.get() rather than MN.getReplica() so we don't throw an InvalidToken exception when passing in a null Session. The D1Client object is never used for this local call.

6816 12/22/2011 09:24 AM ben leinfelder

interpret permissions as hierarchical
https://redmine.dataone.org/issues/2150

6814 12/21/2011 01:00 PM ben leinfelder

remove flag for independent system metadata replication -- these entries are replicated along with the data/metadata objects or via hazelcast when the actual object is not on the server.

6808 12/20/2011 11:25 AM ben leinfelder

process the current revision, not the latest!
use direct object/system metadata insertion for ORE maps.

6807 12/20/2011 11:21 AM ben leinfelder

allow other Metacat process (system metadata and ORE generation) to directly insert objects and system metadata without having to go through the MN/CN methods.

6806 12/20/2011 11:19 AM ben leinfelder

sort the docids so that "old" revisions are processed before newer ones

6805 12/19/2011 01:07 PM ben leinfelder

only attempt to unlock a lock if it was created (in the finally block)

6803 12/16/2011 04:24 PM ben leinfelder

new jars with many changes -- including new CN methods: ping, describe, listChecksumAlgorithm. Removed MN.setAccessPolicy. Refactored CN.setOwner() to CN.setRightsHolder().

6802 12/16/2011 12:21 PM ben leinfelder

refresh the SystemMetadata entry for EML and referenced data files when parsing EML access rules -- this ensures our in-memory system metadata map is up to date WRT the DB entries.

6800 12/16/2011 11:00 AM ben leinfelder

add revision history to the generated ORE objects -- we use the revision history of the EML package as a basis because the each ORE revision mirrors the revision of the EML package.
Add a placeholder for checking if an equivalent ORE map exists in the DataONE infrastructure - this will be a call to CN.search() that looks at the solr index for OREs based on the EML package ID.

6799 12/16/2011 09:56 AM Chris Jones

Update the parameter names expected for listObjects() to reflect the MN API changes in the architecture docs.

6798 12/16/2011 09:44 AM Chris Jones

Change the query semantics such that we implement the MN.listObjects() where the lower datetime bound is inclusive (greater than or equal to" and the upper datetime bound in exclusive (less than). This allows easier paging in client applications.

6795 12/16/2011 07:42 AM Chris Jones

In the call to MNReplication.replicate(), call back to CNReplication.setReplicationStatus() and set the status to failed when we get local exceptions, exceptions from the source MN when calling getReplica(). Send back an exception with a description when setting the status. Add a private setReplicationStatus() method to refactor these calls out.

6794 12/16/2011 07:36 AM Chris Jones

Modify CNresourceHandler.setReplicationStatus() to use the new API signature, including the failure BaseException that is parsed out of the MMP as a file section. Log the exception message. Since this is an asynchronous call, ReplicationManager won't see a failed status, but the MNAuditTask eventually will.

6793 12/16/2011 07:32 AM Chris Jones

Add collectReplicationStatus() to CNResourcHandler to return the BaseException or it's subclass, if any, provided in the the call to setReplicationStatus. The exception will be reported on the CN.

6792 12/16/2011 07:29 AM Chris Jones

Change setReplicationStatus() to drop serialVersion and report the failure exception message in the CN log.

6787 12/15/2011 01:29 PM ben leinfelder

set SystemMetadata.archived=true on MN.delete
There is ongoing discussion on what the exact behavior should be here, but this mimics Metacat's delete-as-archive action.
http://redmine.dataone.org/issues/882

6786 12/14/2011 08:48 AM Chris Jones

In MNodeService.replicate(), check to see if we have a replica (via an out of band channel) before we call sourceMN.getReplica().

6783 12/13/2011 05:06 PM ben leinfelder

only create guid->docid mapping during metadata replication if it does not already exist
http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5520

6782 12/13/2011 04:50 PM ben leinfelder

do not treat access change as an update -- it should not attempt to retrieve the contents of the object
http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5520

6780 12/13/2011 04:18 PM ben leinfelder

only create guid->docid mapping during data replication if it does not already exist
http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5520

6779 12/13/2011 03:30 PM ben leinfelder

remove xml_acccess.docid reference (oops)
http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5560

6777 12/13/2011 12:22 PM ben leinfelder

updated D1 API -- removed Permission.REPLICATE and associated parameters

6775 12/13/2011 11:34 AM ben leinfelder

process system metadata before access rules (access control is now driven by GUID so the mapping needs to be there)

6774 12/13/2011 11:06 AM Jing Tao

Change the key of query result cache. The key now has the real search value.

6773 12/13/2011 10:58 AM ben leinfelder

include SerialVersion in describe response
https://redmine.dataone.org/issues/2135
NOTE: d1 jars should be replaced once all schema changes are finalized and the generate d1_common code is committed to svn

6766 12/12/2011 10:50 AM ben leinfelder

include 'archived' system metadata element in backing DB store

6757 12/09/2011 09:05 AM Chris Jones

If a member node cannot be found in the node list matching the targetNodeSubject given in isNodeAuthorized(), throw a ServiceFailure exception.

6756 12/09/2011 08:04 AM Chris Jones

Minor reformatting for readability.

6754 12/08/2011 01:51 PM ben leinfelder

update with latest d1_common/d1_lib (includes latest schema changes)

6752 12/08/2011 01:12 PM ben leinfelder

only handle 100 (consecutive!) docId generations per millisecond, otherwise the generated docid part is bigger than Long.MAX_VALUE and Metacat cannot fully handle that.

6750 12/08/2011 11:22 AM ben leinfelder

check previous revision when attempting to update access control with EML 2.0.x docs
http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5560

6749 12/08/2011 10:17 AM ben leinfelder

remove old access rules for a data object when they are being updated by rules contained in an EML document. Now the OnlineDataAccessTest EML 2.1.0 tests pass.
http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5560

6748 12/07/2011 05:31 PM ben leinfelder

construct the proper previousDocId when checking for update permission

6747 12/07/2011 05:05 PM ben leinfelder

for now, look up SystemMetadata directly from the table otherwise we won't have the latest access information. Need to refresh the in-memory copy everytime we edit the access policy via Metacat (includes EML parser)

6746 12/07/2011 05:04 PM ben leinfelder

check previous revision for permissions to update (includes data described by EML)

6745 12/07/2011 05:02 PM ben leinfelder

use correct "rev" column in xml_revisions table

6744 12/07/2011 12:18 PM ben leinfelder

refactor Metacat access handling to be on a per-revision basis so that it more closely aligns with the DataONE approach
http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5560

6743 12/07/2011 12:16 PM Chris Jones

To avoid id generation conflicts happening at the same millisecond, append a 5 character random string to the end of the docid.

6740 12/06/2011 03:03 PM ben leinfelder

retry: add node name in the correct order for predicate navigation
http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5561

6738 12/06/2011 02:50 PM ben leinfelder

add node name in the correct order for predicate navigation
http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5561

6734 12/02/2011 05:32 PM ben leinfelder

handle queries with predicates correctly.
when docids are used in the return field look up, we need to make sure they are included in the values in the correct order for their corresponding parameter place holders (?)

6732 12/02/2011 11:27 AM ben leinfelder

close prepared statement only if not null
http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5562

6728 12/01/2011 03:38 PM Jing Tao

Use accessblock in setaccess method. So user can grant/revoke public readable access.

6727 12/01/2011 01:30 PM ben leinfelder

ensure that the revision list is ordered ascending in case someone changes the sql query without realizing that it matters...

6726 12/01/2011 01:21 PM ben leinfelder

set the byte size of the ORE map before adding it

6725 12/01/2011 01:12 PM ben leinfelder

set/update the obsoletes/obsoletedBy fields in system metadata so that we always have a complete revision history for each object.
Note: ORE maps do not have revision history...yet(?)

6724 12/01/2011 01:10 PM ben leinfelder

order the revision list, ascending.

6723 12/01/2011 11:57 AM ben leinfelder

removing unused class -- can't find a reference to it and it's causing compilation errors for me.

6722 12/01/2011 11:53 AM ben leinfelder

for "all" permission, return a list of READ, WRITE, CHANGE_PERMISSION

6721 11/30/2011 05:31 PM ben leinfelder

generating ORE maps and creating/updating system metadata now. There are some Permission conversion issues to be worked out yet

6720 11/30/2011 05:25 PM ben leinfelder

look up access policy by guid or local id
TODO: resolve the Metacat/EML "all" permission as it relates to DataONE (there is only READ, WRITE, CHANGE_PERMISSION). for now I am using CHANGE_PERMISSION when it is a Metacat "all"

6719 11/30/2011 05:23 PM ben leinfelder

make exception/error reporting clearer -- was getting lock messages when perhaps that was not the correct exception.

6718 11/30/2011 05:22 PM ben leinfelder

look up all docids is now a static method (ORE/SystemMetadata generation)

6717 11/30/2011 02:22 PM Chris Jones

Add log statements for each call to ILock.unlock() for debugging.

6714 11/29/2011 03:43 PM ben leinfelder

evict the HazelCast SystemMetadata entry if we update the access control rules via Metacat's legacy API, otherwise stale SystemMetadata stays in memory instead of being looked up from the backing table store.

6713 11/29/2011 03:41 PM ben leinfelder

optionally include ORE generation/insertion into Metacat when generating SystemMetadata
https://redmine.dataone.org/issues/2056

6712 11/29/2011 02:15 PM ben leinfelder

optionally include ORE generation/insertion into Metacat when generating SystemMetadata
https://redmine.dataone.org/issues/2056

6711 11/29/2011 01:44 PM Chris Jones

Set a default HazelcastInstance after init() is called, and use this instance in getLock() to acquire a lock in the cluster.

6710 11/29/2011 01:41 PM ben leinfelder

no need to cast docInfo entries to String -- they are all strings

6709 11/29/2011 01:39 PM ben leinfelder

set revision history, the create/update dates and the owner/submitter (correctly)

6708 11/29/2011 01:05 PM ben leinfelder

use shared method for looking up "docInfo" map -- both in Metacat replication and in D1 system metadata generation

6707 11/29/2011 12:38 PM ben leinfelder

make default formatting a little bit easier to read

6706 11/29/2011 12:33 PM ben leinfelder

reformat code -- no changes

6705 11/29/2011 12:31 PM ben leinfelder

refactor SystemMetadata creation into separate class from the MetacatHandler -- this will be shared by upgrade code and normal metacat api.

6704 11/29/2011 11:00 AM ben leinfelder

include all document revisions when generating "missing" system metadata
TODO: revision graph captured in obsoletes/obsoletedBy

6703 11/29/2011 10:10 AM Chris Jones

When using ILock.lock(), get a lock on the string value of the Identifier, not the Identifier object itself. Hazelcast locking won't work otherwise.

6702 11/29/2011 08:55 AM Chris Jones

Use the Hazelcast ILock mechanism to lock the system metadata identifier rather than using IMap.lock(pid).

6701 11/28/2011 04:09 PM ben leinfelder

simplify SystemMetadata generation -- will be done during Metacat upgrade for D1 features/support.

6700 11/28/2011 03:53 PM ben leinfelder

clean up populator; use IOUtils library to do string<->stream conversions

6696 11/28/2011 02:22 PM Jing Tao

Set sessionid for clientVeiwBean when it handle a request.
Always set the order type to "allowFirst".

6693 11/23/2011 12:10 PM ben leinfelder

verify checksum when retrieving replica from another member node.
https://redmine.dataone.org/issues/1794

6692 11/23/2011 11:59 AM ben leinfelder

make sure to get/put system metadata to the HZ map instead of using IdentifierManager directly
verified changes for: https://redmine.dataone.org/issues/1999

6691 11/23/2011 11:37 AM ben leinfelder

match documentation for the MN.describe() header
https://redmine.dataone.org/issues/1904

6690 11/23/2011 11:31 AM ben leinfelder

configure synch schedule in the admin screen
https://redmine.dataone.org/issues/1933

6689 11/23/2011 11:03 AM ben leinfelder

look-up sych schedule from metacat properties instead of hardcoding them
https://redmine.dataone.org/issues/1933

6688 11/23/2011 10:19 AM ben leinfelder

when comparing D1 Subject objects, use the equals() method not direct string comparison
https://redmine.dataone.org/issues/2050

6687 11/23/2011 10:07 AM ben leinfelder

access nodeList list correctly
https://redmine.dataone.org/issues/2049

6686 11/22/2011 04:10 PM Jing Tao

When read a FGDC document, Metacat will add a new parameter enableFGDCediting params for the xml transforming.

6684 11/21/2011 02:16 PM Jing Tao

set the uploade file size -1.

6680 11/21/2011 11:21 AM ben leinfelder

allow unknown content sizes
http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5543

6679 11/18/2011 05:00 PM ben leinfelder

run replicate() in a separate thread so that we don't wait for potentially large data objects to be moved around the system.

6678 11/18/2011 12:13 PM Chris Jones

Call replicate() asynchronously.

6677 11/18/2011 09:13 AM Chris Jones

Use status.toLowerCase() to deal with ReplicationStatus conversion issues. This needs to be reviewed.

6676 11/18/2011 09:10 AM Chris Jones

Use Subject.equals() when comparing DNs rather than CertificateManager.equalsDN(). Don't lock the pid in isNodeAuthorized() to debug for timeout issues. Minor debugging changes.

6675 11/17/2011 03:44 PM ben leinfelder

give the Metacat admin users FULL permissions on all data/docs
http://bugzilla.ecoinformatics.org/show_bug.cgi?id=4728

6672 11/17/2011 02:47 PM ben leinfelder

replication control panel now fully implemented as an admin configuration screen
http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5528

6669 11/17/2011 12:19 PM ben leinfelder

move replication configuration actions to the admin servlet and out of the replication servlet
http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5528

6666 11/17/2011 11:19 AM ben leinfelder

save SystemMetadata when replicating data and metadata -- this way if/when the node decides to be a DataONE MN it already has the information needed for each object

6665 11/16/2011 06:15 PM Chris Jones

Minor logging for isNodeAuthorized(), and compare subjects properly. Change this to Subject.compareTo() when it is vetted.

6664 11/16/2011 04:09 PM ben leinfelder

check for authenticated and verified user permissions

6663 11/16/2011 03:34 PM ben leinfelder

throw NotAuthorized when there is no session

6662 11/16/2011 02:27 PM Chris Jones

Catch RuntimeExceptions thrown by Hazelcast as opposed to general Exceptions to we don't catch exceptions we're trying to throw.

6660 11/16/2011 12:58 PM ben leinfelder

get params from multipart params for systemMetadataChanged call