Add the methond named isAuthoritativeMNodeAdmin method. It applies to both CN and MN methods.
Fixed a bug that the group information couldn't be retrived from the session.
comment out the index queue call when archive() is called - I think it is causing the duplicate events for the listener. https://projects.ecoinformatics.org/ecoinfo/issues/6030
correct regex for whitespace in D1 identifier.
use an independent ISet<SystemMetadata> structure to communicate objects that should be indexed by metacat-index. https://projects.ecoinformatics.org/ecoinfo/issues/5943
limit /log and /object calls to configurable maximum count for paging. defaults to existing Metacat value of 7000
no need to mark SM as archived now that DocumentImpl.delete() does it.https://redmine.dataone.org/issues/3406
Implement MNQuery for "pathquery" engine. Optionally include guid in the pathquery results (https://redmine.dataone.org/issues/3083)
include the subjects we are testing for authentication.https://redmine.dataone.org/issues/2778
make sure data objects correctly use force replicate with action "insert" https://redmine.dataone.org/issues/3138
check if the caller is the Node admin (the member node calling itself) as well as the existing check for the CN calling the service. Both of those callers should be given full admin rights.
check for invalid (!) pids. thanks, M. Reyes for catching thishttps://redmine.dataone.org/issues/3047
check for whitespace in identifiers during create() and update()https://redmine.dataone.org/issues/3047
first pass: DataONE-specific log retrieval to avoid java-based post-processing.
Add debug logging to delete() to understand why we're getting InsufficientKarmaException.
Since we already have determined access via isAuthorized() and isAdminAuthorized(), act as the Metacat administrator during calls to DocumentImpl.delete() in archive(), passing in null username and group.
restrict getLogRecrods (both MN and CN) to be called only by admin users (the CN)https://redmine.dataone.org/issues/2855
include pidFilter handling - only matches the complete pid. Issues a warning in the Metacat logs when pidFilter cannot be applied but allows the call to getLogs() to return as though there was no pidFilter given.https://redmine.dataone.org/issues/2798
handle authorization for delete() differently for CN vs MN.On the CN, only the CN (or tbd admin user) can call it.On the MN, both the CN (or admin user) and the same MN can call it.
add Session-less archive() method
only admin users can call MN/CN.delete(). This is limited to any CN and only the MN that is calling itself
update the sysmeta data modified when setting archived=truehttps://redmine.dataone.org/issues/882
optionally remove the document/data file from the filesystem completely when 'deleting' it.https://redmine.dataone.org/issues/2677
newer d1 jars that include shared AuthUtilsmethod for isAuthorized() consistencyhttps://redmine.dataone.org/issues/2661
implement MN and CN.archive() method -- really just the existing delete() methods.https://redmine.dataone.org/issues/2674https://redmine.dataone.org/issues/2675
call MN.delete() for each replica when CN.delete() is calledhttps://redmine.dataone.org/issues/2676
defer to AuthUtils for flattening out the equivIdent subject list.https://redmine.dataone.org/issues/2661
check normal access control rules for getSystemMetadata before deferring to MN replica information that may grant MNs additional access to the SM.https://redmine.dataone.org/issues/2656
remove extraneous pid and permission parameters from isAdminAuthorized() method and make public so that it can be called in other locations - namely before our asynchronous replicate() implementation on the MN.
check for empty null (missing) node.subjectList. This should probably be a required element in the D1 schema, but it appears not. (ORNL entry was missing subjects in cn-dev environment)
needed to initialize the nodeList that stores matching nodes (by subject) -- this was the source of a NPE when we had a matching node subject.
As Ben suggested, don't compare to the node list if there are no replicas listed. This reduces the number of calls to listNodes() on the CN.
Minor logging change in throwing ServiceFailure when Hazelcast throws a RuntimeException.
Modify getSystemMetadata() to allow nodes that are listed as replicas to access the system metadata. Use the Session.Subject to find a list of nodes from the CN that match the subject, and compare those node ids to the listed replica node ids. Add listNodesBySubject() helper method to do so.
add a parameter for optionally writing EML-embedded access control rules to the Metacat DB.https://redmine.dataone.org/issues/2584https://redmine.dataone.org/issues/2583
additional debug logging for tracking down MN replication errors
change ordering of getLogRecords() parameter -- pidFilter is in the middle now
upgrade to latest RC in libclient and common jars -- includes updated getLogRecords and new mn.generateIdentifier method
Don't check for populated obsoletes and obsoletedBy fields during CN.create(), only MN.create(). The CN should expect that the MN has populated this field because of existing revision information, and should trust the MN information. Addresses https://redmine.dataone.org/issues/2507.
use isAdminAuthorized() to check access to CN.create(). Note this method takes a pid and permission parameter and neither is used. Also removed the NotFound exception because it would never come up.
include CN.delete()https://redmine.dataone.org/issues/2506
Add some debugging statements in isAuthorized().
check for session when checking administrative authorization
Incorporate isAdminAuthorized() into isAuthorized() for blanket CN access to objects.
transitive properties for mapped subjects:-group membership-verified flaghttps://redmine.dataone.org/issues/2430https://redmine.dataone.org/issues/2432
check group membership defined at group level (in addition to membership defined as part of of the Person level)https://redmine.dataone.org/issues/2429
logging for permission checks - trying to nail down details of MN checking
use Event.CREATE.xmlValue() when converting "insert" to "create" http://redmine.dataone.org/issues/2471
translate "insert" events in Metacat as Event.CREATE events ("create") for DataONEhttps://redmine.dataone.org/issues/2461
log record paging:-use start and count parameters-if start+count exceeds the total number of records, then only return from start to the end of the list-if start exceeds total record count, start at the end of the list (will be empty list)https://redmine.dataone.org/issues/2458
catch additional NotFound exception for: "do not include log entries for documents that the caller is not allowed to read." https://redmine.dataone.org/issues/2444
do not include log entries for documents that the caller is not allowed to read. https://redmine.dataone.org/issues/2444
use revision provided in the docid when looking up guid. had been using latest revision which I think incorrectly reports on the log history.noticed this when looking at: https://redmine.dataone.org/issues/2444
A minor change to isAuthorized() - compare each Person in the SubjectInfo (not just the primary Subject) since each person could have an equivalent identity mapped to the primary Subject. Add debug logging for the comparison.
added debug logginghttps://redmine.dataone.org/issues/2429
check if verified flag is null before evaluating (NPE during MN Auth test)https://redmine.dataone.org/issues/2429
Globally change the property 'dataone.memberNodeId' to 'dataone.nodeId'. This is more useful for both MNs and CNs implemented in Metacat. Also, change D1NodeService.getLogRecords() to return log entries with the actual node id rather than the IP address (looks like a cut/paste error)....
throw InvalidToken when an invalid Permission is passed in. THis requires that internal calls to the method also check for this exception.https://redmine.dataone.org/issues/2388
only generate system metadata when the call comes from the legacy Metacat API, not the D1 API.https://redmine.dataone.org/issues/2362 (I think this was the culprit)
remove ID mapping when a create()/"insert" call fails so that subsequent calls do not return an IdentifierNotUnique error. In this case it was due to invalid XML.https://redmine.dataone.org/issues/2341
use RC-3 DataONE jars and fix compilation error that arose. https://redmine.dataone.org/issues/2351
Update D1NodeService to reflect new ObjectFormatCache signature.
use updated authorization policies as discussed in:https://redmine.dataone.org/issues/2277andhttp://epad.dataone.org/20120131-authn-authz-questions
use UTC serialization for log entries so that the timestamp, not just the date, is preservedhttps://redmine.dataone.org/issues/2257
Don't throw a NotAuthorized exception in isAdminAuthorized() - just return false.
Add isAdminAuthorized() to D1NodeService to check if the operation is being requested from a CN. Consult the NodeList from the CN and test the NodeType of the given node and the X509 certificate Subject. Perhaps we should expand this to also check for service-level access in the future.
After reviewing CNodeService and D1NodeService prompted by Robert comparing the Hazelcast locking with the d1_synchronization locking, I've made a number of changes that will prevent locking problems:
1) Multiple methods contained try/catch blocks that would:...
exapnd permissions on the exisiting access rule not on the permission being checked. (hierarchical permissions)
interpret permissions as hierarchicalhttps://redmine.dataone.org/issues/2150
allow other Metacat process (system metadata and ORE generation) to directly insert objects and system metadata without having to go through the MN/CN methods.
new jars with many changes -- including new CN methods: ping, describe, listChecksumAlgorithm. Removed MN.setAccessPolicy. Refactored CN.setOwner() to CN.setRightsHolder().
update with latest d1_common/d1_lib (includes latest schema changes)
make exception/error reporting clearer -- was getting lock messages when perhaps that was not the correct exception.
when comparing D1 Subject objects, use the equals() method not direct string comparisonhttps://redmine.dataone.org/issues/2050
check for authenticated and verified user permissions
correct typo
D1NodeService get(), getSystemMetadata(), and isAuthorized() no longer throw InvalidRequest.
make MNodeServiceTest pass JUnit testing
Modify isAuthorized() to get the most up to date system metadata from the hzSystemMetadata map.
Update getSystemMetadata() to lock(); get(); unlock() to ensure we have the latest version of system metadata from the hzSystemMetadata map. Remove the setAccessPolicy() method since it is being deprecated in the MNAuthorization API.change insertSystemMetadata() to use a finer grained Date object on insertion. Locking of the pid happens in the subclass prior to the insert.
move the DataONE 1.0.0-SNAPSHOT
add User-Agent logging to support D1 requirements
update D1 jars to include recent SubjectList -> SubjectInfo refactoring and the SUBJECT_PUBLIC constant
throw InvalidToken when the Session parameter is null for create()https://redmine.dataone.org/issues/1850
do not allow system metadata to have obsoletes or obsoletedBy fields when calling the create() method -- these are only allowed for updates so that we do not subvert object versioning by [un]knowingly submitting system metadata that directs one id to another.
set sysmeta submitter based on the subject given in the certificate
log errors on create() and registerSM
catch runtime exceptions that arise from hazelcast storage errors in the system metadata map
implicit success for setting accessPolicy - trust that the MapStore persists the updated system metadata
only "save" to the shared system metadata map - not directly to the table store.
rely on Hazelcast to store the SystemMetadata locally for the node. Entry event listeners store the shared system metadata on their local node when alerted. TODO: remove old replication code that included system metadata xml when replicating scimeta and data
verify that the sysmeta checksum value matches the computed checksum value when calling create()https://redmine.dataone.org/issues/1795
check for null pointers when adding system metadata/creating records during cn.create()
make isScienceMetadata() method public/static to be called throughout Metacat
check for null session before looking at subject
check session != null before checking authorization
Catch D1nodeService up to the DataONE 0.6.4 schema where there is no ObjectFormat.isScienceMetadata() method, but rather ObjectFormat.getFormatType() where type is currently one of 'DATA, 'METADATA', or 'RESOURCE'.
add getReplica() implementation. same as get() but with different logging. seems silly, but maybe I missed something important that distinguishes this method.
Update classes to use the DataONE 0.6.4 schema and types. Major changes involve using BigInteger vs long in SystemMetadata.size, and using ObjectFormatIdentifier rather than Object format.
refactor Constants