add a parameter for optionally writing EML-embedded access control rules to the Metacat DB.https://redmine.dataone.org/issues/2584https://redmine.dataone.org/issues/2583
additional debug logging for tracking down MN replication errors
change ordering of getLogRecords() parameter -- pidFilter is in the middle now
upgrade to latest RC in libclient and common jars -- includes updated getLogRecords and new mn.generateIdentifier method
Don't check for populated obsoletes and obsoletedBy fields during CN.create(), only MN.create(). The CN should expect that the MN has populated this field because of existing revision information, and should trust the MN information. Addresses https://redmine.dataone.org/issues/2507.
use isAdminAuthorized() to check access to CN.create(). Note this method takes a pid and permission parameter and neither is used. Also removed the NotFound exception because it would never come up.
include CN.delete()https://redmine.dataone.org/issues/2506
Add some debugging statements in isAuthorized().
check for session when checking administrative authorization
Incorporate isAdminAuthorized() into isAuthorized() for blanket CN access to objects.
transitive properties for mapped subjects:-group membership-verified flaghttps://redmine.dataone.org/issues/2430https://redmine.dataone.org/issues/2432
check group membership defined at group level (in addition to membership defined as part of of the Person level)https://redmine.dataone.org/issues/2429
logging for permission checks - trying to nail down details of MN checking
use Event.CREATE.xmlValue() when converting "insert" to "create" http://redmine.dataone.org/issues/2471
translate "insert" events in Metacat as Event.CREATE events ("create") for DataONEhttps://redmine.dataone.org/issues/2461
log record paging:-use start and count parameters-if start+count exceeds the total number of records, then only return from start to the end of the list-if start exceeds total record count, start at the end of the list (will be empty list)https://redmine.dataone.org/issues/2458
catch additional NotFound exception for: "do not include log entries for documents that the caller is not allowed to read." https://redmine.dataone.org/issues/2444
do not include log entries for documents that the caller is not allowed to read. https://redmine.dataone.org/issues/2444
use revision provided in the docid when looking up guid. had been using latest revision which I think incorrectly reports on the log history.noticed this when looking at: https://redmine.dataone.org/issues/2444
A minor change to isAuthorized() - compare each Person in the SubjectInfo (not just the primary Subject) since each person could have an equivalent identity mapped to the primary Subject. Add debug logging for the comparison.
added debug logginghttps://redmine.dataone.org/issues/2429
check if verified flag is null before evaluating (NPE during MN Auth test)https://redmine.dataone.org/issues/2429
Globally change the property 'dataone.memberNodeId' to 'dataone.nodeId'. This is more useful for both MNs and CNs implemented in Metacat. Also, change D1NodeService.getLogRecords() to return log entries with the actual node id rather than the IP address (looks like a cut/paste error)....
throw InvalidToken when an invalid Permission is passed in. THis requires that internal calls to the method also check for this exception.https://redmine.dataone.org/issues/2388
only generate system metadata when the call comes from the legacy Metacat API, not the D1 API.https://redmine.dataone.org/issues/2362 (I think this was the culprit)
remove ID mapping when a create()/"insert" call fails so that subsequent calls do not return an IdentifierNotUnique error. In this case it was due to invalid XML.https://redmine.dataone.org/issues/2341
use RC-3 DataONE jars and fix compilation error that arose. https://redmine.dataone.org/issues/2351
Update D1NodeService to reflect new ObjectFormatCache signature.
use updated authorization policies as discussed in:https://redmine.dataone.org/issues/2277andhttp://epad.dataone.org/20120131-authn-authz-questions
use UTC serialization for log entries so that the timestamp, not just the date, is preservedhttps://redmine.dataone.org/issues/2257
Don't throw a NotAuthorized exception in isAdminAuthorized() - just return false.
Add isAdminAuthorized() to D1NodeService to check if the operation is being requested from a CN. Consult the NodeList from the CN and test the NodeType of the given node and the X509 certificate Subject. Perhaps we should expand this to also check for service-level access in the future.
After reviewing CNodeService and D1NodeService prompted by Robert comparing the Hazelcast locking with the d1_synchronization locking, I've made a number of changes that will prevent locking problems:
1) Multiple methods contained try/catch blocks that would:...
exapnd permissions on the exisiting access rule not on the permission being checked. (hierarchical permissions)
interpret permissions as hierarchicalhttps://redmine.dataone.org/issues/2150
allow other Metacat process (system metadata and ORE generation) to directly insert objects and system metadata without having to go through the MN/CN methods.
new jars with many changes -- including new CN methods: ping, describe, listChecksumAlgorithm. Removed MN.setAccessPolicy. Refactored CN.setOwner() to CN.setRightsHolder().
update with latest d1_common/d1_lib (includes latest schema changes)
make exception/error reporting clearer -- was getting lock messages when perhaps that was not the correct exception.
when comparing D1 Subject objects, use the equals() method not direct string comparisonhttps://redmine.dataone.org/issues/2050
check for authenticated and verified user permissions
correct typo
D1NodeService get(), getSystemMetadata(), and isAuthorized() no longer throw InvalidRequest.
make MNodeServiceTest pass JUnit testing
Modify isAuthorized() to get the most up to date system metadata from the hzSystemMetadata map.
Update getSystemMetadata() to lock(); get(); unlock() to ensure we have the latest version of system metadata from the hzSystemMetadata map. Remove the setAccessPolicy() method since it is being deprecated in the MNAuthorization API.change insertSystemMetadata() to use a finer grained Date object on insertion. Locking of the pid happens in the subclass prior to the insert.
move the DataONE 1.0.0-SNAPSHOT
add User-Agent logging to support D1 requirements
update D1 jars to include recent SubjectList -> SubjectInfo refactoring and the SUBJECT_PUBLIC constant
throw InvalidToken when the Session parameter is null for create()https://redmine.dataone.org/issues/1850
do not allow system metadata to have obsoletes or obsoletedBy fields when calling the create() method -- these are only allowed for updates so that we do not subvert object versioning by [un]knowingly submitting system metadata that directs one id to another.
set sysmeta submitter based on the subject given in the certificate
log errors on create() and registerSM
catch runtime exceptions that arise from hazelcast storage errors in the system metadata map
implicit success for setting accessPolicy - trust that the MapStore persists the updated system metadata
only "save" to the shared system metadata map - not directly to the table store.
rely on Hazelcast to store the SystemMetadata locally for the node. Entry event listeners store the shared system metadata on their local node when alerted. TODO: remove old replication code that included system metadata xml when replicating scimeta and data
verify that the sysmeta checksum value matches the computed checksum value when calling create()https://redmine.dataone.org/issues/1795
check for null pointers when adding system metadata/creating records during cn.create()
make isScienceMetadata() method public/static to be called throughout Metacat
check for null session before looking at subject
check session != null before checking authorization
Catch D1nodeService up to the DataONE 0.6.4 schema where there is no ObjectFormat.isScienceMetadata() method, but rather ObjectFormat.getFormatType() where type is currently one of 'DATA, 'METADATA', or 'RESOURCE'.
add getReplica() implementation. same as get() but with different logging. seems silly, but maybe I missed something important that distinguishes this method.
Update classes to use the DataONE 0.6.4 schema and types. Major changes involve using BigInteger vs long in SystemMetadata.size, and using ObjectFormatIdentifier rather than Object format.
refactor Constants
use new "v1" types from DataONE
In D1NodeService.getLogRecords(), don't pass in null start and count params - set them to the defaults (0 and 1000).
use objectFormatIdentifier for listObjects()remove provisional system metadata indicator - Metacat will not implement reserveIdentifier()
simplify the get() method -- no need to use temp files for this operation
implement d1 paging for the log record results
save systemmetadata when create() is called
Add the missing URL delimiter when building the D1 base service URL.
handle data objects (not sci meta) and also set the resulting pid so that create() can succeed
beef up isAuthorized method to check for "public" access rules and also for the rights holder
add space to error message
implement reserveIdentifier() and check whether the id is reserved when creating records (only allow the create when the Subject creating matches the Subject who reserved it -- currently stored in rightsHolder)
remove extraneous update() call when create() does the call for us
Make isScienceMetadata() protected for access from subclasses.
Add insertSystemMetadata() to D1NodeService, wrap the exception handling from calls to IdentifierManager.
Add updateSystemMetadata() to D1NodeService as a helper method to wrap the exception handling from calls to IdentifierManager.
At Ben's suggestion, add metacatUrl to D1NodeService and make it available to subclasses. Set the metacatUrl in the constructor using SystemUtil rather than all roll your own PropertyService calls. More concise. Also, log the delete event in MNodeService.delete().
isAuthorized: check for nulls in Session subjects, catch any unexpected errors and deny access when in doubt
Remove setParamsFromRequest() from D1NodeService. This was called (previously as CrudService) from ResourceHandler, but will be deprecated in favor of manually creating a param map for each method that needs to pass params on to MetacatHandler.
Implement [MN|CN]Storage.create() in D1NodeService. Since MetacatHandler requires an IP for event logging, we pass in the metacat URL (hold over from CrudService). To do this in the abstract D1NodeService, change the constructors to take metacatUrl as a parameter and get the URL from the metacat properties file in getInstance() of the subclasses. Needs testing.
Change Metacathandler.read() to be public since it's internal to Metacat, and use read() in D1NodeService after isAuthorized() for the calling Subject from the Session object.
Minor formatting changes - tabs to spaces, indents, etc.
Implement [MN|CN]Read.get() in D1NodeService. Added setParamsFromRequest() to pass through parameters from the request object. Since the D1 Authorization API doesn't specify which authentication system a subject belongs to, we don't know if the subject listed is a KNB LDAP DN. isAuthorized() may return true for a mapped identity, but we don't know the DN of the KNB identity per se. This needs to be tested.
-use every Subject in the session (alt Ids and Group membership)-consolidate to single isAuthorized method
implement getChecksum() in the superclass
use Permission, not Event for isAuthorized() methods
implement CNAuthorization
implement getLogRecords
Initial check in of the MNodeService stub methods that implement the D1 MN* interfaces. CrudService methods will be transitioned into this class. The methods follow the D1 0.6.2 API thus far.
Also changed CNodeService to reflect minor changes to the D1NodeService class.
CNAuthorization.isAuthorized() and MNAuthorization.isAuthorized() differ. Removed it from the base class.
getChecksum() is intentionally different on the CN vs MN, so I'm removing it as a common method.
Initial check in of the D1NodeService class that provides methods common to both CNodeService and MNodeService implementation classes. The common API methods are:
Methods common to CNCore and MNCore APIsgetLogRecords()
Methods common to CNRead and MNRead APIs...