Modify CNresourceHandler.setReplicationStatus() to use the new API signature, including the failure BaseException that is parsed out of the MMP as a file section. Log the exception message. Since this is an asynchronous call, ReplicationManager won't see a failed status, but the MNAuditTask eventually will.
Add collectReplicationStatus() to CNResourcHandler to return the BaseException or it's subclass, if any, provided in the the call to setReplicationStatus. The exception will be reported on the CN.
updated D1 API -- removed Permission.REPLICATE and associated parameters
include SerialVersion in describe responsehttps://redmine.dataone.org/issues/2135NOTE: d1 jars should be replaced once all schema changes are finalized and the generate d1_common code is committed to svn
Minor reformatting for readability.
match documentation for the MN.describe() headerhttps://redmine.dataone.org/issues/1904
run replicate() in a separate thread so that we don't wait for potentially large data objects to be moved around the system.
Call replicate() asynchronously.
Use status.toLowerCase() to deal with ReplicationStatus conversion issues. This needs to be reviewed.
get params from multipart params for systemMetadataChanged call
get pid from normal params, not the URL -- the client should include them in the params -- and not as a serialized "object" since it is just a string value
The ReplicationStatus parameter is 'status', not 'replicationStatus', in the architecture documentation.
Although parameters for setReplicationStatus() are expected as multipart/form-data fields, they seem to be added to the HttpServletRequest as URL parameters during the proxy forwarding in d1_cn_rest_proxy. Test for their existence as multipart fields, but fall back to request params, otherwise, throw an InvalidRequest exception.
lookup stylesheet from metacat.properties for CN list objects and list formats. This is used in conjunction with the CN rest service deployment where the xslt is actually kept.
Since we're using a multipart form to encode parameters, extract the params out of the multipart form prior to doing a get() from the multipartparams map in setReplicationStatus(). Likewise, do the same in updateReplicationMetadata() and setReplicationPolicy().
List the correct REST endpoints for replication in the documentation.
upgrade to 1.0.1-SNAPSHOT DataONE jars
Keep /dirtySystemMetadata as the REST endpoint for systemMetadataChanged() for now.
Fix getReplica() handling code for getReplica() and systemMetadataChanged(). Calls to getReplica() in MNode were calling get(), so the lack of resource handling was being missed.
Handle multipart params where the libclient methods are using them.
Handle calls to CNReplication REST services. Modify handle() to field calls to /replicaPolicies, /replicaMetadata, /replicaAuthorizations, and /replicaNotifications. Add the isNodeAuthorized(), setReplicationPolicy(), setReplicationStatus(), and updateReplicationMetadata() methods to parse and pass multipart form data and params on to the CNodeService implementing class....
Add support for the various CNReplication calls. Add collectReplicationPolicy() to parse the policy out of the multipart form, and collectReplicaMetadata() to parse out the replica to be updated.
move the DataONE 1.0.0-SNAPSHOT
The incoming source node param is just a string, not XML, so don't attempt to deserialize it. Rather, just make a new NodeReference object and set it's value to the value of the incoming param.
add User-Agent logging to support D1 requirements
remove old RestServlet handler -- not used now
MN.setAccess() is a PUT
correctly handle incoming "accessPolicy" parameters for the setAccess() method
swap the parameters for update: existing pid should be first, and then newPid
use "action" parameter instead of "permission"
include checksum algorithm when setting describehttps://redmine.dataone.org/issues/1799
correctly implement MN.describehttps://redmine.dataone.org/issues/1799
return Node not NodeList for getCapabilitieshttps://redmine.dataone.org/issues/1800
collect "message" param from multipart request for MN.synchronizationFailed method
use d1_common_java's date serialization utility for parsing parameters
only create ObjectFormatId when we actually have the parameter (downstream we rely on it being null or not when constructing the query)
changes for schema update (d1_common)
do not parse from last slash ("/") to the end when processing the request uri - otherwise things like format ids (text/csv) will be handled incorrectly.https://redmine.dataone.org/issues/1773
Update classes to use the DataONE 0.6.4 schema and types. Major changes involve using BigInteger vs long in SystemMetadata.size, and using ObjectFormatIdentifier rather than Object format.
latest D1 jars - changes include:updateSystemMetadata() impl for CNnew identifier methods (generate is its own method)removal of the resourceMap pointer from system metadata
use correct enum method
use NodelistUtil
remove ServiceTypeUtil - replace with TypeMarshaller
use new "v1" types from DataONE
remove CrudService -- replaced by MNodeService and CNodeService
MNResourceHandler.getObject() was making a call to ObjectFormat.getFmtid() when an object format was null. Check that it is not null before trying getFmtid().
use objectFormatIdentifier for listObjects()remove provisional system metadata indicator - Metacat will not implement reserveIdentifier()
Modify monitor() to accept new parameters for getOperationStatisics() (fromDate, toDate) instead of 'period'. Added getDateAsUTC() to parse incoming fromDate and toDate parameter strings.
Changed 'guid' to 'pid' to be in line with the D1 API.
simplify the MN rest servlet mapping to match CN mappings - also streamlined the handler code to share extra path info parsing
route all /d1/cn/* traffic through the CNRestServlet/Handler -- inspect the pathInfo when deciding what action to take and also strip off any extra parts (like pid).include query() method pass-through to CNodeService
allow service implementation method to throw exception when guid parameter is null
allow parameters to be omitted in reserveIdentifier handling
use monitor() return value when processing response
throw exceptions up the call stack and handle exception reporting/serialization centrally in the handle() method
use logging, not system.out
consolidate session management and parameter preparation in the superclass
throw exceptions up the call stack rather than catching and handling them differently for each possible rest path
consolidate multi part handling in the super class - subclasses need only call the appropriate helper to get access to the needed resources. superclass does some validation to make sure the files are in place in the request
cleaning up the handlers -- removing superclass methods.
refactor to use D1RestServlet and D1ResourceHandler for the D1 rest interface
deprecate the old rest servlet in favor of the new D1-specific (CN/MN) versions
MN rest servlet, handler and mappings
share getSystemMetadata (before refactoring the superclass)
share getSystemMetadata (before refactoring this superclass)
add CNRestServlet and mappings for urls in the form:/d1/cn/*This extends existing code for CN-specific handling. MN handling will be analogous and is forthcoming
share the deserializer method with subclasses
add shared operations for authorization
share tempDir method with subclasses
make RestServlet and ResourceHandler extendible for D1 CN and MN handlers
use session member variable now that we actually have it
remove CrudService calls in favor of MNodeService
CNCoreImpl is replaced by CNodeService
include latest D1 common/lib changes
Added support in ResourceHandler for the /formats collection. Added listFormats() and getFormat() method, both of which call CNCoreImpl methods to handle the call.
When calling SystemMetadata.getObjectFormat(), return the string value of the ObjectFormatIdentifier rather than ObjectFormat.toString() (which no longer returns the fmtid string).
organize imports so that it is clearer what dependencies exist on the D1 jars
Merged in the D1_0_6_2_BRANCH changes that include the transition from ObjectFormat calls to ObjectFormatCache calls.
include CNCore implementation - only registerSystemMetadata is implemented at the moment. also - updated d1 jar (0.6.2) should be used since that is where the method is defined.would like to consider making ResourceHandler more modular - seems like it does A LOT of different things
do not use XML files for storing SystemMetadata - use DB tables only.
In order to sync up with DataONE 0.6.1 changes, I'm backing out ObjectFormatService changes temporarily in Metacat. Most functionality will be rolled back in using the DataONE 0.6.2 tag, but some methods in ObjectFormatService (such as getListFromDisk()) will be moved into d1_libclient_java.
Changes in the DataONE ObjectFormat class deprecate the convert() method, and we're now using Metacat's ObjectFormatService to look up object format attributes. The following changes replace ObjectFormat.convert() with ObjectFormatService.getFormat() in several classes....
committing changes related to the new restservice update specification (newPid vs. obsoletedGuid)
Use SystemUtil.getContextURL() in ResourceHandler to construct the DataONE service URL (rather than direct calls to PropertyService). This handles http and https URLs, and strips the :80 or :443 for the well known ports.
improved multipart handling (improved logging messages, code, and error checking). Added exception classname to error output when the generic Exception is thrown. Added error check for cases of null value for file parts 'sysmeta' and 'object.'
Removed hardcoded D1 node type in ResourceHandler and added in a new 'dataone.nodeType' property. Also added 'dataone.coordinatingNodeBaseURL' property which points to the CN that stores the authoritative object format list. If this instance of Metacat is a CN, it may point to itself.
ResourceHandler in Metacat was set to return the KNB site URL as the MN base URL rather than the node Id. Fixed. https://redmine.dataone.org/issues/1390
added file extension for txt or csv files
fixed node response bug
fixed update problem
put the pid in the info section of the url
fixed content type problem where csv files were set as text/xml
removed debug statements
remove httpclient 3.1 and custom-built httpclient.jarrework MetacatClient (and other classes) to use httpclient 4updated build to not create httpclient.jarencoding tests now pass.
some new code for debugging mmp
added more code for new mmp requests
bug fixes
refactor checksum and some other stuff
trying to get the new MMP handler working with ResourceHandler