Write the input stream into the file system without alteration in dataone create and update methods.
add support for v2 DataONE API.
switch to use FIleUpload instead of O'Reilly COS library for handling chunked file uploads. https://projects.ecoinformatics.org/ecoinfo/issues/6517
Add a method to get the user management url.
change method name from copy and paste.
Fixed a bug that the configuration page shows complete even though the DataONE configuration hadn't done yet.
Add a util class for dataone configuration.
Reviewed code for all uses of FileInputStream, checking to see if the method should be closing the stream, and if so, closing it in the method as well as in the finally clause to ensure we don't leak file descriptors.
include the dataone.subject (node admin) in the list of administrators. This allows full administrative access to objects when using certificates+d1 api. https://projects.ecoinformatics.org/ecoinfo/issues/6086
use consistent file names and zip content names. Opted for "-" separator so that the zip writer does not remove the unique part of the filename. https://projects.ecoinformatics.org/ecoinfo/issues/6054
use custom FileInputStream subclass to delete the temporary bagit zip when the inputstream is closed (after someone has downloaded the zip).
use standard File.createTempFile() method for uploaded data files and delete them when we are done with them. https://projects.ecoinformatics.org/ecoinfo/issues/6008
allow getlog action to use docid parameters that do not include revision. In these cases, the latest revision will be used.
use secure Metacat context URL for D1 registrationhttps://redmine.dataone.org/issues/3030
use metacat.properties to specify the default checksum algorithm to use -- this way it will be easy for us to switch to whatever DataONE decrees. https://redmine.dataone.org/issues/2834
add a parameter for optionally writing EML-embedded access control rules to the Metacat DB.https://redmine.dataone.org/issues/2584https://redmine.dataone.org/issues/2583
Globally change the property 'dataone.memberNodeId' to 'dataone.nodeId'. This is more useful for both MNs and CNs implemented in Metacat. Also, change D1NodeService.getLogRecords() to return log entries with the actual node id rather than the IP address (looks like a cut/paste error)....
new jars with many changes -- including new CN methods: ping, describe, listChecksumAlgorithm. Removed MN.setAccessPolicy. Refactored CN.setOwner() to CN.setRightsHolder().
only handle 100 (consecutive!) docId generations per millisecond, otherwise the generated docid part is bigger than Long.MAX_VALUE and Metacat cannot fully handle that.
refactor Metacat access handling to be on a per-revision basis so that it more closely aligns with the DataONE approachhttp://bugzilla.ecoinformatics.org/show_bug.cgi?id=5560
To avoid id generation conflicts happening at the same millisecond, append a 5 character random string to the end of the docid.
clean up populator; use IOUtils library to do string<->stream conversions
move the DataONE 1.0.0-SNAPSHOT
use default fmtid if we can't find the user-supplied data mime type in our list
including newer d1 libclient that uses Foresite (and Jena) to construct/parse ORE resource maps for DataONE
cleaned up to use for populating a DataONE MN using the D1 api. Retrieves packages from a Metacat and generates system metadata for them before calling MN.create() for both data and metadata.NOTE: you need a client certificate that the target server accepts (either DataONE-generated for testing or a CILogon one for more official use). I was only able to get the former certificate type to work with our existing MN servers
include SystemMetadata when replicating data and metadata documents -- this allows us to establish the guid-to-docid mapping that is crucial for being able to read the replicated document by guid (d1 api)
do not send <systemMetadata> with the <docInfo> replication information - this is handled by the Hazelcast shared map
changes for schema update (d1_common)
Update MetacatPopulator to use the DataONE 0.6.4 schema and types.
use new "v1" types from DataONE
-handle NotFound object format when generating system metadata-improve EML DML for null entity/attribute elements
placeholder for setting up certificate manager
remove AuthToken (use session). Remove login() call. Use AccessPolicu object to set public read permission
Updated MetacatPopulator to now use ObjectFormatCache.getInstance(). Note: problems remain with the authentication API changes - calls to mn.login(), etc. need to be addressed.
use Data Manager Library to parse EML when needed in DataONE classes.(augmented DML to parse data format elements in EML to estimate MIME type)https://redmine.dataone.org/issues/1634
Merged in the D1_0_6_2_BRANCH changes that include the transition from ObjectFormat calls to ObjectFormatCache calls.
transfer full System Metadata (as XML) during document and data replication
Modified Metacat to build against the D1_SCHEMA_0_6_1 branch of the dataone schemas by incorporating the 0.6.1-SNAPSHOT version of d1_common and d1_libclient libraries, and refactoring Metacat code references to the d1 schema changed types.
In order to sync up with DataONE 0.6.1 changes, I'm backing out ObjectFormatService changes temporarily in Metacat. Most functionality will be rolled back in using the DataONE 0.6.2 tag, but some methods in ObjectFormatService (such as getListFromDisk()) will be moved into d1_libclient_java.
Changes in the DataONE ObjectFormat class deprecate the convert() method, and we're now using Metacat's ObjectFormatService to look up object format attributes. The following changes replace ObjectFormat.convert() with ObjectFormatService.getFormat() in several classes....
zero padded date string in DocumentUtil.generateDocumentId() for readability
Add in correct node references in system metadata.
remove httpclient 3.1 and custom-built httpclient.jarrework MetacatClient (and other classes) to use httpclient 4updated build to not create httpclient.jarencoding tests now pass.
some new code for debugging mmp
updated the populator
always re-write web.xml in case geoserver has been redeployedhttp://bugzilla.ecoinformatics.org/show_bug.cgi?id=4307
Modified MetacatPopulator to deal with change in D1Client static methods.
rework geoserver configuration:-geoserver context is set to 'geoserver' by default, but can be reconfigured-data directory is set in the geoserver web.xml file (we have a template, set the value accordingly, then overwrite the deployed version in the geoserver webapp)...
fixed bug with http/https port
use detected document encoding or Metacat's default encoding (UTF-8)
fixed bug 818
updated the metacatpopulator to use the d1 eml parser to create system metadata in a smarter fashion.
changed improper use of getFS() to a /. getFS() should never be used with a URL since it will return \ on windows which is not a valid separator in a URL.
updated for new dataone jars
fixed identifier manager problem with getting guid for systememtadaa doc
added functionality to set access permissions to system metadata the same as the document that it describes
MetacatPopulator is now working. It allows you to do a query from one metacat instance, then insert any returned docs into another metacat instance
tracked down streaming bug. now need to figure out a better way to fix it.
refactored the sessionService to use a correct singleton initialization scheme. Added true authentication to ResourceHandler.
Merge 1.9.2 changes back into the trunk
Added support to ResourceHandler to allow the putObject method to usearbitrary guid strings as input. These strings are examined, and if theymatch the Metacat docid format (scope.id.rev), they are used directly. Ifthe id is a string in another format, a new localId is generated based on...
Move the chunking of large test element data to centralized location in DBSAXNode.writeChildNodeToDB(). Beff up logging
Pass the doc xml as a string to docImpl.write and writeRepication. This is so a reader can be create for the parsing and for the write to disk. Also created a db access class for xml query result deletion.
Change RequestUtil forwardRequest() method to throw MetacatUtilException.
Add authorization functionality
Add user name to SessionData object
change pw.println to print to remove newline at end of url
Change location of PropertyService to properties directory
Change MetaCatVersion to MetacatVersion
Move document specific utilities to DocumentUtil from MetacatUtil. This makes it easier to define a layer between the core metacat services and the rest of the code.
Create database and shared directories for database management code and shared code respectively.
Add the ability to delete a scheduled workflow (move status to deleted in database)
Created new AuthStub class to always authenticate.
Workflow scheduler and archiving fixes for TPC workflow engine
Added generic scheduler and workflow scheduling functionality.
Add scheduler and workflow schedule functionality
Beef up exception handling from file utilities. Move UtilException to MetacatUtilException to eliminate conflict with similar exception in utility package.
Format indexPaths in metacat.properties. Remove from build.properties and build.xml. Move indexPath list getter from MetacatUtil to SystemUtil.
Separate the concept of a backup file location and a metacat external directory location. These can be totally different.
Allow for backup properties to be written to context based subdirectory of the external configuration directory. This allow multiple instances of metacat to be run side by side.
Change name of writeTempFile to writeTempUploadFile since this instance deals with the temporary upload dir specifically
writeTempFile was trying to write the same file as the registry. Changed to write the file to an "upload" subdirectory. Also throws IOException so errors get reported back to api.
Handle sessions with null ids gracefully.
Get the organization list from org.name.<orgname>= property. Add these values back into metacat.properties.
Try to create the backup directories before creating th ebackup files.
Add external (backup) directory discovery methods.
Renamed MetaCatUtil to MetacatUtil
Add method to save transforming message into files.
Use a while loop to replace if.
Add code to go through eml201 document list to transform them to eml210.
Add a new class which will transform eml201 and eml 200 to eml210.
Discover the external (backup) directory based on OS
Catch login errors and report the details via an exception.
Added code to populate moderators from metacat.properties
Add release info property to provide a brief description of the release (release candidate info mostly)
Rename LDAPUtil to AuthUtil
Add some generic typing
Fix the deploy directory autodiscovery code so that it works regardless of where the metacat application directory is.