Metacat

In order to sync up with DataONE 0.6.1 changes, I'm backing out ObjectFormatService changes temporarily in Metacat. Most functionality will be rolled back in using the DataONE 0.6.2 tag, but some methods in ObjectFormatService (such as getListFromDisk()) will be moved into d1_libclient_java.

Changes in the DataONE ObjectFormat class deprecate the convert() method, and we're now using Metacat's ObjectFormatService to look up object format attributes. The following changes replace ObjectFormat.convert() with ObjectFormatService.getFormat() in several classes....

initialize the HandlerPluginManager

add support for temporal element query in pathquery
http://bugzilla.ecoinformatics.org/show_bug.cgi?id=2084

use debug level for request encoding message

use detected XML encoding when reading/writing files
use UTF-8 as default when performing queries in the DB (assume DB is using UTF-8)
remove as many PrintWriters (uses system default character encoding only) as possible and construct OutputStreamWriters where explicit encoding can be given....

include MetacatHandlerPlugin interface so that other projects [semtools] can extend the actions handled by Metcat without editing the codebase directly. The SemtoolsPlugin is the prototype for this and can be registered using the Metacat admin interface. Note that the plugin implementation class and supporting jars must be included in the webapp in order for the request to be handled correctly (the semtools build is injecting these as part of it's build process).

refactored the sessionService to use a correct singleton initialization scheme. Added true authentication to ResourceHandler.

removed CrudService dependency on servlet params. CrudService is now a singleton. I'm getting an error from metacat saying it can't find teh systemmetadata schema, even though it is, in fact, registered with metacat. need to identify why this is happening.

Modifications to support the DataONE service API version 0.1.0. For DataONE, the get() and
create() services are partially complete. Several more functions and checks need to be added to
create() before it is viable. This DataONE support is not complete, and the current support breaks the MetacatRestClientTest for the time being (this client will eventually be removed).

Merge 1.9.2 changes back into the trunk

Merging in REST interface implementation that was created by Serhan Akin. Main change is
a refactored MetaCatServlet.java class, in which all of the handle* methods were moved
into a separate MetacatHandler.java class. This allows both the standard MetaCatServlet...

Adding missing brackets in else clause. Need to verify with Daigle that this is what he meant.

Move the chunking of large test element data to centralized location in DBSAXNode.writeChildNodeToDB(). Beff up logging

Pass the doc xml as a string to docImpl.write and writeRepication. This is so a reader can be create for the parsing and for the write to disk. Also created a db access class for xml query result deletion.

add method name to log statements

Make sure session data exists before extracting information

Do not register the session service

Add success message to metacat response for setaccess action

Change access section in getaccesscontrol and getdocumentinfo apis to be more eml 2.1.0 compliant. Add a block access option to setaccess api.

change AccessControlForSingleFile to only be instantiated for one file. move ACL methods to AccessControlForSingleFile. Change format of access sections returned to EML 2.1.0.

Move access control source to it's own directory.

Add ability for doc owner to add user access to workflows.

Change RequestUtil forwardRequest() method to throw MetacatUtilException.

Change sanparks skin to log in with browser session instead of metacat client session so session can be validated on other pages.

Make some error messages more understandable

change name of workflowjobid parameter to be workflowjobname which is more accurate

Add session Validation action and session timeout functionality.

Change location of PropertyService to properties directory

Move document specific utilities to DocumentUtil from MetacatUtil. This makes it easier to define a layer between the core metacat services and the rest of the code.

Updated comments to more accurately reflect the API

Create database and shared directories for database management code and shared code respectively.

Add the ability to delete a scheduled workflow (move status to deleted in database)

Implement stop method in services. Stop all services when shutting down metacat. This primarily keeps scheduler from hanging with open threads.

Handle extracting a docid from a urn (lsid) during a read action

Workflow scheduler and archiving fixes for TPC workflow engine

Added generic scheduler and workflow scheduling functionality.

Add archival read funtionality (jar/kar/war files)

remove DataTranspose from core metacat - only used in FIRST skin for this 1.9.1 release

new datamanager feature!
Pivot a result set (rows to columns).
Handles simplest case:
-single id column (observation)
-single pivot column (pivot)

Beef up exception handling from file utilities. Move UtilException to MetacatUtilException to eliminate conflict with similar exception in utility package.

Format indexPaths in metacat.properties. Remove from build.properties and build.xml. Move indexPath list getter from MetacatUtil to SystemUtil.

Replace System.out.println statements with logMetacat statements.

Change name of writeTempFile to writeTempUploadFile since this instance deals with the temporary upload dir specifically

Added exception to error xml to clarify error to api.

Handle sessions with null ids gracefully.

Renamed MetaCatUtil to MetacatUtil

Robustly handle file naming as per bug #2566 and the feedback of the scientific programmers (http://tinyurl.com/4n4jve).

revert the changes for: -- On update, check if the accession number exists in the db. If not, change the action to insert.

Quote filenames as per RFC2616 (http://www.w3.org/Protocols/rfc2616/rfc2616-sec19.html#sec19.5.1), this prevents filename truncation on Firefox for files with spaces within the names (part of #2566).

On update, check if the accession number exists in the db. If not, change the action to insert.

Update metacat api documentation

Rename LDAPUtil to AuthUtil

Remove the code to read and write metadata to disk. It is getting put into DocumentImpl

Add some generic typing

Debug and comment changes

check for existing access control rows before inserting (we do not need duplicate rows specifying the same access control)

Added refreshServices action to allow tests to refresh service caches. This will be expanded to only allow for admins.

Move xml schema specific functionality to XMLSchemaService

replicate xml_access entries across servers.
see: http://bugzilla.ecoinformatics.org/show_bug.cgi?id=3464
note: still needs to be tested...but needed to be checked in for other metacat installations to be updated....

adding document "cart" feature to metacat. will not be highly published as a feature until well-vetted.

set the sessionId from the SessionData object

Created a method in DocumentImpl to read a document file from disk. Created a support method to strip inline data for users that don't have read access.

Add functionality to write and read metadata documents to and from the file system.

remove skin.configs directory.
migrate those properties to the style/skins/* properties files
do not load them during Metacat init.

Get session information from RequestUtil. Change some variable names to camel case.

add ability to access protected data via the datamanager/dataquery feature.
Note: the datamanager features rely on the ecogrid stubs being available in Metacat. When Metacat is installed with ecogrid enabled, the required jars are included in the war...something to keep in mind.

add dataquery (from EML's Data Manager library) to the metacat servlet actions

qualify xml and eml properties with an xml. prefix

Continue to qualify property names

Change sitemapInterval property to be sitemap.interval. Add testing property

Separate the init function to pre-confiugured and post-configured methods. the initSecondHalf method allows for the intial configuration of metacat without a restart.

Fully qualify spatial and replication properties

Use qualified properties

Add functionality to allow developer to bypass configuration utility

Use SystemUtil.getContextDir() instead of constructing directory path.

Differentiate between tomcat directory and deployment directory.

Create PropertyService using log directory instead of servlet context.

Change default-style property to application.default-style

Merge 1.9 changes into Head

fixed bug 3403

added support to upload an xml document using multipart encoding. the enables better support for web interfaces that wish to use multipart to upload xml documents. the action is 'insertmultipart'

made it so that the error and success messages from a multipart upload can be transformed using xslt. also added upload functionality to the sms skin

Check into cvs head. Add a debug statement for printing out eml document

Add to cvs headd. Add code to correct invalidated eml 201 doucments.

Enable file naming when returning a zip file of the requested documents.
File names for each docid are provided as additional parameters named by the actual docid value that they are providing a name for. An example query string might look like:

action=read&docid=doc.1.1&docid=doc.2.1&qformat=zip&doc.1.1=fileName1.txt&doc.2.1=fileName2.jpg...

Using reponse.getOutputStream to replace reponse.getWriter in query and read method. So metacat can send correct special character back.

ignore commented out DOCTYPE declarations (don't force validation on them)

Modification for changes to the field "MetacatUtil.skinconfigs".

Fixed bug that should throw an exception in upload method.

Change the debug level for Metacat initialization message.

Added param to indicate if public has read access or not, in the method "readFromMetacat". It only does this if reading a metadata document, for a skin-format,
and the user is not public. This is useful to the skin's metadata display/editing form, for indicating public accessiblity (or inaccessibility)....

Find out if the document is a Member of an EML package, and if that's not the case, than skip the test to see if access is set through EML metadata.

Refactored upload code to allow arbitrary POST parameter ordering, and open the door to multiple files in one HTTP request. Two new methods to MetaCatUtil to facilitate the file processing: writeTempFile and copyFile.

Backing out change, certain cases cause the file to upload as 0 bytes, which pass all tests but leave us with no data. Will re-commit after 1.8 release and more complete testing.

Updated com.oreilly.servlet (cos.jar) from 19Jun2001 to current 05Nov2002 version. Fixed uploading in MetaCatServlet.java to allow parameters after file object, was disabled for a workaround in the previous library.

Add a delay time for initial indexing

Add debug in query method.

Add statement to output search time to a delimitered file

Add new method to record the total search time to a file.