Metacat

Merging in changes made in branch 'dataaccess' by Jing Tao.

Added upload functionality to metacat client for upload online data

Add warning if default harvest list file is not found on disk.

Change relative path to harvestList.xsd schema to allow harvest list editor to run standalone.

Changed path for eml xsd stylesheet

Modified directory name for eml-2.0.x checkouts to match with that in build.xml

Removed some debugging code added in last commit

Added code to read EML2.0.1 into source code.

Made change in the registry so that new documents produced are of eml-2.0.1 standard.
Also made changes in create-registry folder

Added code to make enteries for eml 2.0.1 into the database

Improvements to Harvester registration web form.

Add diagnostic output

Get rid of some useless code.

Improve handling of default values.
Call fileNew() if no default Harvest List file is specified.
Improve documentation for default properties.

Fixed schema loading scripts to use the new ant token for the EML2 namespace.

Add File Validate menu item
Add column to display the row number
Add name of file to the title bar
Load properties from .harvestListEditor in home directory instead
of from metacat.properties

Harvest List Editor, a GUI tool for editing Harvest List XML files

Fixed a bug which was introduced because 'eml2namespace' was replaced by 'eml2_0_0namespace' in metacat.properties

Adding a new perl script. Running this script creates a new skin.

Store session attributes as username and password for consistency with Metacat

Improvements to Harvester registration form

Made initial changes for accepting EML version 2.1.0 in Metacat

Revised the api and ecogrid can use it easily.

Fixed a bug that couldn't find eml2 data format.

Improve Harvester registration login

fixed the bug that couldn't assign access rule for the data file come with morpho.

Change method accessNumberUsed to static method.

Change eml2 to eml200

Change the code when Eml parser name changed.

New class to replace EmlSAXHandler.

Rename this class to Eml200SAXHandler.

Integrate Harvester properties with build.properties and metacat.properties.
Eliminate the need for a separate harvester.properties file.
Improve email message to the site to include more summary information.

Fixed bug# 1309. Also put a regex check on ending year and beginning year.

Made changes to fix bug # 1365.

Made changes to fix bug# 1303 and 1367.(Both are related)
Following variables can now be specifieD from the cfg file to control behaviour of the skin: hasKeyword, hasMethod, hasSpatial, hasTaxonomic and hasTemporal. Following two also can be used: spatialRequired and temporalRequired

Made changes to fix bug# 1303 and 1367.(Both are related) - corresponding changes need to be made in tmpl file and testing needs to be done before bugs can be closed.

Made changes to fix 1301. Works fine with eml2 - need to check if works with emlbeta6.

Refactored a number of methods to allow for more effective JUnit testing

Made changes to control whether a skin wants to use keywords or not. Keywords probably will be used by all skins alsways. But similar changes can be made for Geographic Coverage or Temporal COverage to control whether a particular section of the insert data template needs to be shown or not. This can be controlled from .cfg file. For example, in nceas.cfg is you define hasKeyword = false, none of the templates will show the keyword part. (Unless, you are using NCEAS skin and you try to modify a document which already has keywords.)

Removed beta6 specific code which is no longer being used.

Removed repetition of code. Also fixed the link to the modified file submitted from responseTemplate

Changes to comments

Replace Harvester dbDriver property with database property

Added ou also to getprincipal output. So now organizationUnitName is also returned in getprincipal output.

Made changes so that PISCO and NRS trees are also returned in getPrincipals action.

Allow user to set frequency to 0 for testing purposes

Modify schema location

Fixed getprincipals output. Earlier DNs for LTER were in URL form. Now they appear in DN form.
Earlier:
ldap://knb.lternet.edu:389/DN,base
Now:
DN

Have to make changes so that PISCO and NRS also show up.

Fixed two small bugs.

Made changes in XML that is sent back as result of action=getprincipals. Changes are as following:

1. <authSystem> tag contains organization attribute which specifies organization name
2. <user> tag now has <organization> tag which specifies the organization name of the user. This is being generated as null for LTER as of now.

Perl script for transfering documents from one metacat to another metacat. list_of_docids contains list of names of the documents that will be transfered.

Rename url property to defaultDB

Added the EventLog functionality to the servlet as action=getlog. Valid parameters are ipAddress, principal, docid, event, start, end. All parameters can be repeated except start and end.

Correct minor problems with foreign keys in Harvester tables

Partial implementation of a query facility for the access log. Need to fix problems with the date filtering, and tie this into the servlet action when completed and tested.

Interim checkin of EventLog reporting structure. Still basically incomplete but compiles.

Add three new Harvester tables

Omit HARVEST_OPERATION_TABLE and store harvest operations in properties file instead

Fixed the returndoctype bug in DBQuery as described in bug 1451. Now,
results are returned even if returndoctype is null, which is the
appropriate behavior.

Implement sending email reports to harvest administrator and site administrator

Patched to make the date-time value insertion work under oracle. Now it
tests fine under both oracle and postgres. Still need to write an
access/query function.

Added additional logging calls for read, upload, insert, update, delete, and
other commands. Now need to get a reporting function working and this
bug 101 will be largely finished.

Modified EventLog to eliminate the separate revision column and instead fold
that into the docid string. Log now includes read events, but need to update
the data sent there.

Modified SQL scripts to add the new access_log table. I've partially tested this under postgres but have not yet tested under oracle.

Reformatted code for readability and consistency.

Added javadoc that was missing for the EventLogData class.

Added classes for basic event logging support. These classes write log entries to the database connection. Still to do: query the event log, write oracle sql code (postgres is done), possibly allow the log to be archived to a text file, and possibly do the dtabase writes asynchronously via a caching mechanism.

Made change in result tag that is returned in getprincipals action. Replaced userdn with username.

Added GPL license text to harvester source files.

Fixed a bug the text filed in path xml query maybe splited.

Fixed a bug if a document already has index, but now metacat doesn't use index. If you try to delete this one, will cause error.

Clean up some code.

Revise the handle query and squery method, not finished yet.

Add a new transform method.

Add a new transferm method for string reader.

Implement harvester scheduing capabilities and sending email reports to the Harvester Administrator

Validate user input data in registration servlet

Improve authorization logic for single file uload servlet

Add delete capability to single file upload servlet

Set content type properly for getversion action.

Added a simple action for finding out the current metacat version. The information is pulled from the property file, so be sure to update the version in the property file before shipping a release.

Fixed a bug in the new recursive SQL statements that prevented them from operating on oracle. Now they should work on both oracle and postgres.

Reformatted source for readability and conformance.

Excluded the update of the xml_index table in insert,update, and delete if the usexmlindex option is set to false.

Reformatted DBQuery.java for readability and consistency.

Changed printAttributeQuery() to no longer depend on the xml_index table. Now, if useXMLIndex is false, the method will use a recursive query on xml_nodes to match paths instead of using xml_index.

Some refactoring of the printExtendedSQL()
methods to eliminate redundancy in the code and
to provide more readable documentation.

Moved decision about whether to use xml_index for
query to the metacat.properties as "usexmlindex". Default
is now false. Still have some refactoring to do to remove a
few more uses of the xml_index table.

Created new printExtendedSQL function that can get the extended
return fields using only xml_nodes and not xml_index. One
step closer to eliminating the dependence ofn xml_index.

Created test class QuerySpecificationTest and started process of removing the xml_index from the QuerySpecification code. Reformatted some classes for readability.

Removed the "getdataguide" action as it was always experimental and is no longer used in metacat or morpho.

removed the 3 versions of printPackageSQL which are no longer used by metacat. This eliminates some dependency on xml_index.

Implement Harvester Registration Servlet

Additional scheduling logic

Changes to Harvester properties

Changes to log entries

Made changes in these three files so that getPrincipal returns back more details about users and groups. Users now contain userDN, user Name and user Email. Groups now contain Group name and Group description. So some function calls which returned single string array earlier now returns multiple string arrays. AuthSession.java called one of these functions - so accordingly changes were made to fix that part of the code.

Fixed a bug which doesn't occur everytime. The bug occurs when lastid in (cfg).lastid is not unique. i.e. there is already a document in the server which has same docid. The earlier algorithm was:

while(not unique){
get accession number;
update accession number written in document; (BUG)...

Added extra parameters (cfg & docid) to be passed on to generic response screen.

made changes so that organization name for the site or project is entered before
any other creater.