Metacat

Integrate Harvester properties with build.properties and metacat.properties.
Eliminate the need for a separate harvester.properties file.
Improve email message to the site to include more summary information.

Refactored a number of methods to allow for more effective JUnit testing

Changes to comments

Replace Harvester dbDriver property with database property

Added ou also to getprincipal output. So now organizationUnitName is also returned in getprincipal output.

Made changes so that PISCO and NRS trees are also returned in getPrincipals action.

Allow user to set frequency to 0 for testing purposes

Modify schema location

Fixed getprincipals output. Earlier DNs for LTER were in URL form. Now they appear in DN form.
Earlier:
ldap://knb.lternet.edu:389/DN,base
Now:
DN

Have to make changes so that PISCO and NRS also show up.

Fixed two small bugs.

Made changes in XML that is sent back as result of action=getprincipals. Changes are as following:

1. <authSystem> tag contains organization attribute which specifies organization name
2. <user> tag now has <organization> tag which specifies the organization name of the user. This is being generated as null for LTER as of now.

Rename url property to defaultDB

Added the EventLog functionality to the servlet as action=getlog. Valid parameters are ipAddress, principal, docid, event, start, end. All parameters can be repeated except start and end.

Partial implementation of a query facility for the access log. Need to fix problems with the date filtering, and tie this into the servlet action when completed and tested.

Interim checkin of EventLog reporting structure. Still basically incomplete but compiles.

Omit HARVEST_OPERATION_TABLE and store harvest operations in properties file instead

Fixed the returndoctype bug in DBQuery as described in bug 1451. Now,
results are returned even if returndoctype is null, which is the
appropriate behavior.

Implement sending email reports to harvest administrator and site administrator

Patched to make the date-time value insertion work under oracle. Now it
tests fine under both oracle and postgres. Still need to write an
access/query function.

Added additional logging calls for read, upload, insert, update, delete, and
other commands. Now need to get a reporting function working and this
bug 101 will be largely finished.

Modified EventLog to eliminate the separate revision column and instead fold
that into the docid string. Log now includes read events, but need to update
the data sent there.

Modified SQL scripts to add the new access_log table. I've partially tested this under postgres but have not yet tested under oracle.

Reformatted code for readability and consistency.

Added javadoc that was missing for the EventLogData class.

Added classes for basic event logging support. These classes write log entries to the database connection. Still to do: query the event log, write oracle sql code (postgres is done), possibly allow the log to be archived to a text file, and possibly do the dtabase writes asynchronously via a caching mechanism.

Made change in result tag that is returned in getprincipals action. Replaced userdn with username.

Added GPL license text to harvester source files.

Fixed a bug the text filed in path xml query maybe splited.

Fixed a bug if a document already has index, but now metacat doesn't use index. If you try to delete this one, will cause error.

Clean up some code.

Revise the handle query and squery method, not finished yet.

Add a new transform method.

Add a new transferm method for string reader.

Implement harvester scheduing capabilities and sending email reports to the Harvester Administrator

Validate user input data in registration servlet

Improve authorization logic for single file uload servlet

Add delete capability to single file upload servlet

Set content type properly for getversion action.

Added a simple action for finding out the current metacat version. The information is pulled from the property file, so be sure to update the version in the property file before shipping a release.

Fixed a bug in the new recursive SQL statements that prevented them from operating on oracle. Now they should work on both oracle and postgres.

Reformatted source for readability and conformance.

Excluded the update of the xml_index table in insert,update, and delete if the usexmlindex option is set to false.

Reformatted DBQuery.java for readability and consistency.

Changed printAttributeQuery() to no longer depend on the xml_index table. Now, if useXMLIndex is false, the method will use a recursive query on xml_nodes to match paths instead of using xml_index.

Some refactoring of the printExtendedSQL()
methods to eliminate redundancy in the code and
to provide more readable documentation.

Moved decision about whether to use xml_index for
query to the metacat.properties as "usexmlindex". Default
is now false. Still have some refactoring to do to remove a
few more uses of the xml_index table.

Created new printExtendedSQL function that can get the extended
return fields using only xml_nodes and not xml_index. One
step closer to eliminating the dependence ofn xml_index.

Created test class QuerySpecificationTest and started process of removing the xml_index from the QuerySpecification code. Reformatted some classes for readability.

Removed the "getdataguide" action as it was always experimental and is no longer used in metacat or morpho.

removed the 3 versions of printPackageSQL which are no longer used by metacat. This eliminates some dependency on xml_index.

Implement Harvester Registration Servlet

Additional scheduling logic

Changes to Harvester properties

Changes to log entries

Made changes in these three files so that getPrincipal returns back more details about users and groups. Users now contain userDN, user Name and user Email. Groups now contain Group name and Group description. So some function calls which returned single string array earlier now returns multiple string arrays. AuthSession.java called one of these functions - so accordingly changes were made to fix that part of the code.

Fixed the bug for sessionid storing, revision couldn't be zero and Ldap DN is case sensitive.

Replaced Hash table with Vector of Vector datastructure in getting <param> from Query. This was done so that the results are represented in same order as they appeared in the document.

Fixed bug that version could not be zero in replication.

Additional Harvester development

Additional development of Harvester implementation

Harvester source files

Restoring AuthLdap to previous search filter. Determined that the problem was
a missing o: attribute in the UCNRS LDAP directory. Adding it in makes the old
method work. The change I had made caused some confusion about authentication
because using just uid for a filter caused too many return dn's, and it was just...

Modified LDAP authentication to support a lookup for the NRS system. If
the auth on the provided string fails as a DN, try to look up a new
DN based on the UID. If we get a match, use that DN for auth and see
if that works. This allows us to use a referral within the...

Enabled stylesheet parameters to be passed through metacat even when the
'query' function is called. Previously, query would interpret all
parameters except a few hardcoded ones to be pathexpressions that should be
built into a squery. Now, metacat.properties contains a new configuration...

Moved the metacat.properties file from the jar file to now be located in
the tomcat context directory in WEB-INF. Now the metacat configuration is
editable in place. This change depends on a new Options.java class in the
utilities module, so be sure you do a "clean" build in metacat so that you...

Improvements to Harvester single file upload servlets

This is the merge from the branch WEB_UI_REDESIGN_BRANCH back into the head.
The only files that have been removed are those in lib/style/eml2; other files
that have been cvs removed (only from lib/ and lib/style) have been re-added
elsewhere (i.e. moved from one directory to another)....

Remove unneeded file Upload999.java

If the sessionid is null and user will be treated as public.

Get rid of doctype declare in transfer.

Set system id for transforming.

Modified a method and can be reused.

If db pool size is maxium and no connection is free. The checkout will be wait.

Add a checking for xml_nodes insert finished, then start build index table.

Remove UPPER function for >, < operator.

Add a new search mode not-contains.

Modified a debug Message.

Add a new method to find document from queryspecification.

Revise code so it can be called directly.

Revied the class.

New class will be used in metacat ecogrid impl.

• empty log message ***

Updated more tests on session reuse. Made the http calls set the sessionid
explicitly rather than relying on cookies being passed around statically.
Still need to modify HttpMessage to get rid of the static cookie.

New versions of the Metacat client API that allow access to the sessionId
so that a session can be reused over several different instances.

Fixed the sessionId handling code to now properly return the sessionId
in the login response message. Updated junit tests to test this issue.

Modified metacat to now provide the session identifier in the response
xml message when a login is called. This has not been fully tested.
I removed the HttpMessage class from metacat in favor of using the version
of that class that is found in the utilities module. This may cause some...

Fixed a bug for importing package

Fixed the bug that if path query without xml declaration will cause a non well-formed result doc.

A sample application program to call metacat string service.

Fixed some bugs in this file.

Implementation for the interface.

Add a new client interface with string type for web service.

in toxml method, add new code to handle dtd.

Add a new constructor to hanlde DTD node.

Add code to handle public id and system id.

Implement logout method.

Add a logout method.

Implemented the update and delete methods in the interface, along with tests.

Implemented the "insert()" method and wrote a test to test it. The new test
depends on the property "newdocid" be set to something unique in the build.xml
file so that the insert will work properly. Probably need to fix this to be
some kind of autoincrement counter or random number so that it can be run...

Refactored to use IOUtil routines. Reformatted for within-file consistency.