Metacat

support for the new terms included in pathquery.dtd

http://bugzilla.ecoinformatics.org/show_bug.cgi?id=248
Included utility function about gettig the latest docid for a user:
DBUtil.getLastDocid(username).
Metacat parameters:
action='getlastdocid'
username

don't need to override the old meta_file_id tags with docid without rev
since Morpho inserts docids before sending the docs to Metacat

added the new software module

added constructor in DocumentImpl for use in DBQuery.findDocuments();
this new constructor is used with getDocumentInfo() to get the doc info only;
because the other constructor builds the whole xml doc which is not needed here and time consuming;
this happens on backtracking only.

updated script with new eml20 dtds

fixed bug - hardcoded LDAP URL

made use of the new property for ldaps url;
it is used for secure connection to LDAP server listening on second port 636 by default with SSL sockets;
it is used from Metacat for the authetication process only;
all the rest communications with LDAP server are made on the default port 389 with plain sockets

moved delete from xml_relation before delete from xml_documents
because of the parent-child relation b/w these 2 tables

fixed error with exception handling in metacat servlet and added new dtds to the sql script

changes according to bug# 234 - metacat accession # handling, done

changed all mentions of DBAdapter to the new name AbstractDatabase

changed all mentions of DBAdapter to the new name AbstractDatabase

rename the DB* classes to:
AbstractDatabase
OracleAdapter
PostgresqlAdapter
SqlserverAdapter

use the non-null function name from db adapter

included new function that returns db specific non-null function name

cleared the COLLATION specifiers on column definitions:
it is new in SQL Server 2000 and not supported in its earlier versions

excluded revisionid from the insert statement for xml_revisions.
It is identity column for SQL Server or filled by sequence for db as Oracle or Postgres

changed like with =

changed ... WHERE ... LIKE ? ...
to ... WHERE ... = ? ...

MS SQL Server stores trailing spaces in varchar fields which shouldn't be
thus added docid.trim() in the Java code

the ODBC multi-threading problem:
in DBQuery we open ResultSet in the db connection and we are trying to
open another Resultset by AccessControlList which does not work
thus open separate connection for use by AccessControlList.

changed
SELECT ... WHERE LIKE ? ...
to:
SELECT ... WHERE = ? ...

It should be changed everywhere it is found

seems that there are problem in JDBC-ODBC bridge used with SQL Server
when using sql statements like:
SELECT ...
WHERE LIKE ? ...
changed to:
SELECT ...
WHERE = ? ...
and now it works

commented out the use of oracle.jdbc.driver
this was harcoded use for batching the sql calls at once in order to speed up the insert of XML Index

added
INSERT INTO xml_replication (serverid, server, replicate) VALUES ('1', 'localhost', '0');
which was missing

several changes were needed related to the change of the unique id generation and use

Changed the order of generating and getting Unique ID for
xml_nodes.nodeid and for uniqueid part of the accession#
because of SQl Server capabilities:
first let the db to generate unique id on insert (by db trigger and sequence or by IDENTITY key),
then get that unique id for further use by the application....

- new db adapter for MS SQL Server
- changed the getUniqueID(conn, tableName) to return the last generated by the db unique ID
instead of generating unique ID first by the appl and then supplied to the insert,
because SQL Server doesn't have capabilities
unique ID to be generated on the client and then supplied to the insert.

- made xmltables-sqlserver.sql from SQL Server running here on rincon machine
- included db index on xml_nodes.nodename which was missing

change the call from getDateFunction() to getDateTimeFunction()

changed the getDate method to getDateTime and changed the return
string in the postgresql implementation to "now()" from now()::date

change the call from getDateString() to getDateFunction()

change the method name getDateString() to getDateFunction()
return "now()::date" in the DBPostgresql implementation of that method

The Postgres implementation of the dbadapter package, still needs testing and
the getDate... method just returns the db function to return the date; to retreive the
current time is a second function so we may want to add a second method to return the
current time.

changed the db sequence name used for the accession# uniqueid
to xml_documents_id_seq.
All db sequences now are named of form <table_name>_id_seq

- changed to use the DBAdapter uniform interface for all apperance of "sysdate"
and uniqueid generation from sequence
MetaCatUtil:
- reads from metacat.properties the dbadapter class implemented the interface
and creates an instance of that class (in static public variable named dbAdapter) for use....

the abstract DBAdapter class and its Oracle implementation
for the new dbadapter package

Fixed bug in metacat where return doctypes were not being properly
back traced. Changed DBQuery.findDocuments() so that it no longer
takes a returndoc array, but instead retrieves the returndoc array
fromthe QUERYSpecification. The QuerySpecification was augmented...

Added ability to process multiple "returndoc" elements in
a pathquery document. Now, any query hit will be back traced to an
associated package if that package is listed in one of the
"returndoctype" elements.

Modifications that change the package handling behavior of metacat. Package
membership is now determined by an identifier being used as either the subject
or object in a triple. One can not request arbitrary returndocs -- only
those types that have package information embedded can be "returndoctype"...

fix in the return of the last revisionid on UPDATE instead of the previous one

this is sql script does the same as the sql script named
xmltables.sql except that this script is to be use to
create the database tables on a Postgresql backend rather
than an Oracle Backend

the foreign-key constraints still need to be fixed

fixed the problem with empy password authentication.
Ldap somehow allows DirContext to be created
when empty password is provided as in our case
instead of comlaining like with wrong password string.
So included a check for empty password in order to reject the login.

as John were testing Metacat on postgres,
he discovered that Connection needs more precise check
whether it is open because of db timing outs.
So used to be:
if ( conn null ) { get new connection from db; }
Now changed to:
if ( conn null || conn.isClosed() ) { get new connection from db; }...

fix to view abstract only when abstractpath parameter was specified

put exception handler messages when acc# is not in the correct format as:
<sitecode>.<uniqueid>.[<revisionid>]

fix - added back generate(docid, action) method called from DataFileServer.java where rev is defaulted to 1

- changes to expect revisionid to come from the client on INSERT/UPDATE/DELETE;
now the client should send accession# as:
sitecode.uniqueid.revisionid
INSERT checks if sitecode.uniqueid part does not exist in xml_documents and xml_revisions and revisionid=1...

Tomcat has got configurated mime type mappings, but some are
missing. Thus added support for them in MetaCatServlet as:
File type > MIME type
------------- ----------------
.xml -> "text/xml"
.xsd -> "text/xml"
.dtd -> "text/plain"
.css -> "text/css"...

- merged "getabstract" and "getdatadoc" actions to "read" action;
- put MIME types everywhere for the returned data;
- zip when more than one doc/files were requested or when specified by qformat="zip" parameter;
- get rid of "relation" parameter, instead use "docid" parameter name only - when more that one docid - zip them;...

Changes related to running LDAP servers referred each other in one tree modeling the KNB tree.
Currently there are running LDAP server on dev that holds the KNB root and the NCEAS' s subtree.
It also refers to a subtree modeling the LTER's tree which is held by another LDAP server runnig on alpha .

uncompleted script file to create DB on SQL server

fixes on getting information from LDAP services

added check to see if a replication server's IP matched it's DNS entry and the entry in the replication table

fixes around getUsers and getGroups.
"getprincipals" action can now produce output like:

<principals>
<group>
<groupname>Administrators<groupname>
<user>
<username>uid=admin,o=NCEAS, c=US<username>
</user>
</group>...

Included back getting the list of users and groups stored in auth scheme
through new action="getprincipals". No extra parameters are needed.
Any logged in users are able to get this information

removed download.xsl entry

included missing implementation of getUsers(), getGroups() interfaces

no message

no message

Modifications to allow printing of SQL with and without INDEX option.

added print info and timing info

print the nested statements sql query

included flag in DBQuery to run pathqueries with or without the index,
but for the command line only.

added timing info to main

added removal of relations

added new stylesheet for download

implemented interface for download of DTD or Schema file from Metacat file system
through the params:
action="getdtdschema"
doctype

- new attribute catalog_id added in xml_documents and xml_revisions
as a FK to xml_catalog

- fixed missing replication of public_access
- new attribute catalog_id added in xml_documents and xml_revisions
as a FK to xml_catalog; catalog_id is replicated also

Included check up for <!DOCTYPE ... PUBLIC/SYSTEM ... >
in order to set the validation parser at runtime
In case of:
- no DOCTYPE declaration
- <!DOCTYPE docname>
validation is turned "off"
In case of:
- <!DOCTYPE ... PUBLIC ...>
- <!DOCTYPE ... SYSTEM ...>...

reject writing DTD file if it already exists

there were more mentions of DOCTITLE attr - cleared them

cleared DOCTITLE attribute in xml_documents and xml_revisions
cleared all mentions of doctitle

cleared DOCTITLE attribute in xml_documents and xml_revisions

cleared the UNIQUE constraint in xml_access - moved in AccessControlList
added FOREIGN keys for xml_relation.docid and xml_access.accessfileid referencing xml_dociments.docid

added new "getaccesscontrol" action for a given docid

fixed white space problem in toXML that was preventing in line end tags from being popped from the stack.

Solving the problem with relication of access file where access files were rejected from the replication servers.
In AccessControlList check for serverCode is included,
so only on the local server the connected user is checked for having "all" permissions on the resources specified in it...

fixed some problems with database connections getting a time out error. Also updated some catch statements

fixed typo

included new servlet parameter "public" specifing public read access for the submitted document
the expected values are "yes" or "no"
public parameter is optional and if it is not specified, means "no"

fixed bug that was accidentally introduced when the change from metacatURL to URL was done.

added location info to catch statements

fixed comments

added comments and removed xml_acc_numbers

added precise location information (class.method) to each catch statement so that errors are more easily traced.

removed print statements

fixed typo

included implementation for public "read" access as specified from the access file

fixed introduced problem with preparedStatements

Added license terms to source code files, and cleaned up some javadoc
documentation in a few places.

Added License terms to source files, added more documentation to protocol
handler classes.

closed all preparedStatement variables

implementation for multiple <resourceIndentifier> tags under <resource>

removed dependence on MetacatURL for URL handling. Added more flexible support for identifiers in the subject and object fields. They can now be just a docid, an http url or a metacat url.

fixed bug in data file permission checking. (commited wrong file last time)

- turned on the validation in order only valid xml docs comformed to the specified dtd (if any) to be submitted in metacat
- with validation "on" white spaces are reported from ignorableWhitespace() callback, not from characters() (as with validation "off")...

added more documentation