Metacat

changes in replication to use https

fix around one way replication:
when server A sends document to the servers in its list of replication servers with replicate=1 in xml_replication table,
the server B received the document does NOT need the opposite check on its end.

fixed error:
writing on the replication server required the whole acc# (including the rev#)

Fixed bug in metacat where an Exception was thrown when a search returned
an empty resultset. Now we avoid the function call that threw the exeption.

Minor bug fixes following change in style handling. Fixed image locations,
cleaned up main resultset stylesheet.

Added support for multiple user interfaces by enabling style sheet
transformations to be configured from within metacat rather than
from in the database. Now, metacat uses the qformat parameter to determine
a "style set", which is mapped to an xml configuration file that...

small fix to enable resolving external entities included in the DTD;
external entities are now resolved successfully by DBEntityResolver using the specified system ids
and thus there no need of registering the relative system ids,
This is because EntityResolver uses its internal implementation to map the relative system ids to the...

Removed the requirement that the relationship for ACL associations be
"isRelatedTo". Now the relationship in the triple can be anything, like
"describes access control rules for". This doesn't affect the ACL
processing because it is based on the ACL document type, not the...

More fully completed the eml-dataset XSLT stylesheet for translating
into HTML format. Now more of the relevant fields are completed, and
the coverages are shown. Still need to systematically address every
field that could be present, but I hit the major ones.

Added new filter for style-path

Modified metacat stylesheets to now use a "style-path" filter for setting
the path to XSL and CSS style sheets. this allows the style
location to be configured independently of the main web url for metacat
links.

changes to store namespace prefixes separately from the local names of nodes (elements and attributes)
in xml_nodes.nodeprefix column

added nodeprefix column to xml_nodes table for storing of namespace prefixes per nodes

Updated index page to launch proper initial query.

Updated stylesheet to reflect the new eml2 schema.

Revised build.xml and metacat.properties to make it easier to
install alternative stylesheets for metacat. In the process,
replaced the deprecated copydir and copyfile commands in build.xml
with appropriate "copy" commands.

added support for Namespaces.
Metacat now can store and retrieve XML documents with Namespaces.
Namespace is stored as separate record in xml_nodes table with type "NAMESPACE"
where nodename is the prefix and nodedata is the uri of the namespace.

fixed an error of NullPointerException

- when the Access file goes first before the Package file (eml-dataset-2.0)
relations are not available in xml_relation, thus updated the code
to check and run ACL also after the Package file is saved.
- cut out the rev# from subject and object in xml_relation as needed by ACL...

fixed error with out of cursors error because pstmts were not getting closed in DBQuery

Updated build file to resotre changes that were lost when version
1.54 was committed.

look for relationship "isRelatedTo" b' that is the word that is used in the triples

Fixed returndoctype param.

Updated stylesheet to use new package returndoc type for searches
(eml-dataset-2.0).

updated with the new eml modules

changed web-base-url to http://knb.ecoinformatics.org

made ant copy the dtds

added new dtds

dtds that have entities removed for morpho/metacat compatibility

Clean up after building distribution.

Updated build.xml with new targets to create metacat distributions.
Run "ant dist" to create a non-source distribution and "ant distsrc"
to create a source distribution. Best if this is run on a clean
checkout to be sure not to include extraneous files that are not in CVS.

Updated README file for the beta2 release. Added list of known issues,
which will need to be edited for the final release.

fixed the compilation bug about the change of group parameter in DocumentImpl.write()

changed accessdoctype=-//NCEAS//eml-access-2.0//EN
b' AccessControlList is changed to use eml-access-2.0.dtd already

added support for multiple group membership

Changed the constructors so that the nodeRecordList is automatically
retrieved by the DocumentImpl(Connection,String) constructor. this
restores its earlier behavior, but the implementation is totally
done by DocumentImpl(Connection,String,boolean).

Added fix to DocumentImpl that showed problems when the DocumentImpl(conn)
constructor was called. Now there is a new constructor:
DocumentImpl(Connection, String, boolean)
where the boolean value is used to determine if the nodeset should be
read a t the time the cosntructor is called. if false, then...

Completely removed the socket-server feature that used to provide
file upload. It is now replaced by multipart/form-data over http.

Modified Metacat to support large data file uploads. This is accomplished
by supporting a new content type for data sent to metacat:
multipart/form-data
which allows multiple files to be sent in a standard MIME format. The
MetacatServlet.handleGetOrPost() method now checks the incoming content...

Simplified ReplicationHandler:
It is run now in the same thread started from DBSAXHandler;
thus the same connection opened in that thread is used,
i.e. no need for new connection and no complains as "Connection is closed" in ReplicationHandler;
"Connection is closed" error happens when in the current thread there are try to use connection opened in another thread,...

tests made on Oracle and SQL Server.

support for the new terms included in pathquery.dtd

included new terms for searchmode:
equals (former matches-exactly)
isnot-equal
less-than
greater-than
less-than-equals
greater-than-equals

http://bugzilla.ecoinformatics.org/show_bug.cgi?id=248
Included utility function about gettig the latest docid for a user:
DBUtil.getLastDocid(username).
Metacat parameters:
action='getlastdocid'
username

don't need to override the old meta_file_id tags with docid without rev
since Morpho inserts docids before sending the docs to Metacat

added the new software module

added constructor in DocumentImpl for use in DBQuery.findDocuments();
this new constructor is used with getDocumentInfo() to get the doc info only;
because the other constructor builds the whole xml doc which is not needed here and time consuming;
this happens on backtracking only.

updated script with new eml20 dtds

fixed bug - hardcoded LDAP URL

made use of the new property for ldaps url;
it is used for secure connection to LDAP server listening on second port 636 by default with SSL sockets;
it is used from Metacat for the authetication process only;
all the rest communications with LDAP server are made on the default port 389 with plain sockets

new property for ldaps:// url for LDAP connection over SSL;
it is set on second port - by default 636;
for plain LDAP communications the default port 389 is used.

moved delete from xml_relation before delete from xml_documents
because of the parent-child relation b/w these 2 tables

cleared the text under the figure; should go in the Word text

EPS files saved as AdobeIllustrator v7.0

EPS file for the IC paper

fixed error with exception handling in metacat servlet and added new dtds to the sql script

changes according to bug# 234 - metacat accession # handling, done

changed all mentions of DBAdapter to the new name AbstractDatabase

changed all mentions of DBAdapter to the new name AbstractDatabase

changed dbAdapter property to edu...OracleAdapter

rename the DB* classes to:
AbstractDatabase
OracleAdapter
PostgresqlAdapter
SqlserverAdapter

use the non-null function name from db adapter

included new function that returns db specific non-null function name

cleared the COLLATION specifiers on column definitions:
it is new in SQL Server 2000 and not supported in its earlier versions

excluded revisionid from the insert statement for xml_revisions.
It is identity column for SQL Server or filled by sequence for db as Oracle or Postgres

changed like with =

changed ... WHERE ... LIKE ? ...
to ... WHERE ... = ? ...

MS SQL Server stores trailing spaces in varchar fields which shouldn't be
thus added docid.trim() in the Java code

the ODBC multi-threading problem:
in DBQuery we open ResultSet in the db connection and we are trying to
open another Resultset by AccessControlList which does not work
thus open separate connection for use by AccessControlList.

changed
SELECT ... WHERE LIKE ? ...
to:
SELECT ... WHERE = ? ...

It should be changed everywhere it is found

seems that there are problem in JDBC-ODBC bridge used with SQL Server
when using sql statements like:
SELECT ...
WHERE LIKE ? ...
changed to:
SELECT ...
WHERE = ? ...
and now it works

commented out the use of oracle.jdbc.driver
this was harcoded use for batching the sql calls at once in order to speed up the insert of XML Index

added
INSERT INTO xml_replication (serverid, server, replicate) VALUES ('1', 'localhost', '0');
which was missing

several changes were needed related to the change of the unique id generation and use

Changed the order of generating and getting Unique ID for
xml_nodes.nodeid and for uniqueid part of the accession#
because of SQl Server capabilities:
first let the db to generate unique id on insert (by db trigger and sequence or by IDENTITY key),
then get that unique id for further use by the application....

- new db adapter for MS SQL Server
- changed the getUniqueID(conn, tableName) to return the last generated by the db unique ID
instead of generating unique ID first by the appl and then supplied to the insert,
because SQL Server doesn't have capabilities
unique ID to be generated on the client and then supplied to the insert.

Updated pathquery.dtd to reflect new addition of filterdoctype element.

- made xmltables-sqlserver.sql from SQL Server running here on rincon machine
- included db index on xml_nodes.nodename which was missing

change the call from getDateFunction() to getDateTimeFunction()

changed the getDate method to getDateTime and changed the return
string in the postgresql implementation to "now()" from now()::date

change the call from getDateString() to getDateFunction()

change the method name getDateString() to getDateFunction()
return "now()::date" in the DBPostgresql implementation of that method

The Postgres implementation of the dbadapter package, still needs testing and
the getDate... method just returns the db function to return the date; to retreive the
current time is a second function so we may want to add a second method to return the
current time.

changed the db sequence name used for the accession# uniqueid
to xml_documents_id_seq.
All db sequences now are named of form <table_name>_id_seq

added new property for the dbadapter class as:
dbAdapter=edu.ucsb.nceas.dbadapter.DBOracle
the installation procedure should change it to the appropriate class

- changed to use the DBAdapter uniform interface for all apperance of "sysdate"
and uniqueid generation from sequence
MetaCatUtil:
- reads from metacat.properties the dbadapter class implemented the interface
and creates an instance of that class (in static public variable named dbAdapter) for use....

the abstract DBAdapter class and its Oracle implementation
for the new dbadapter package

Fixed bug in metacat where return doctypes were not being properly
back traced. Changed DBQuery.findDocuments() so that it no longer
takes a returndoc array, but instead retrieves the returndoc array
fromthe QUERYSpecification. The QuerySpecification was augmented...

Added ability to process multiple "returndoc" elements in
a pathquery document. Now, any query hit will be back traced to an
associated package if that package is listed in one of the
"returndoctype" elements.

Modifications that change the package handling behavior of metacat. Package
membership is now determined by an identifier being used as either the subject
or object in a triple. One can not request arbitrary returndocs -- only
those types that have package information embedded can be "returndoctype"...

fix in the return of the last revisionid on UPDATE instead of the previous one

this is sql script does the same as the sql script named
xmltables.sql except that this script is to be use to
create the database tables on a Postgresql backend rather
than an Oracle Backend

the foreign-key constraints still need to be fixed

fixed the problem with empy password authentication.
Ldap somehow allows DirContext to be created
when empty password is provided as in our case
instead of comlaining like with wrong password string.
So included a check for empty password in order to reject the login.

as John were testing Metacat on postgres,
he discovered that Connection needs more precise check
whether it is open because of db timing outs.
So used to be:
if ( conn null ) { get new connection from db; }
Now changed to:
if ( conn null || conn.isClosed() ) { get new connection from db; }...

fix to view abstract only when abstractpath parameter was specified

specified the correct param list about the "read" action changes

put exception handler messages when acc# is not in the correct format as:
<sitecode>.<uniqueid>.[<revisionid>]

fix - added back generate(docid, action) method called from DataFileServer.java where rev is defaulted to 1

- changes to expect revisionid to come from the client on INSERT/UPDATE/DELETE;
now the client should send accession# as:
sitecode.uniqueid.revisionid
INSERT checks if sitecode.uniqueid part does not exist in xml_documents and xml_revisions and revisionid=1...

Tomcat has got configurated mime type mappings, but some are
missing. Thus added support for them in MetaCatServlet as:
File type > MIME type
------------- ----------------
.xml -> "text/xml"
.xsd -> "text/xml"
.dtd -> "text/plain"
.css -> "text/css"...