Metacat

Revised the method writeDataFile, writeReplication and writeToDB

Fixed some DB connection bug.

Add the feature to get doc home server into method getDocumentInfo.
Add a method to return docHomeServer.

Add three methods into this class. One is named getServerCode, to given a server name, it return server code in xml_replication table.
One named insertServerIntoReplicationTable, if a server is not in the server list, it will insert it.
One is named WriteReplication, this method will be used in forceReplication. It delete the replication code in it.

Add a hub control in write and writeData method. If hub==super, metacat can replicate any document in its database to any server in server list. If it is not super, metacat only replicate local documents to any server in server list. If a document' home host is not itself, metacat only force replicate the document to the document's home host.

In writeDataFile method, using getDataFileLockGrant(docid) replace getDataFileLockGrant(docid, serverCode). The latter will cause a problem

Add a method named writeDataFile. This method will register data file into xml_documents table and save a input stream to a file.

Add method getDataFileLockGrant(). This method will check if xml-document of data file can be store into database.

Add a new method named unRegisterDocument. This method will delete the record in xml_document table which is for a data file

Fixed the bug metacat will set server location to 1, no matter it is.

In order to increase data file docid, method registerDocument was revised. Now it not only insert a new data file record to xml_documents table, but also can update the data file documents. Moreover, it can archieve the old version to xml_revisions table. Two other private methods were added to handl these futures: deleteXMLDocuments and getLatestRevisionNumber.

Add a throwing exception in hasPermission and hasReadPermission methods.

A constructor for this class was added. In the new constructor, user can specify the revision number he want to update. In method writeDocumentToDB, when a update action happend, the rev number would not automatictlly to add one, but it can go to specified number. This is for bug 417.

A method named hasReadPermission was added. The method will check if a user has permission toread a xml document.

made a ton of changes related to keeping oracle SQL code out of the main classes. fixed a bug where the timing of the index thread was off so when it went to index a document, the document was not already in xml_documents thus breaking the FK relation between xml_documents and xml_index. I think that bug might be the reason for the blank resultset screens in morpho. made the postgres implementation much more robust.

changes in replication to use https

fixed error:
writing on the replication server required the whole acc# (including the rev#)

changes to store namespace prefixes separately from the local names of nodes (elements and attributes)
in xml_nodes.nodeprefix column

added support for Namespaces.
Metacat now can store and retrieve XML documents with Namespaces.
Namespace is stored as separate record in xml_nodes table with type "NAMESPACE"
where nodename is the prefix and nodedata is the uri of the namespace.

fixed error with out of cursors error because pstmts were not getting closed in DBQuery

added support for multiple group membership

Changed the constructors so that the nodeRecordList is automatically
retrieved by the DocumentImpl(Connection,String) constructor. this
restores its earlier behavior, but the implementation is totally
done by DocumentImpl(Connection,String,boolean).

Added fix to DocumentImpl that showed problems when the DocumentImpl(conn)
constructor was called. Now there is a new constructor:
DocumentImpl(Connection, String, boolean)
where the boolean value is used to determine if the nodeset should be
read a t the time the cosntructor is called. if false, then...

Modified Metacat to support large data file uploads. This is accomplished
by supporting a new content type for data sent to metacat:
multipart/form-data
which allows multiple files to be sent in a standard MIME format. The
MetacatServlet.handleGetOrPost() method now checks the incoming content...

added constructor in DocumentImpl for use in DBQuery.findDocuments();
this new constructor is used with getDocumentInfo() to get the doc info only;
because the other constructor builds the whole xml doc which is not needed here and time consuming;
this happens on backtracking only.

moved delete from xml_relation before delete from xml_documents
because of the parent-child relation b/w these 2 tables

changes according to bug# 234 - metacat accession # handling, done

changed all mentions of DBAdapter to the new name AbstractDatabase

excluded revisionid from the insert statement for xml_revisions.
It is identity column for SQL Server or filled by sequence for db as Oracle or Postgres

changed ... WHERE ... LIKE ? ...
to ... WHERE ... = ? ...

change the call from getDateFunction() to getDateTimeFunction()

change the call from getDateString() to getDateFunction()

- changed to use the DBAdapter uniform interface for all apperance of "sysdate"
and uniqueid generation from sequence
MetaCatUtil:
- reads from metacat.properties the dbadapter class implemented the interface
and creates an instance of that class (in static public variable named dbAdapter) for use....

fix in the return of the last revisionid on UPDATE instead of the previous one

- changes to expect revisionid to come from the client on INSERT/UPDATE/DELETE;
now the client should send accession# as:
sitecode.uniqueid.revisionid
INSERT checks if sitecode.uniqueid part does not exist in xml_documents and xml_revisions and revisionid=1...

- fixed missing replication of public_access
- new attribute catalog_id added in xml_documents and xml_revisions
as a FK to xml_catalog; catalog_id is replicated also

Included check up for <!DOCTYPE ... PUBLIC/SYSTEM ... >
in order to set the validation parser at runtime
In case of:
- no DOCTYPE declaration
- <!DOCTYPE docname>
validation is turned "off"
In case of:
- <!DOCTYPE ... PUBLIC ...>
- <!DOCTYPE ... SYSTEM ...>...

there were more mentions of DOCTITLE attr - cleared them

cleared DOCTITLE attribute in xml_documents and xml_revisions
cleared all mentions of doctitle

added new "getaccesscontrol" action for a given docid

fixed white space problem in toXML that was preventing in line end tags from being popped from the stack.

included new servlet parameter "public" specifing public read access for the submitted document
the expected values are "yes" or "no"
public parameter is optional and if it is not specified, means "no"

added precise location information (class.method) to each catch statement so that errors are more easily traced.

Added license terms to source code files, and cleaned up some javadoc
documentation in a few places.

closed all preparedStatement variables

- turned on the validation in order only valid xml docs comformed to the specified dtd (if any) to be submitted in metacat
- with validation "on" white spaces are reported from ignorableWhitespace() callback, not from characters() (as with validation "off")...

- appling acl through access files only
- clearing around update/delete of access files - to delete the related records from both xml_access and xml_relation tables
- included check for user permissions for setting acl on the resources in the access file
- moved RelationHandler.deleteRelations(docid) down in RelationHandler obj just before the write of the new relations

included support for submition of access files
+ storing the acl info in xml_access table and relationship records in xml_relation table like
<aclfile, "isaclfilefor", recourceIdentifier> for every resource included in the access file

updating a package file now updates the relations that are created from it. Also, if you delete a package file, it will delete the relations associated with it.

added revision support to the delete function

cleaned up code

changed accession number schema to <sitecode>.<serialnumber>.<revisionnumber>. DocumentIdentifier.java parses docids into parts for easy manipulation. You can now specifiy a specific revision by asking for document <sitecode>.<serialnumber>.<revisionnumber> or you can get the newest revision by asking for <sitecode>.<serialnumber> or by asking for <sitecode>.<serialnumber>.newest.

one more write() method was needed

DBEntityResolver:
- added new parameter dtd for upload on Metacat file system; optional; Reader
- new routines to upload dtd if provided
- systemID of metadata document is used to exctract the filename of new dtd;
- the new dtd is uploaded on Metacat file system as specified by dtdPath property;...

added replication for the xml_catalog table. Right now it is only replicated when the deltaT handler is called. I will add more replication times later.

added a logging system for replication. The log file location is set in metacat.properties

made replication on insert or update us. Also made a method in AccessionNumber public so that you can tell if an accession number has already been used.e place be

updated file locking mechanisms to use revisions instead of dates.

added replication_on_insert handling. Changed replication from date_updated base replication to revision number replication.

added revision number tracking support to xml_documents and xml_revisions. Also added an updated flag as a replacement for date based replication handling.

replication on insert functionality is now working.

added more replication/file locking functionality.

added more locking support and support for remote update of files.

added more locking functionality for replication

AccessControlList
- methods for parsing and loading acl file
- checkup method for permission for given principal on given resource
DBQuery
- checkup for READ permission using AccessControlList.hasPermission()
DocumentImpl
- using AccessControlList object to parse and load an acl file into xml_access table...

cleaned up code, added new static methods to MetacatReplication for handling static queries and getting URL content.

created locking action in replication servlet and added semi-support to documentImpl to handle documents updatedee by replicatio

added functionality to allow the replication servlet to assertain and insert user and group info into the local database. started implementation of insert replication handler (it is commented out in this commit)

fixed compatibility problem by overloading the write method. Jivka's new parameter (Reader acl) is now defaulted to null if it is not explicitly declared.

new class for parsing ACL XML file and loading acl data into metacat db

added support for server_location in the xml_revisions table.

removed dependence on a server code in the accession number of each document. the file's resident server is now located in xml_documents.server_location.

fixed bug/typo

Added backtrack functionality. Backtracking works by passing a returndoc parameter. There can be more than one. If a document that is hit by a query is not of type returndoc then it searches the database for a related file of type returndoc. If one is found it is displayed, if no relation is found, the original is displayed....

added extra timing information to the main() driver method.

change Assession# generation to use the same db connection

changes related to decrease the time of INSERT of document.
With these changes I inserted 200KB file for 3 minutes, 50KB for 50sec.
This is mainly simplifing the DBSAXNode class and
using batching feature of Oracle JDBC driver.

fix bug in toXML() about getting output of XML document like:
<paragraph

fix for "Document not found" err message on "getdocument" action

added check from "read" permission on "query" and "squery" actions
for connected user or for "public" connection

Fixed document reading bug (bugzilla bug #111) so that reading documents
is no longer a power function of the number of nodes in the document
(which used to be the case). Now, reading a document occurs entirely
within DocumentImpl, by making a single SQL call to get the document data,...

change the 2 executions in archiveDocRevision()
with 1 statement:
"insert into xml_revisions
select ... from xml_documents where docid = ..."

small changes on hasWritePermission()

storing user_owner and user_updated where needed
new function in DocumentImpl checking for "write" perm on UPDATE or DELETE
added delete from xml_index of the old version of docid on UPDATE

cleared static methods in AccessionNumber classes for fixing bug found
when multiple requests to the servlet at a time.

Minor update to remove unneeded constructor in DocumentImpl class.

Continued code redesign for the DocumentImpl class. Now the "delete" and
"write" methods are static, so a DocumentImpl objject need not be created
in order to initiate a INSERT, UPDATE, or DELETE action (it is created
implicitly by the DBSAXHandler). When doing a "READ" action, one still...

Folded the functionality from DBWriter into DocumentImpl, continuing the
work started earlir to create a more DOM-like model for the classes, in
which a single DocumentImpl class handles both reading and writing of
documents to the database. Modified shell scripts and MetaCatServlet to...

Folded the functionality from DBSAXDocument into the DocumentImpl class.
Now DocumentImpl handles both the reads and the writes to the database.
Eventually, it will implment the full DOM Document interface. Eliminated
the DBSAXDocument.java file as it is no longer relevant....

Created new class "DocumentImpl" which represents an XML Document. This
document will eventually implment the DOM Document interface. For now,
it implments all of the functionality that was present in DBReader, plus
cuts down on the number of database calls required to read a document. And...