Metacat

Down the debug level from debug to trace.

Disable the feature of downloading external schemas.

If a format id in the system metadata is registered in the xml_catalog table, we will use the schema location for the format id to validate the xml instance;
otherwise, we will use our previous way.

remove morpho.jar -- moved needed classes into shared utilities project. (currently building form utilities trunk -- be sure to 'ant fullclean' to get the latest utilities.jar built)

no not record EML access rules that use the "denyFirst" permOrder.
https://redmine.dataone.org/issues/2614

add a parameter for optionally writing EML-embedded access control rules to the Metacat DB.
https://redmine.dataone.org/issues/2584
https://redmine.dataone.org/issues/2583

uses prepared statement instead of plain old statement.
deprecated the DBConnection.createStatement() method to discourage direct parameter value use in favor of parameter binding.
http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5527

http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5527

pass the root exception message up the call chain so that it can effectively be reported as a helpful error message. also, the JUnit test expects the specific error message (SchemaRegistryTest)

use detected XML encoding when reading/writing files
use UTF-8 as default when performing queries in the DB (assume DB is using UTF-8)
remove as many PrintWriters (uses system default character encoding only) as possible and construct OutputStreamWriters where explicit encoding can be given....

debug statements in dbsaxhandler

Move the chunking of large test element data to centralized location in DBSAXNode.writeChildNodeToDB(). Beff up logging

Change location of PropertyService to properties directory

Create database and shared directories for database management code and shared code respectively.

Renamed MetaCatUtil to MetacatUtil

Add some generics typing. Fixed comments.

Store schemaLocations as they are found in startElement. Save them in endDocument. In this way, schemas will only get saved locally if the document parses successfully.

qualify xml and eml properties with an xml. prefix

Merge 1.9 changes into Head

Add debug info for special charater.

paging now works well. sped up the caching by using SAX instead of DOM parsing. it seems fast. need some other people to test for me though. the two params needed are 'pagesize' and 'pagestart'. I haven't updated any of the web interfaces to use this, so if you want to try it, you'll have to type in the url the old fashioned way

Removed the release ant token from all files in 'src'.

removed the indexing code from DBSAXHandler

Fix for bug in the previous commit

Modified the debug levels of debug statements

Replacing MetaCatUtil.debugMessage or MetaCatUtil.logMetacat call with logMetacat (private Logger object) call

fixe bug which currentDoc maybe null.

Add code to get date info

Add code to handle replication deleted docs.

Add a new indicator - is revision document.

Fix for bug 2060.

Moved the call to starting of indexing thread from endDocument to DocumentImpl after commit has been done. This way when ever a document is indexed it has already been entered in xml_nodes and xml_documents

Fixed a bug from previous commit which removed whitespaces from documents.

Fixed a bug in previous commit. Moved normalization function before the string size is counted so that size change due to normalization is taken into account.

Made changes to fix bug# 1538. Changed the code of the normalize function in MetaCatUtil.java. Earlier code was not taking care of characters above 123.

In DBSAXHandler.java, added call to normalize function before text is written to db.

Changed DBSAXHandler.run() to now use the new DocumentImpl.buildIndex() method
for populating the xml_index table. The new method uses a jdbc ResultSet
for populating the index rather than doing it with the DBSAXNode, so it is
faster and can be run at any time on any document. This will allow us to...

Changed error text that is returned when an invalid eml is inserted.

fixed the bug that couldn't assign access rule for the data file come with morpho.

Excluded the update of the xml_index table in insert,update, and delete if the usexmlindex option is set to false.

Add a checking for xml_nodes insert finished, then start build index table.

Add code to handle public id and system id.

When character size is greater than 4000, it will write to db.

Change a debug statement.

Add debugMessage.

Add code to parse triple.

Get rid of access and relationa runner from run() method.

Add run access control even for access document.

Change the debugMessage for characters.

Add code to handle check inline data if a user update it but the user doesn't have permission to do it.

Change processDTD to protected.

Add code to debug warning and error message.

Add a debug message for fatal error.

Fixed a bug the for comment in xml documents. It cause a Unterminated quoted string problem in psql.

Add a varible to keep track node id and every call for write to xml_nodes will return a node id.

Add the code the handle text node was splitted

Change a varible from private to protected.

Add code to call SchemaLocationResolver.

If insert a schema xml, the doctype will be root element name space.

In the end of dtd, set processingDTD false.

Merge the code for monarch.

Get rid of running parser access documents into rules when it handle a access document. (we only need to parser it when handling a dataset file).

Merge DBConnection branch to head.

Assign debug level to parser. (it level is 50)

A constructor was added to this class. In this constructor, user can specify the revision number he want to update. And the number can be passed into a DocumentImpl object constructor in startElement method. But the number should be greater than current number in database.

made a ton of changes related to keeping oracle SQL code out of the main classes. fixed a bug where the timing of the index thread was off so when it went to index a document, the document was not already in xml_documents thus breaking the FK relation between xml_documents and xml_index. I think that bug might be the reason for the blank resultset screens in morpho. made the postgres implementation much more robust.

made it so we can now use multiple accessfilestypes and packagdfiletypes in the metacat.properties file. Also fixed a bug introduced when the 'http://' was removed from behind the server name in the loaddtd.sql script and the knb.xml file

small fix to enable resolving external entities included in the DTD;
external entities are now resolved successfully by DBEntityResolver using the specified system ids
and thus there no need of registering the relative system ids,
This is because EntityResolver uses its internal implementation to map the relative system ids to the...

changes to store namespace prefixes separately from the local names of nodes (elements and attributes)
in xml_nodes.nodeprefix column

added support for Namespaces.
Metacat now can store and retrieve XML documents with Namespaces.
Namespace is stored as separate record in xml_nodes table with type "NAMESPACE"
where nodename is the prefix and nodedata is the uri of the namespace.

- when the Access file goes first before the Package file (eml-dataset-2.0)
relations are not available in xml_relation, thus updated the code
to check and run ACL also after the Package file is saved.
- cut out the rev# from subject and object in xml_relation as needed by ACL...

added support for multiple group membership

Simplified ReplicationHandler:
It is run now in the same thread started from DBSAXHandler;
thus the same connection opened in that thread is used,
i.e. no need for new connection and no complains as "Connection is closed" in ReplicationHandler;
"Connection is closed" error happens when in the current thread there are try to use connection opened in another thread,...

- fixed missing replication of public_access
- new attribute catalog_id added in xml_documents and xml_revisions
as a FK to xml_catalog; catalog_id is replicated also

cleared DOCTITLE attribute in xml_documents and xml_revisions
cleared all mentions of doctitle

Solving the problem with relication of access file where access files were rejected from the replication servers.
In AccessControlList check for serverCode is included,
so only on the local server the connected user is checked for having "all" permissions on the resources specified in it...

included new servlet parameter "public" specifing public read access for the submitted document
the expected values are "yes" or "no"
public parameter is optional and if it is not specified, means "no"

added precise location information (class.method) to each catch statement so that errors are more easily traced.

Added license terms to source code files, and cleaned up some javadoc
documentation in a few places.

- turned on the validation in order only valid xml docs comformed to the specified dtd (if any) to be submitted in metacat
- with validation "on" white spaces are reported from ignorableWhitespace() callback, not from characters() (as with validation "off")...

- appling acl through access files only
- clearing around update/delete of access files - to delete the related records from both xml_access and xml_relation tables
- included check for user permissions for setting acl on the resources in the access file
- moved RelationHandler.deleteRelations(docid) down in RelationHandler obj just before the write of the new relations

Use the new property "accessdoctype" to check if this is access file.
If it is, run AccessControlList for wirting into xml_access and xml_relation tables.
This is the thread that writes XML Index but afterwards.

included support for submition of access files
+ storing the acl info in xml_access table and relationship records in xml_relation table like
<aclfile, "isaclfilefor", recourceIdentifier> for every resource included in the access file

removed dependence on a server code in the accession number of each document. the file's resident server is now located in xml_documents.server_location.

Fixed typo in DBSAXHandler debug message.

changed naming scheme

bug fix about util.openDBConnection() for xmlIndex tread

removed debug print statements.

allowed both indexing threads to run without interfering with each other.

Separate thread used for writing into xml_index table.
This cut the time of the response on insert almost in half.
The tread is started afterwards
(on end of document parsing and inserting into xml_nodes) from
DBSAXHandler and uses new(separate) db connection....

on character(cbuf, start, len) included:
if currentNode.getTagName().equals("title")
currentDocument.setTitle(cbuf)
instead of using:
currentDocument.setTitleFromChildElement() in endDocument() call

changes related to decrease the time of INSERT of document.
With these changes I inserted 200KB file for 3 minutes, 50KB for 50sec.
This is mainly simplifing the DBSAXNode class and
using batching feature of Oracle JDBC driver.

storing user_owner and user_updated where needed
new function in DocumentImpl checking for "write" perm on UPDATE or DELETE
added delete from xml_index of the old version of docid on UPDATE

Continued code redesign for the DocumentImpl class. Now the "delete" and
"write" methods are static, so a DocumentImpl objject need not be created
in order to initiate a INSERT, UPDATE, or DELETE action (it is created
implicitly by the DBSAXHandler). When doing a "READ" action, one still...

Folded the functionality from DBSAXDocument into the DocumentImpl class.
Now DocumentImpl handles both the reads and the writes to the database.
Eventually, it will implment the full DOM Document interface. Eliminated
the DBSAXDocument.java file as it is no longer relevant....

Added "release" keyword to all metacat source files so that the release
number will be evident in software distributions.

Call to DBSAXNode constructor simplified

Fixed bug in DBEntityResolver so that it now properly delegates to
the system id found inthe database.
Changed DBValidate to use DBEntityResolver, rather than the OASIS
catalog, and to return validation results in XML format.

Fixed bug where TEXT nodes couldn't be longer than 4000 characters, which
is the maximum length of a VARCHAR2 field in Oracle. Now, if text
exceeds the field length, I break the text up into a series of TEXT
nodes each of the max field length, and the remainder in the last...

Fixed bugs associated with the new UPDATE and DELETE functions of
DBWriter. There were problematic interactions between some static
variables used in DBEntityResolver and the way in which the
Servlet objects are re-used across multiple client invocations....

Merged in substantial changes to DBWriter and associated classes and to
the MetaCatServlet in order to accomodate the new UPDATE and DELETE
functions. The command line tools and the parameters for the
servlet have changed substantially.

Fixed bug in DBWriter using the new SAX2 interface. Turned out it was a bug in xerces-1_1_1, which was fixed by Andy Clark after I reported it.