Metacat

I originally tried to write DOMToDBMS as a SAX application but concluded

that it was impossible. However, this conclusion was based on the

assumption that the class did no caching of data and, in this case, it

was impossible. However, if some data is cached as Row objects, it is

quite possible that SAXToDBMS is possible. The problem that needs to

be solved is as follows.

Consider the sample Sales Order document. This has data for four tables:

Sales, Customers, Lines, and Parts.  Now consider the key relationships:

Sales has a foreign key that points to Customers and a primary key that

points to Lines, and Lines has a foreign key that points to Parts.  The

trick is that, to meet referential integrity constraints in the database,

you need to add the rows with primary keys first. Thus, you need to add the

Customers row before you can add the Sales row, you need to add the Sales

row before you can add the Lines row, and you need to add the Parts row

before you can add the Lines row.

The tree-walking algorithm I use is as follows. Start from the root. When

you encounter a node mapped to a table, such as the SalesOrder node, create

a buffer for a row in that table, then processes the children of that node.

If a child is mapped to a column, such as the OrderDate node, store the

node's value in the buffer. If a child is mapped to a table that stores a

foreign key in the current table, such as the Customer node, process the

node immediately and store the foreign key in the current row buffer. If a

child is mapped to a table that uses the primary key of the current table,

such as the Line nodes, save the node is for later processing, which you do

after inserting the parent node, so you can pass the primary key to the

child node.

In my original SAX implementation, I did not use row buffers. Thus, the

implementation was clearly impossible -- I would, for example, have to

immediately process (and insert) the sales data, which comes before the

customer data. This leads to referential integrity problems. However,

you've got me thinking now and it's quite possible you might be able to

build the row buffers as you go, then insert them in the correct order. If

so, this would be terrific.

Anyway, the general strategy is something along the lines of:

**********

In startElement().

1) Get the map for the element.

2) If the element is mapped as a class:

a) create a new Row for the element.

b) read the attributes linearly, getting each name and value.

c) get the property map for each attribute. if it is stored as TOCOLUMN,

store the value in the current Row. if it is stored in as TOPROPERTYTABLE,

create a Row for the property table and store the value in it.

3) If the element is mapped as a property then, if the element is mapped as

TOCOLUMN, do nothing now. If it is mapped as TOPROPERTYTABLE, create a Row

for the property table.

********

In characters():

1) If you are processing an element mapped as a property then:

a) If the element is mapped as TOCOLUMN, store the value in the parent

element's Row.

b) If the element is mapped as TOPROPERTYTABLE, store the value in the

property table Row.

2) If you are processing an element mapped as a class, get the map for the

PCDATA. If this is mapped as TOCOLUMN, store the value in the element's

Row. If this is mapped as TOPROPERTYTABLE, create a Row for the property

table and store the value in it.

3) IMPORTANT! A single piece of PCDATA can result in multiple calls to

characters, so you need to deal with this somehow -- for example, appending

data to column values rather than just creating a string and storing it in

the Row.

*********

In endElement():

1) If the element is mapped as a property, I don't think there is anything

to do.

2) If the element is mapped as a class, then (I'm thinking out loud here),

by reaching the end of the element, you know that all child elements are

done. You don't need to worry about child elements-as-properties, as

processing for them is finished.  All you care about is child

elements-as-classes and their offspring.  I think what you do is

recursively process these as follows:

a) First process all child elements-as-classes that contain primary keys.

These need to be inserted and their foreign keys stored in the parent row.

b) Next insert the parent row.

c) Finally, insert the primary key from the parent row into the children

skipped in (a) and insert these children.

Note that when I talk about child elements-as-classes, I think I am also

referring to property tables for the current parent.

This is the only really tricky part of the code, as these elements could be

nested quite deep and you need to keep track of which have been inserted

and which haven't. However, the more I talk about this, the more I believe

it is possible. Furthermore, although you end up caching a certain amount

of data, I think it's a good bet that this approaches the same memory

requirements as the DOM for trees in which there are no siblings, only

parents and children. (For example, if your tree has a root with 10000

children, each of which has three children and no grandchildren, then

assuming the root is ignored, you will, at most, only cache the original

child and its three children at any one point in time. On the other hand,

if your tree has only parents and children, then depending on the PK/FK

relationships, you might have to cache the whole thing and insert the most

deeply nested child first.  In any case, caching the rows and keeping their

maps around is likely to be far less expensive than the equivalent DOM.)

The last thing to consider is that you need to keep order information about

the children of each element-as-class. Because elements can be nested,

there will be a stack of order information, as you need to maintain this on

a per-level basis.  I don't think this will be too hard -- just a push

every time you start an element-as-class, a pop every you end an

element-as-class, and an increment every time you start an

element-as-property or call to characters (remembering, of course, the

problem with multiple calls to characters for a single piece of PCDATA).