Modified Metacat to support large data file uploads. This is accomplished by supporting a new content type for data sent to metacat: multipart/form-data which allows multiple files to be sent in a standard MIME format. The MetacatServlet.handleGetOrPost() method now checks the incoming content type and sends multipart/form-data encoded submissions to MetacatServlet.handleMultipartForm(), which in turn delegates to MetacatServlet.handleUploadAction(). When a client wants to load a data file to metacat, now create a "multipart/form-data" encoded stream that contains an "action" parameter set to the value "upload", a "docid" parameter set to the value of the accession number fro the file, and a "datafile" part that contains the actual file contents. Because of a bug / shortcoming in the com.oreilly.servlet library that I am using to parse this mime encoding, the "action" and "docid" paramters MUST come before the "datafile" part or metacat won't find the paramters.
The provided docid is inserted into xml_documents (after checking that it meets expectations [unique, rev 1, etc]). Then the data file recieved from the "upload" action is stored on the filesystem using the Accession number provided in the docid. See DocumentImpl.registerDocument for details.
This functionality completely replaces the socket-based functionality found in DataFileUploadInterface and DataFileServer. It would still be nice to abstract away the file storage functions so that they can be either written to the filesystem or other locations (like the db ina BLOB) when they are recieved in this new encoding format.
There are concomitant changes in Morpho to produce these types of file uploads. Morpho now uses the HTTPClient library for http communication, which allows streaming data between client and server (Sun's http protocol handler buffers all data before sending, which limits data transfer size to memory available). No changes in metacat were needed to accomodate this switch of protocol handlers on the client side; all changes were due to the need for an alternative data encoding (other than URL encoding) that efficiently supports binary files).
The cos.jar file is now needed for parsing the new encoding.
Modified Metacat to support large data file uploads. This is accomplished
by supporting a new content type for data sent to metacat:
multipart/form-data
which allows multiple files to be sent in a standard MIME format. The
MetacatServlet.handleGetOrPost() method now checks the incoming content
type and sends multipart/form-data encoded submissions to
MetacatServlet.handleMultipartForm(), which in turn delegates to
MetacatServlet.handleUploadAction(). When a client wants to load a data
file to metacat, now create a "multipart/form-data" encoded stream that
contains an "action" parameter set to the value "upload", a "docid"
parameter set to the value of the accession number fro the file, and
a "datafile" part that contains the actual file contents. Because of a
bug / shortcoming in the com.oreilly.servlet library that I am using to parse
this mime encoding, the "action" and "docid" paramters MUST come before
the "datafile" part or metacat won't find the paramters.
The provided docid is inserted into xml_documents (after checking that it
meets expectations [unique, rev 1, etc]). Then the data file
recieved from the "upload" action is stored on the filesystem
using the Accession number provided in the docid. See
DocumentImpl.registerDocument for details.
This functionality completely replaces the socket-based functionality found
in DataFileUploadInterface and DataFileServer. It would still be nice to
abstract away the file storage functions so that they can be either written
to the filesystem or other locations (like the db ina BLOB) when they are
recieved in this new encoding format.
There are concomitant changes in Morpho to produce these types of file uploads.
Morpho now uses the HTTPClient library for http communication, which
allows streaming data between client and server (Sun's http protocol
handler buffers all data before sending, which limits data transfer size
to memory available). No changes in metacat were needed to accomodate this
switch of protocol handlers on the client side; all changes were due to the
need for an alternative data encoding (other than URL encoding) that
efficiently supports binary files).
The cos.jar file is now needed for parsing the new encoding.