|
1
|
<!--
|
|
2
|
* datafiles.html
|
|
3
|
*
|
|
4
|
* Authors: Chad Berkley
|
|
5
|
* Copyright: 2000 Regents of the University of California and the
|
|
6
|
* National Center for Ecological Analysis and Synthesis
|
|
7
|
* For Details: http://www.nceas.ucsb.edu/
|
|
8
|
* Created: 2001 January 23
|
|
9
|
* Version:
|
|
10
|
* File Info: '$ '
|
|
11
|
*
|
|
12
|
*
|
|
13
|
-->
|
|
14
|
<HTML>
|
|
15
|
<HEAD>
|
|
16
|
<TITLE>Metacat</TITLE>
|
|
17
|
<link rel="stylesheet" type="text/css" href="@docrooturl@default.css">
|
|
18
|
</HEAD>
|
|
19
|
<BODY>
|
|
20
|
<table width="100%">
|
|
21
|
<tr>
|
|
22
|
<td class="tablehead" colspan="2"><p class="label">Non-XML Data Files</p></td>
|
|
23
|
<td class="tablehead" colspan="2" align="right">
|
|
24
|
<a href="./replication.html">Back</a> | <a href="./metacattour.html">Home</a> |
|
|
25
|
<a href="./properties.html">Next</a>
|
|
26
|
</td>
|
|
27
|
</tr>
|
|
28
|
</table>
|
|
29
|
|
|
30
|
<p>Metacat has the capability of proxying a connection to a data file
|
|
31
|
server. This capability is provided through an abstract class called
|
|
32
|
DataFileUploadInterface. Currently, there is a data file server implemented
|
|
33
|
for this interface that can store non-XML data files on the local UNIX
|
|
34
|
file system. Theoretically, any file storage system could be proxied
|
|
35
|
through this interface.</p>
|
|
36
|
<P> <img alt="architecture diagram of the Metacat Data File Upload process"
|
|
37
|
src="dataload.gif"> </p>
|
|
38
|
|
|
39
|
<b>Reasoning</b>
|
|
40
|
<p>Since Metacat has been designed as a metadata storage system for
|
|
41
|
ecological data, there was a concern that data and metadata files were being
|
|
42
|
stored seperately and that users would find it impractical to have to use
|
|
43
|
two different systems to get their data and metadata. This system also allows
|
|
44
|
users to specify <a href="./acontrol.html">access control</a> restraints on their
|
|
45
|
data files the same way that they are specified on the xml metadata files.</p>
|
|
46
|
|
|
47
|
<b>Data Download (GET)</b>
|
|
48
|
<p>Metacat Server can retrieve data files that are stored on Metacat
|
|
49
|
file system or in any other file system on Internet. <br>
|
|
50
|
Metacat stores data files in a directory under the servlet context.
|
|
51
|
It writes information about these data files in xml_documents just like
|
|
52
|
about any XML file. <br>
|
|
53
|
An example of HTTP request for download of data file on Metacat is shown below: </p>
|
|
54
|
<pre><i>http://server.domain.com/metacat?action=read&docid=nceas.55</i></pre>
|
|
55
|
<p>An example of HTTP request for download of data file on Internet is shown below:</p>
|
|
56
|
<pre><i>http://server.domain.com/metacat?action=read&docid=http://otherserver.domain.com/filename</i></pre>
|
|
57
|
<p>Note in <i>docid=http://otherserver.domain.com/filename</i> HTTP protocol is used,
|
|
58
|
i.e. currently for file download HTTP protocol is only supported.</p>
|
|
59
|
|
|
60
|
<p><b>Data Upload (PUT)</b>
|
|
61
|
<p>Due to a Java limitation on the HTTP PUT comand, the data upload portion
|
|
62
|
of Metacat deviates from the standard HTTP interface. A standard bidirectional
|
|
63
|
TCP/IP socket is used for tranfering the data. The procedure for uploading a
|
|
64
|
file is as follows.</p>
|
|
65
|
<ol>
|
|
66
|
<li>The client must login to Metacat and get a session_id</li>
|
|
67
|
<li>The client sends a request to the servlet with an action of 'getdataport'.</li>
|
|
68
|
<li>The server responds with an xml message that includes a port number. The
|
|
69
|
message looks like: <pre><xml version="1.0"?><port>xxxx</port></pre>
|
|
70
|
where xxxx is an open port between 0 and 65000.</li>
|
|
71
|
<li>The client then can create a socket connection to the returned port.
|
|
72
|
Note that the client must make this connection within 30 seconds or the
|
|
73
|
port will close.</li>
|
|
74
|
<li>The data can now be sent but first, some extra information must be
|
|
75
|
appended onto the beginning of the data stream. The extra information looks
|
|
76
|
like: <pre> [filename]0[sessionID]0[filelength]0[DATA] </pre> The filename,
|
|
77
|
sessionID and filelength must be converted into a byte string, terminated
|
|
78
|
with a 0 (zero) byte and inserted into the stream in the order shown. The
|
|
79
|
filelength is in bytes. The DATA stream does not need to be terminated with
|
|
80
|
a 0 byte.</li>
|
|
81
|
<li>After the upload the server will return either an error message or
|
|
82
|
a success message which will include the docid of the new data file. Both
|
|
83
|
messages are encoded in xml (like the port message). The success message
|
|
84
|
looks like:
|
|
85
|
<pre><?xml version="1.0"?><docid>yyyy</docid></pre>
|
|
86
|
where yyyy is the new docid.</li>
|
|
87
|
</ul>
|
|
88
|
<p>The file DataStreamTest.java is a test class that shows how a client
|
|
89
|
must operate to successfully upload a data file to Metacat.</p>
|
|
90
|
|
|
91
|
<br>
|
|
92
|
<a href="./replication.html">Back</a> | <a href="./metacattour.html">Home</a> |
|
|
93
|
<a href="./properties.html">Next</a>
|
|
94
|
</BODY>
|
|
95
|
</HTML>
|
