Project

General

Profile

1
<!--
2
  * datafiles.html
3
  *
4
  *      Authors: Chad Berkley
5
  *    Copyright: 2000 Regents of the University of California and the
6
  *               National Center for Ecological Analysis and Synthesis
7
  *  For Details: http://www.nceas.ucsb.edu/
8
  *      Created: 2001 January 23
9
  *      Version: 
10
  *    File Info: '$ '
11
  * 
12
  * 
13
-->
14
<HTML>
15
<HEAD>
16
<TITLE>Metacat</TITLE>
17
<link rel="stylesheet" type="text/css" href="@docrooturl@default.css">
18
</HEAD> 
19
<BODY>
20
  <table width="100%">
21
    <tr>
22
      <td class="tablehead" colspan="2"><p class="label">Non-XML Data Files</p></td>
23
      <td class="tablehead" colspan="2" align="right">
24
        <a href="./replication.html">Back</a> | <a href="./metacattour.html">Home</a> | 
25
        <a href="./properties.html">Next</a>
26
      </td>
27
    </tr>
28
  </table>
29
  
30
  <p>Metacat has the capability of proxying a connection to a data file 
31
  server.  This capability is provided through an abstract class called
32
  DataFileUploadInterface.  Currently, there is a data file server implemented
33
  for this interface that can store non-XML data files on the local UNIX
34
  file system.  Theoretically, any file storage system could be proxied
35
  through this interface.</p>
36
  <P> <img alt="architecture diagram of the Metacat Data File Upload process" 
37
           src="dataload.gif">  </p>
38

    
39
  <b>Reasoning</b>
40
  <p>Since Metacat has been designed as a metadata storage system for 
41
  ecological data, there was a concern that data and metadata files were being
42
  stored seperately and that users would find it impractical to have to use
43
  two different systems to get their data and metadata.  This system also allows
44
  users to specify <a href="./acontrol.html">access control</a> restraints on their
45
  data files the same way that they are specified on the xml metadata files.</p>
46

    
47
  <b>Data Download (GET)</b>
48
  <p>Metacat Server can retrieve data files that are stored on Metacat
49
     file system or in any other file system on Internet. <br>
50
     Metacat stores data files in a directory under the servlet context.
51
     It writes information about these data files in xml_documents just like
52
     about any XML file. <br>
53
     An example of HTTP request for download of data file on Metacat is shown below: </p>
54
  <pre><i>http://server.domain.com/metacat?action=read&amp;docid=nceas.55</i></pre>
55
  <p>An example of HTTP request for download of data file on Internet is shown below:</p>
56
  <pre><i>http://server.domain.com/metacat?action=read&amp;docid=http://otherserver.domain.com/filename</i></pre>
57
  <p>Note in <i>docid=http://otherserver.domain.com/filename</i> HTTP protocol is used,
58
     i.e. currently for file download HTTP protocol is only supported.</p>
59

    
60
  <p><b>Data Upload (PUT)</b>
61
  <p>Due to a Java limitation on the HTTP PUT comand, the data upload portion
62
  of Metacat deviates from the standard HTTP interface.  A standard bidirectional 
63
  TCP/IP socket is used for tranfering the data.  The procedure for uploading a 
64
  file is as follows.</p>
65
  <ol>
66
    <li>The client must login to Metacat and get a session_id</li>
67
    <li>The client sends a request to the servlet with an action of 'getdataport'.</li>
68
    <li>The server responds with an xml message that includes a port number.  The
69
    message looks like: <pre>&lt;xml version="1.0"?&gt;&lt;port&gt;xxxx&lt;/port&gt;</pre>
70
    where xxxx is an open port between 0 and 65000.</li>
71
    <li>The client then can create a socket connection to the returned port.
72
    Note that the client must make this connection within 30 seconds or the 
73
    port will close.</li>
74
    <li>The data can now be sent but first, some extra information must be
75
    appended onto the beginning of the data stream.  The extra information looks
76
    like: <pre> [filename]0[sessionID]0[filelength]0[DATA] </pre> The filename, 
77
    sessionID and filelength must be converted into a byte string, terminated
78
    with a 0 (zero) byte and inserted into the stream in the order shown.  The
79
    filelength is in bytes.  The DATA stream does not need to be terminated with
80
    a 0 byte.</li>
81
    <li>After the upload the server will return either an error message or
82
    a success message which will include the docid of the new data file.  Both
83
    messages are encoded in xml (like the port message). The success message
84
    looks like:
85
    <pre>&lt;?xml version="1.0"?&gt;&lt;docid&gt;yyyy&lt;/docid&gt;</pre>
86
    where yyyy is the new docid.</li>
87
  </ul>
88
  <p>The file DataStreamTest.java is a test class that shows how a client
89
  must operate to successfully upload a data file to Metacat.</p>
90
  
91
  <br>
92
  <a href="./replication.html">Back</a> | <a href="./metacattour.html">Home</a> | 
93
  <a href="./properties.html">Next</a>
94
</BODY>
95
</HTML>
(5-5/46)