/docs/user/datafiles.html - Annotate - Metacat - Ecoinformatics Redmine

878

berkley

<!--

2

  * datafiles.html

3

4

  *      Authors: Chad Berkley

5

  *    Copyright: 2000 Regents of the University of California and the

6

  *               National Center for Ecological Analysis and Synthesis

7

  *  For Details: http://www.nceas.ucsb.edu/

8

  *      Created: 2001 January 23

9

  *      Version:

10

  *    File Info: '$ '

11

12

13

-->

14

<HTML>

15

<HEAD>

16

<TITLE>Metacat</TITLE>

17

<link rel="stylesheet" type="text/css" href="@docrooturl@default.css">

18

</HEAD>

19

<BODY>

20

  <table width="100%">

21

    <tr>

22

      <td class="tablehead" colspan="2"><p class="label">Non-XML Data Files</p></td>

23

      <td class="tablehead" colspan="2" align="right">

24

        <a href="./replication.html">Back</a> | <a href="./metacattour.html">Home</a> |

25

        <a href="./properties.html">Next</a>

26

      </td>

27

    </tr>

28

  </table>

29

30

  <p>Metacat has the capability of proxying a connection to a data file

31

  server.  This capability is provided through an abstract class called

32

  DataFileUploadInterface.  Currently, there is a data file server implemented

33

  for this interface that can store non-XML data files on the local UNIX

34

  file system.  Theoretically, any file storage system could be proxied

35

  through this interface.</p>

36

  <P> <img alt="architecture diagram of the Metacat Data File Upload process"

37

           src="dataload.gif">  </p>

38

39

  <b>Reasoning</b>

40

  <p>Since Metacat has been designed as a metadata storage system for

41

  ecological data, there was a concern that data and metadata files were being

42

  stored seperately and that users would find it impractical to have to use

43

  two different systems to get their data and metadata.  This system also allows

44

  users to specify <a href="./acontrol.html">access control</a> restraints on their

45

  data files the same way that they are specified on the xml metadata files.</p>

46

47

  <b>Data Download (GET)</b>

48

  <p>Metacat Server can retrieve data files that are stored on Metacat

49

     file system or in any other file system on Internet. <br>

50

     Metacat stores data files in a directory under the servlet context.

51

     It writes information about these data files in xml_documents just like

52

     about any XML file. <br>

53

     An example of HTTP request for download of data file on Metacat is shown below: </p>

54

  <pre><i>http://server.domain.com/metacat?action=read&amp;docid=nceas.55</i></pre>

55

  <p>An example of HTTP request for download of data file on Internet is shown below:</p>

56

  <pre><i>http://server.domain.com/metacat?action=read&amp;docid=http://otherserver.domain.com/filename</i></pre>

57

  <p>Note in <i>docid=http://otherserver.domain.com/filename</i> HTTP protocol is used,

58

     i.e. currently for file download HTTP protocol is only supported.</p>

59

60

  <p><b>Data Upload (PUT)</b>

61

  <p>Due to a Java limitation on the HTTP PUT comand, the data upload portion

62

  of Metacat deviates from the standard HTTP interface.  A standard bidirectional

63

  TCP/IP socket is used for tranfering the data.  The procedure for uploading a

64

  file is as follows.</p>

65

  <ol>

66

    <li>The client must login to Metacat and get a session_id</li>

67

    <li>The client sends a request to the servlet with an action of 'getdataport'.</li>

68

    <li>The server responds with an xml message that includes a port number.  The

69

    message looks like: <pre>&lt;xml version="1.0"?&gt;&lt;port&gt;xxxx&lt;/port&gt;</pre>

70

    where xxxx is an open port between 0 and 65000.</li>

71

    <li>The client then can create a socket connection to the returned port.

72

    Note that the client must make this connection within 30 seconds or the

73

    port will close.</li>

74

    <li>The data can now be sent but first, some extra information must be

75

    appended onto the beginning of the data stream.  The extra information looks

76

    like: <pre> [filename]0[sessionID]0[filelength]0[DATA] </pre> The filename,

77

    sessionID and filelength must be converted into a byte string, terminated

78

    with a 0 (zero) byte and inserted into the stream in the order shown.  The

79

    filelength is in bytes.  The DATA stream does not need to be terminated with

80

    a 0 byte.</li>

81

    <li>After the upload the server will return either an error message or

82

    a success message which will include the docid of the new data file.  Both

83

    messages are encoded in xml (like the port message). The success message

84

    looks like:

85

    <pre>&lt;?xml version="1.0"?&gt;&lt;docid&gt;yyyy&lt;/docid&gt;</pre>

86

    where yyyy is the new docid.</li>

87

  </ul>

88

  <p>The file DataStreamTest.java is a test class that shows how a client

89

  must operate to successfully upload a data file to Metacat.</p>

90

91

  <br>

92

  <a href="./replication.html">Back</a> | <a href="./metacattour.html">Home</a> |

93

  <a href="./properties.html">Next</a>

94

</BODY>

95

</HTML>

Project

General

Profile

Metacat

Project

General

Profile

Metacat

metacat/docs/user/datafiles.html @ 942