Metacat

  <div class="header1">Table of Contents</div>

  <div class="toc1"><a href="#Intro">Metacat Replication</a></div>

    <div class="toc2"><a href="#DatabasedInfo">Databased Information</a></div>

    <div class="toc2"><a href="#Example">Example</a></div>

      <div class="toc3"><a href="#gamma">What happens with gamma?</a></div>

      <div class="toc3"><a href="#alpha">What happens with alpha?</a></div>

      <div class="toc3"><a href="#lamda">What happens with lamda?</a></div>

  <div class="toc1"><a href="#Certificates">Certificates</a></div>

    <div class="toc2"><a href="#GenerateCertificates">Generate Certificates on both the replication client and server.</a></div> 

      <div class="toc3"><a href="#GenerateCertTomcat">Generate Certificate for Tomcat standalone (no Apache)</a></div>

      <div class="toc3"><a href="#GenerateCertApache">Generate Certificate for Apache/Tomcat</a></div>

    <div class="toc2"><a href="#RegisterPartner">Register the partner machines certificate</a></div> 

  <a name="Intro"></a><div class="header1">Metacat Replication</div>

  <p>Metacat has built-in replication to allow different Metacat servers to 

  share data between themselves. Metacat not only replicates XML documents but 

  also data files. </p>

  <p>Metacat's hub feature allows it to replicate not only it's own server's original

  documents, but also those that were replicated from other servers.  This functionality

  allows for a more complex chaining replication structure.</p>

  several triggers that can start a replication mechanism: </p>

  <ul class="list1">

    <li><b>Delta-T monitoring</b> - at a set time interval a server checks each of the

    <li><b>INSERT trigger</b> - Whenever a document is inserted, the server notifies

    <li><b>UPDATE trigger</b> - Whenever a document is updated, the server notifies

    <li><b>File locking</b> - When a local user tries to alter a document on a local 

  <a name="DatabasedInfo"></a><div class="header2">Databased Information</div>

  <br>

  The server list is kept in a table in the database called xml_replication.

  The database fields are:

  <ul class="list1">

  <li><b>serverid</b> - a unique ID that is generated by the database when a new field is added.</li>

  <li><b>server</b> - this field always points to the partner server's replication servlet,

  hence the "servlet/replication" on the end of both of the sample servers.  Note

  also be included. </li>

  <li><b>last_checked</b> - a system generated values that holds the last time that a check was 

  made to see if replication needed to be performed.<li>

  <li><b>replicate</b> - flag that is set to 1 if you want this server to replicate XML 

  metadata documents TO the remote host.  Note that if this flag is set to 0, datareplicate

  and hub fields have no meaning.</li>

  <li><b>datareplicate</b> - flag that is set to 1 if you want this server to copy data 

  files to the remote host.  Note that this field has no meaning if replicate is not set to 1.</li>

  If this server is a hub to the remote host, the hub flag should be set to.

  <li><b>hub</b> - if this flag is set to true, this server will not only replicate it's own

  original documents, it will also replicate documents that were replicated to it.  Thus it 

  acts as a replication hub to one or more other Metacat servers.</li>

  </ul>

  <a name="Example"></a><div class="header2">Example</div>

  Here we show an example setup of three replication servers.  We will discuss each.<br><br>

  First, note that in order for replication to occur, both partner servers must have 

  each other in their respective tables or replication will not take place.  Also, 

  certificates must be set up correctly on both servers in order for replication to 

  work.  See the <a href="#Certificates">certificates</a> section below.<br><br>

     <td>gamma.nceas.ucsb.edu</td>

          <td>lamda.nceas.ucsb.edu/Metacat/servlet/replication</td>

            <td>gamma.nceas.ucsb.edu:8080/berkley/servlet/replication</td>

            <td>lamda.nceas.ucsb.edu/Metacat/servlet/replication</td>

      <td>lamda.nceas.ucsb.edu</td>

            <td>gamma.nceas.ucsb.edu:8080/berkley/servlet/replication</td>

  <a name="gamma"></a><div class="header3">What happens with gamma?</div>

  <ul class="list1">

  <li>The localhost entry is required internally for replication to work on 

      gamma.  As long as we see it there, we can safely disregard it.</li>

  <li>We see the entry for the alpha machine has all zeros in replicate, 

      datareplicate and hub columns.  This means that gamma is configured to

      accept replication information from alpha.  (As we will see in a moment,

      alpha is not actually correctly configured to send data to gamma.)</li>

  <li>We see that the entry for the lamda machine has ones in the replicate

      and data replicate columns and a zero in the hub column.  This tells us

      that gamma will replicate it's original documents to lamda, assuming that

      lambda is configured to accept replication from gamma (we will see that it

      is).  However, because the hub value is zero, any documents that replicate 

      to gamma will not be further replicated to lamda.</li>

  </ul>

  <a name="alpha"></a><div class="header3">What happens with alpha?</div>

  <ul class="list1">

  <li>The localhost entry is required internally for replication to work on 

      alpha.  As long as we see it there, we can safely disregard it.</li>

  <li>We see that the entry for gamma has a zero in the replicate column.  

      This means that all other entries are meaningless and can be disregarded.

      Even though there is a one in the datareplicate column on alpha and gamma 

      is configured to accept replication from alpha, no replicationwill happen 

      from alpha to gamma.</li>

  <li>We see that the entry for lamda is a one in the replicate column and zeros

      in the datareplicate and hub columns.  Assuming lamda is configured to 

      accept replication from alpha, alpha will replicate metadata only to lamda 

      (and indeed, we will see that lambda is set up to accept replication from 

      alpha). </li>

  </ul>

  <a name="lamda"></a><div class="header3">What happens with lamda?</div>

  <ul class="list1">

  <li>The localhost entry is required internally for replication to work on 

      lamda.  As long as we see it there, we can safely disregard it.</li>

  <li>We see that the entry for gamma has all zeros in replicate, datareplicate

      and hub, so lamba is set up to accept replication from gamma.  As we have

      already seen, gamma is correctly configured to replicate metadata and data

      to lambda.  We should see data and metadata replication from gamma to lamda.

  <li>We see that the entry for alpha has ones in the replicate datareplicate and 

      hub columns.  There's a lot going on here:

    <ul class="list2">

    <li>First, lamda will replicate original metadata and data to alpha if 

        alpha is configured to accept replication from lamda.  Because alpha 

        has an entry for lambda, lamba will be allowed to replicate to alpha. </li>

    <li>Second, because the alpha entry has a one in the hub column, lambda 

        will not only replicate it's original data, it will also replicate 

        data that was replicated to it.  Remember that gamma was configured 

        to replicate to lamda.  So any data or metadata that gamma sends to 

        lambda will get further replicated to alpha.</li>

    <li>Finally, the alpha entry in the table allows the alpha server to 

        replicate to lambda.  Since the alpha server is set up to replicate

        metadata only, we would expect any original metadata on alpha to 

        wind up on lambda.</li>

    </ul>

  </ul>

  There is an html control panel for controling replication.  After

  called 'knb' you would probably type 

  <div class="code">http://server.domain.com:8080/Metacat/style/skins/dev/replControl.html</div>  

  The control panel is an easy interface for adding/removing/altering servers and 

  starting the delta-T handler.  It will also allow you to 'force replicate' your 

  server list.  This is useful if you want to initialize the state of one Metacat 

  server from an existing state of another (i.e. copy all of the data from an existing

  <a name="Certificates"></a><div class="header1">Certificates:</div>

  You will need to generate security certificates on both the replication client 

  and server.  The certificates will be exchanged so that each machine understands

  that the other has access for replication.<br><br>

  The following are the steps to generate and exchange certificates on systems

  running Tomcat 5 and java 1.5.  Note that if Tomcat is running in conjunction with

  Apache, the process is somewhat different than if it is running standalone.

  <a name="GenerateCertificates"></a><div class="header2">Generate Certificates on both the replication client and server.</div>  

  <a name="GenerateCertTomcat"></a><div class="header3">Generate Certificate for Tomcat standalone (no Apache)</div>

  <ul class="list1">

  <li>Generate keys in java default key store - this will create a secure key and put it

    into the binary certificates file located at $JAVA_HOME/lib/security/cacerts</li> 

    <ul class="list2">

    <li>Run the command: 

   	  <div class="code">keytool -genkey -alias &lt;aliasname&gt; -keyalg RSA -validity 800 -keystore cacerts</div>

     where <aliasname> is a unique name that you choose for this cert.  Something like "<hostname-tomcat&gt"

     might be appropriate.</li>

    </ul>

  </li>

  <li>Sample values when creating certificate</li>

    <ul class="list2">

    <li>What is your first and last name? <b>myserver.nceas.ucsb.edu </b>

        (note: use the host name without port number)<li>

    <li>What is the name of your organizional unit? <b>NCEAS</b></li>

    <li>What is the name of your organizional unit? <b>UCSB</b></li>

    <li>What is the name of your City or Locality? <b>Santa Barbara</b></li>

    <li>What is the name of your State or Province? <b>California</b> 

        (note: this is spelled in full)<li>

    <li>What is the two-letter country code for this unit? <b>US</b></li>

    </ul>

  <li>Generate certificate - this will pull the certificate you created from the cacerts file

      and put it into a local file</li>

    <ul class="list2">

    <li>Run the command:

      <div class="code">keytool -export -alias &lt;aliasname&gt; -file &lt;outputfile&gt;.cert -keystore cacerts</div>

      where &lt;aliasname&gt; is the same name you used when you created the certificate.  </li>

    <li>A file named &lt;outputfile&gt;.cert will be created in the same directory where you run the keytool 

      command.  You can name the output file anything you like, but keep in mind that it will get sent to the 

      partner machine used for replication.  The filename should have have enough meaning that someone who sees 

      it on that machine can have some idea where it came from.  Again, something like "<hostname>-tomcat.cert"

      will suffice.</li>   

    </ul>

  </li>

  <li>Enable SSL in Tomcat 

    <ul class="list2">

    <li>Edit the Tomcat server file at $TOMCAT_HOME/conf/server.xml</li>

    <li>uncomment the section that starts with "&lt;Connector port="8443" ...</li>

  	<li>add another attribute to that section that reads:

  	  <div class="code">keystoreFile="&lt;JAVA_HOME&gt;/lib/security/cacerts"</div>

  	  where $JAVA_HOME should be the actual java path.

  	</li>

  	</ul>

  </li>

  </ul>  

  <a name="GenerateCertApache"></a><div class="header3">Generate Certificate for Apache/Tomcat</div>

  <ul class="list1">

  <li>Generate keys using openssl

    <ul class="list2">

    <li>Run the command: 

   	  <div class="code">   openssl req -new -out REQ.pem -keyout &lt;hostname&gt;-apache.key</div>

    </li>

    </ul>

  </li>

  <li>Sample values when creating certificate</li>

    <ul class="list2">

    <li>Country Name (2 letter code) [AU]: <b>US</b></li>

    <li>State or Province Name (full name) [Some-State]: <b>California</b> 

        (note: this is spelled in full)</li>

    <li>Locality Name (eg, city) []: <b>Santa Barbara</b></li>

    <li>Organization Name (eg, company) [Internet Widgits Pty Ltd]: </b>UCSB</b></li>

    <li>Organizational Unit Name (eg, section) []: <b>NCEAS</b></li>

    <li>Common Name (eg, YOUR name) []: <b>myserver.mydomain.edu</b>

        (note: use the host name without port number)</li>

    <li>Email Address []:  <b>administrator@mydomain.edu</b></li>

    <li>A challenge password []: (note: leave blank)</li>

    <li>An optional company name []: (note: leave blank)</li>

    </ul>

  </li>    

  <li>Generate certificate - this will create a local file with your certificate</li>

    <ul class="list2">

    <li>Run the command:

      <div class="code">openssl req -x509 -days 800 -in REQ.pem -key &lt;hostname&gt;-apache.key -out &lt;hostname&gt;-apache.crt</div>

      where &lt;aliasname&gt; is the same name you used when you created the certificate.  </li>

    <li>A file named &lt;outputfile&gt;.cert will be created in the same directory where you run the keytool 

      command.  You can name the output file anything you like, but keep in mind that it will get sent to the 

      partner machine used for replication.  The filename should have have enough meaning that someone who sees 

      it on that machine can have some idea where it came from.  Again, something like "<hostname>-tomcat.cert"

      will suffice.</li>   

    </ul>

  </li>   

  <li>Enter the certificate into apache security configuration - you need to register the certificate

      in the local Apache instance.  Note that the security files may be in a different place depending

      on how you installed apache.</li>

    <ul class="list2">

    <li>Copy the certificate and key file to the apache ssl directories and enable ssl.</li>

    <li>For Ubuntu/Debian based systems:

      <ul class="list3">

      <li>sudo cp &lt;hostname&gt;-apache.crt /etc/ssl/certs</li>

      <li>sudo cp &lt;hostname&gt;-apache.key /etc/ssl/private</li>

      <li>As root edit /etc/apache2/sites-available/default.  In the VirtualHost section

          after the DocumentRoot line, add:<br>

          SSLEngine on<br>

          SSLOptions +FakeBasicAuth +ExportCertData +CompatEnvVars +StrictRequire<br>

          SSLCertificateFile /etc/ssl/certs/server.crt<br>

          SSLCertificateKeyFile /etc/ssl/private/server.key<br>

      </li>

      </ul>

    </li>  

    </ul>  

    <ul class="list2">

    <li>For other systems:

      <ul class="list3">

      <li>sudo cp &lt;hostname&gt;-apache.crt $APACHE_HOME/conf/ssl.crt</li>

      <li>sudo cp &lt;hostname&gt;-apache.key $APACHE_HOME/conf/ssl.key</li> 

      <li> ADD STEPS TO ENABLE SSL ON NON_DEBIAN SYSTEMS HERE</li>

      </ul>

    </li>  

    </ul>    	  	        

  <li>scp &lt;hostname&gt;-apache.crt to the replication partner machine.</li>

  </ul>  

  <a name="RegisterPartner"></a><div class="header2">Register the partner machines certificate.</div>   

  At this point, you have created a certificate for each replication server and 

  scp-ed them across to each other.  Now you need to import the remote server's

  certificate on the local machine.  Perform the following steps for each 

  replication server.

  <ul class="list1">

  <li>Import the remote certificate by running:

    <div class="code">keytool -import -alias &lt;remotehostalias&gt; -file &lt;remotehostfilename&gt;.cert -keystore cacerts</div>

    where the <remotehostfilename> is the certificate file you created on the remote machine and

    copied to this machine.  The <remotehostalias> is the name the certificate will use in

    the keystore.  It should be something that identifies the remote host.  

  </li>

  <li>Restart Apache and Tomcat on both replication machines</li>

  </ul>

  </ul>