1 |
6147
|
jones
|
Replication
|
2 |
|
|
===========
|
3 |
6845
|
jones
|
Metacat has a built-in replication feature that allows different Metacat servers
|
4 |
|
|
to share data (both XML documents and data files) between each other. Metacat
|
5 |
|
|
can replicate not only its home server's original documents, but also those
|
6 |
|
|
that were replicated from partner Metacat servers. When changes are made to
|
7 |
|
|
one server in a replication network, the changes are automatically propogated
|
8 |
|
|
to the network, even if the network is down.
|
9 |
6147
|
jones
|
|
10 |
6845
|
jones
|
Replication allows users to manage their data locally and (by replicating them
|
11 |
|
|
to a shared Metacat repository) to make those data available to the greater
|
12 |
|
|
scientific community via a centralized search. In other words, your Metacat can
|
13 |
|
|
be part of a broader network, but you retain control over the local repository
|
14 |
|
|
and how it is managed.
|
15 |
6147
|
jones
|
|
16 |
6845
|
jones
|
For example, the KNB Network (Figure 6.1), which currently consists of ten
|
17 |
|
|
different Metacat servers from around the world, uses replication to "join"
|
18 |
|
|
the disperate servers to form a single robust and searchable data
|
19 |
|
|
repository--facilitating data discovery, while leaving the data ownership and
|
20 |
|
|
management with the local administrators.
|
21 |
6147
|
jones
|
|
22 |
6845
|
jones
|
.. figure:: images/screenshots/image059.jpg
|
23 |
|
|
:align: center
|
24 |
|
|
|
25 |
|
|
A map of the KNB Metacat network.
|
26 |
6147
|
jones
|
|
27 |
6845
|
jones
|
When properly configured, Metacat's replication mechanism can be triggered by
|
28 |
|
|
several types of events that occur on either the home or partner server: a
|
29 |
|
|
document insertion, an update, or an automatic replication (i.e., Delta-T
|
30 |
|
|
monitoring), which is set at a user-specified time interval.
|
31 |
|
|
|
32 |
|
|
+----------------------+----------------------------------------------------------+
|
33 |
|
|
| Replication Triggers | Description |
|
34 |
|
|
+======================+==========================================================+
|
35 |
|
|
| Insert | Whenever a document is inserted into Metacat, the server |
|
36 |
|
|
| | notifies each server in its replication list |
|
37 |
|
|
| | that it has a new file available. |
|
38 |
|
|
+----------------------+----------------------------------------------------------+
|
39 |
|
|
| Update | Whenever a document is updated, the server notifies |
|
40 |
|
|
| | each server in its replication list of the update. |
|
41 |
|
|
+----------------------+----------------------------------------------------------+
|
42 |
|
|
| Delta-T monitoring | At a user-specified time interval, Metacat checks each |
|
43 |
|
|
| | of the servers in its replication list |
|
44 |
|
|
| | for updated documents. |
|
45 |
|
|
+----------------------+----------------------------------------------------------+
|
46 |
|
|
|
47 |
|
|
Configuring Replication
|
48 |
|
|
-----------------------
|
49 |
|
|
To configure replication, you must configure both the home and partner servers:
|
50 |
|
|
|
51 |
|
|
1. Create a list of partner servers on your home server using the Replication Control Panel
|
52 |
|
|
2. Create certificate files for the home server
|
53 |
|
|
3. Create certificate files for the partner server
|
54 |
|
|
4. Import partner certificate files to the home server
|
55 |
|
|
5. Import home certificate to the partner server
|
56 |
|
|
6. Update your Metacat database
|
57 |
|
|
|
58 |
|
|
Each step is discussed in more detail in the following sections.
|
59 |
|
|
|
60 |
|
|
Using the Replication Control Panel
|
61 |
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
62 |
|
|
To add, remove, or alter servers on your home server's Replication list, or to
|
63 |
|
|
activate and customize the Delta-T handler, use the Replication control panel,
|
64 |
|
|
which is accessed at the following URL::
|
65 |
|
|
|
66 |
|
|
http://somehost.somelocation.edu/context/style/skins/dev/replControl.html
|
67 |
|
|
|
68 |
|
|
"http://somehost.somelocation.edu/context" should be replaced with the name
|
69 |
|
|
of your Metacat server and context (e.g., http://knb.ecoinformatics.org/knb/).
|
70 |
|
|
You must be logged in to Metacat as an administrator.
|
71 |
|
|
|
72 |
|
|
.. figure:: images/screenshots/image061.jpg
|
73 |
|
|
:align: center
|
74 |
|
|
|
75 |
|
|
Replication control panel.
|
76 |
|
|
|
77 |
|
|
Note that currently, you cannot use the Replication Control Panel to remove a
|
78 |
|
|
server after a replication has occurred. At this point in time, the only way to
|
79 |
|
|
remove a replication server after replication has occurred is to remove the
|
80 |
|
|
certificates.
|
81 |
|
|
|
82 |
|
|
Also note that you must SCP partner certificates to your machine; you cannot
|
83 |
|
|
use the "Download Certificate from" option on the Control Panel. For more
|
84 |
|
|
information about creating and installing certificates, please see Generating
|
85 |
|
|
and Exchanging Security Certificates.
|
86 |
|
|
|
87 |
|
|
Generating and Exchanging Security Certificates
|
88 |
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
89 |
|
|
Before you can take advantage of Metacat's replication feature, you must
|
90 |
|
|
generate security certificates on both the replication partner and home servers.
|
91 |
|
|
The certificates will be exchanged so that each machine understands that the
|
92 |
|
|
other has replication access.
|
93 |
|
|
|
94 |
|
|
The process for generating certificates is different for Metacat servlets
|
95 |
|
|
running under Tomcat and those under Tomcat/Apache (the recommended configuration).
|
96 |
|
|
For instructions on generating and exchanging certificates on systems running
|
97 |
|
|
only Tomcat (and Java 6), see Generating a Certificate for Tomcat standalone
|
98 |
|
|
(no Apache).
|
99 |
|
|
|
100 |
|
|
Generate Certificates for Metacat running under Apache/Tomcat
|
101 |
|
|
.............................................................
|
102 |
|
|
Note: Instructions are for Ubuntu/Debian systems.
|
103 |
|
|
|
104 |
|
|
1. Generate a certificate key using openssl. The key will be named
|
105 |
|
|
``<hostname>-apache.key``, where ``<hostname>`` is the name of your Metacat
|
106 |
|
|
server. Example values for the individual key fields are included in the
|
107 |
|
|
table below.
|
108 |
|
|
|
109 |
|
|
::
|
110 |
|
|
|
111 |
|
|
openssl req -new -out REQ.pem -keyout <hostname>-apache.key
|
112 |
|
|
|
113 |
|
|
+--------------------------+-------------------------------------------------------------------------+
|
114 |
|
|
| Key Field | Description and Example Value |
|
115 |
|
|
+==========================+=========================================================================+
|
116 |
|
|
| Country Name | Two letter country code (e.g., US) |
|
117 |
|
|
+--------------------------+-------------------------------------------------------------------------+
|
118 |
|
|
| State or Province Name | The name of your state or province spelled in full (e.g., California) |
|
119 |
|
|
+--------------------------+-------------------------------------------------------------------------+
|
120 |
|
|
| Locality Name | The name of your city (e.g., Santa Barbara) |
|
121 |
|
|
+--------------------------+-------------------------------------------------------------------------+
|
122 |
|
|
| Organization Name | The company or organization name (e.g., UCSB) |
|
123 |
|
|
+--------------------------+-------------------------------------------------------------------------+
|
124 |
|
|
| Organizational Unit Name | The department or section name (e.g., NCEAS) |
|
125 |
|
|
+--------------------------+-------------------------------------------------------------------------+
|
126 |
|
|
| Common Name | The host server name without port numbers (e.g., myserver.mydomain.edu) |
|
127 |
|
|
+--------------------------+-------------------------------------------------------------------------+
|
128 |
|
|
| Email Address | Administrator's contact email (e.g., administrator@mydomain.edu) |
|
129 |
|
|
+--------------------------+-------------------------------------------------------------------------+
|
130 |
|
|
| A challenge password | --leave this field blank-- |
|
131 |
|
|
+--------------------------+-------------------------------------------------------------------------+
|
132 |
|
|
| An optional company name | --leave this field blank-- |
|
133 |
|
|
+--------------------------+-------------------------------------------------------------------------+
|
134 |
|
|
|
135 |
|
|
2. Create the local certificate file by running the command:
|
136 |
|
|
|
137 |
|
|
::
|
138 |
|
|
|
139 |
|
|
openssl req -x509 -days 800 -in REQ.pem -key <hostname>-apache.key -out <hostname>-apache.crt
|
140 |
|
|
|
141 |
|
|
Use the same ``<hostname>`` you used when you generated the key. A file named
|
142 |
|
|
``<hostname>-apache.crt`` will be created in the directory from which you
|
143 |
|
|
ran the openssl command. Note: You can name the certificate file anything
|
144 |
|
|
you'd like, but keep in mind that the file will be sent to the partner
|
145 |
|
|
machine used for replication. The certificate name should have enough
|
146 |
|
|
meaning that someone who sees it on that machine can figure out where it
|
147 |
|
|
came from.
|
148 |
|
|
|
149 |
|
|
3. Enter the certificate into Apache's security configuration. You must
|
150 |
|
|
register the certificate in the local Apache instance. Note that the
|
151 |
|
|
security files may be in a different directory from the one used in the
|
152 |
|
|
instructions depending on how you installed Apache. Copy the certificate and
|
153 |
|
|
key file using the following commands:
|
154 |
|
|
|
155 |
|
|
::
|
156 |
|
|
|
157 |
|
|
sudo cp <hostname>-apache.crt /etc/ssl/certs
|
158 |
|
|
sudo cp <hostname>-apache.key /etc/ssl/private
|
159 |
|
|
|
160 |
|
|
4. Apache needs to know about Metacat SSL. The helper file named "knb-ssl" has
|
161 |
|
|
rules that tell Apache which traffic to route to the Metacat SSL port. Set up
|
162 |
|
|
SSL by dropping the knb-ssl file into the sites-available directory and
|
163 |
|
|
running ``a2ensite`` to enable the site:
|
164 |
|
|
|
165 |
|
|
::
|
166 |
|
|
|
167 |
|
|
sudo cp <metacat_helper_dir>/knb-ssl <apache_install_dir>/sites-available
|
168 |
|
|
sudo a2ensite knb-ssl
|
169 |
|
|
|
170 |
|
|
5. Restart Apache to bring in changes by typing:
|
171 |
|
|
|
172 |
|
|
::
|
173 |
|
|
|
174 |
|
|
sudo /etc/init.d/apache2 restart
|
175 |
|
|
|
176 |
|
|
6. SCP ``<hostname>-apache.crt`` to the replication partner machine.
|
177 |
|
|
|
178 |
|
|
Generating a Certificate for Tomcat standalone (no Apache)
|
179 |
|
|
..........................................................
|
180 |
|
|
If you are running Metacat under Tomcat (no Apache), generate keys in the Java
|
181 |
|
|
default key store. The generated key is placed into the binary certificate's
|
182 |
|
|
file located at ``/etc/java-1.5.0-sun/security/cacerts``.
|
183 |
|
|
|
184 |
|
|
1. Generate the key by running the following command (note that you must be
|
185 |
|
|
logged in as the root user to use the keytool):
|
186 |
|
|
|
187 |
|
|
::
|
188 |
|
|
|
189 |
|
|
keytool -genkey -alias <aliasname> -keyalg RSA -validity 800 -keystore /etc/java-1.6.0-sun/security/cacerts
|
190 |
|
|
|
191 |
|
|
``<aliasname>`` is a unique name that you choose for this key. Something
|
192 |
|
|
like "<hostname-tomcat>" might be appropriate, where ``<hostname-tomcat>``
|
193 |
|
|
is the name of the Metacat host.
|
194 |
|
|
|
195 |
|
|
2. The Password-keytool will ask for a password. If writing to a pre-existing
|
196 |
|
|
keystore, you must know the password. If you are creating a new keystore,
|
197 |
|
|
the password you enter will become the keystore password.
|
198 |
|
|
|
199 |
|
|
Sample values when creating certificate:
|
200 |
|
|
|
201 |
|
|
::
|
202 |
|
|
|
203 |
|
|
What is your first and last name? myserver.nceas.ucsb.edu (note: use the host name without port number)
|
204 |
|
|
What is the name of your organizional unit? NCEAS
|
205 |
|
|
What is the name of your organizional unit? UCSB
|
206 |
|
|
What is the name of your City or Locality? Santa Barbara
|
207 |
|
|
What is the name of your State or Province? California (note: this is spelled in full)
|
208 |
|
|
What is the two-letter country code for this unit? US
|
209 |
|
|
|
210 |
|
|
3. Create a certificate by running the command:
|
211 |
|
|
|
212 |
|
|
::
|
213 |
|
|
|
214 |
|
|
keytool -export -alias <aliasname> -file <outputfile>.cert -keystore /etc/java-1.6.0-sun/security/cacerts
|
215 |
|
|
|
216 |
|
|
``<aliasname>`` is the same name you used when you created the key file. A
|
217 |
|
|
file named ``<outputfile>.cert`` will be created in the directory from which
|
218 |
|
|
you ran the keytool command. You can name the output file anything you like,
|
219 |
|
|
but keep in mind that it will be sent to the partner machine used for
|
220 |
|
|
replication. The filename should have enough meaning that someone who sees
|
221 |
|
|
it on that machine can figure out where it came from. Again, something like
|
222 |
|
|
"<hostname>-tomcat.cert" will suffice.
|
223 |
|
|
|
224 |
|
|
4. Edit the Tomcat server file at ``$TOMCAT_HOME/conf/server.xml`` to enable
|
225 |
|
|
SSL in Tomcat.
|
226 |
|
|
|
227 |
|
|
* Uncomment the section that starts with "<Connector port="8443" ...
|
228 |
|
|
(Note: Databased Information comments start with <!-- and end with -->).
|
229 |
|
|
|
230 |
|
|
* Add two attribute to that section:
|
231 |
|
|
|
232 |
|
|
::
|
233 |
|
|
|
234 |
|
|
keystoreFile="/etc/java-1.6.0-sun/security/cacerts"
|
235 |
|
|
keystorePass="<keystore_password>"
|
236 |
|
|
|
237 |
|
|
where ``<keystore_password>`` is the password you used when you created
|
238 |
|
|
or accessed the keystore.
|
239 |
|
|
|
240 |
|
|
5. SCP the certificate to the partner server.
|
241 |
|
|
|
242 |
|
|
To import a certificate
|
243 |
|
|
.......................
|
244 |
|
|
1. Log in as a root user (the keytool must run as a root user)
|
245 |
|
|
|
246 |
|
|
::
|
247 |
|
|
|
248 |
|
|
sudo su –
|
249 |
|
|
|
250 |
|
|
2. Import the remote certificate by running:
|
251 |
|
|
|
252 |
|
|
::
|
253 |
|
|
|
254 |
|
|
keytool -import -alias <remotehostalias> -file <remotehostfilename>.crt -keystore /etc/java-1.6.0-sun/security/cacerts
|
255 |
|
|
|
256 |
|
|
where the ``<remotehostfilename>`` is the name of the certificate file
|
257 |
|
|
created on the remote partner machine and SCP'd to the home machine.
|
258 |
|
|
The ``<remotehostalias>`` is the name the certificate will use in the
|
259 |
|
|
keystore. The name should identify the remote host.
|
260 |
|
|
|
261 |
|
|
Update your Metacat database
|
262 |
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
263 |
|
|
The simplest way to update the Metacat database to use replication is to use
|
264 |
|
|
the Replication Control Panel. You can also update the database u
|
265 |
|
|
sing SQL. Instructions for both options are included in this section.
|
266 |
|
|
|
267 |
|
|
.. figure:: images/screenshots/image063.jpg
|
268 |
|
|
:align: center
|
269 |
|
|
|
270 |
|
|
Using the Replication Control Panel to update the Metacat database.
|
271 |
|
|
|
272 |
|
|
To update your Metacat database to use replication, select the "Add this server"
|
273 |
|
|
radio button from the Replication Control Panel, enter the partner server name,
|
274 |
|
|
and specify how the replication should occur (whether to replicate xml, data,
|
275 |
|
|
or use the local machine as a hub). Note that you cannot download certificates
|
276 |
|
|
using this interface.
|
277 |
|
|
|
278 |
|
|
To update the database using SQL
|
279 |
|
|
................................
|
280 |
|
|
|
281 |
|
|
1. Log in to the database
|
282 |
|
|
|
283 |
|
|
::
|
284 |
|
|
|
285 |
|
|
psql -U metacat -W -h localhost metacat
|
286 |
|
|
|
287 |
|
|
2. Select all rows from the replication table
|
288 |
|
|
|
289 |
|
|
::
|
290 |
|
|
|
291 |
|
|
select * from xml_replication;
|
292 |
|
|
|
293 |
|
|
3. Insert the partner server.
|
294 |
|
|
|
295 |
|
|
::
|
296 |
|
|
|
297 |
|
|
INSERT INTO xml_replication (server,last_checked,replicate,datareplicate,hub) VALUES ('<partner.server/context>/servlet/replication',NULL,1,1,0);
|
298 |
|
|
|
299 |
|
|
Where ``<partner.server/context>`` is the name of the partner server and
|
300 |
|
|
context. The values 'NULL, 1,1,0' indicate (respectively) the last time
|
301 |
|
|
replication occurred, that XML docs should be replicated to the partner
|
302 |
|
|
server, that data files should be replicated to the partner server, and
|
303 |
|
|
that the local server should not act as a hub. Set a value of 'NULL,0,0,0'
|
304 |
|
|
if your Metacat is only receiving documents from the partner site and not
|
305 |
|
|
replicating to that site.
|
306 |
|
|
|
307 |
|
|
4. Exit the database
|
308 |
|
|
5. Restart Apache and Tomcat on both home and partner replication machines
|