Revision 6845
Added by Matt Jones over 12 years ago
replication.rst | ||
---|---|---|
1 | 1 |
Replication |
2 | 2 |
=========== |
3 |
Metacat has a built-in replication feature that allows different Metacat servers |
|
4 |
to share data (both XML documents and data files) between each other. Metacat |
|
5 |
can replicate not only its home server's original documents, but also those |
|
6 |
that were replicated from partner Metacat servers. When changes are made to |
|
7 |
one server in a replication network, the changes are automatically propogated |
|
8 |
to the network, even if the network is down. |
|
3 | 9 |
|
4 |
Under construction! |
|
10 |
Replication allows users to manage their data locally and (by replicating them |
|
11 |
to a shared Metacat repository) to make those data available to the greater |
|
12 |
scientific community via a centralized search. In other words, your Metacat can |
|
13 |
be part of a broader network, but you retain control over the local repository |
|
14 |
and how it is managed. |
|
5 | 15 |
|
6 |
Heading 1 |
|
7 |
------------ |
|
16 |
For example, the KNB Network (Figure 6.1), which currently consists of ten |
|
17 |
different Metacat servers from around the world, uses replication to "join" |
|
18 |
the disperate servers to form a single robust and searchable data |
|
19 |
repository--facilitating data discovery, while leaving the data ownership and |
|
20 |
management with the local administrators. |
|
8 | 21 |
|
9 |
Heading 2 |
|
10 |
------------ |
|
22 |
.. figure:: images/screenshots/image059.jpg |
|
23 |
:align: center |
|
24 |
|
|
25 |
A map of the KNB Metacat network. |
|
11 | 26 |
|
27 |
When properly configured, Metacat's replication mechanism can be triggered by |
|
28 |
several types of events that occur on either the home or partner server: a |
|
29 |
document insertion, an update, or an automatic replication (i.e., Delta-T |
|
30 |
monitoring), which is set at a user-specified time interval. |
|
31 |
|
|
32 |
+----------------------+----------------------------------------------------------+ |
|
33 |
| Replication Triggers | Description | |
|
34 |
+======================+==========================================================+ |
|
35 |
| Insert | Whenever a document is inserted into Metacat, the server | |
|
36 |
| | notifies each server in its replication list | |
|
37 |
| | that it has a new file available. | |
|
38 |
+----------------------+----------------------------------------------------------+ |
|
39 |
| Update | Whenever a document is updated, the server notifies | |
|
40 |
| | each server in its replication list of the update. | |
|
41 |
+----------------------+----------------------------------------------------------+ |
|
42 |
| Delta-T monitoring | At a user-specified time interval, Metacat checks each | |
|
43 |
| | of the servers in its replication list | |
|
44 |
| | for updated documents. | |
|
45 |
+----------------------+----------------------------------------------------------+ |
|
46 |
|
|
47 |
Configuring Replication |
|
48 |
----------------------- |
|
49 |
To configure replication, you must configure both the home and partner servers: |
|
50 |
|
|
51 |
1. Create a list of partner servers on your home server using the Replication Control Panel |
|
52 |
2. Create certificate files for the home server |
|
53 |
3. Create certificate files for the partner server |
|
54 |
4. Import partner certificate files to the home server |
|
55 |
5. Import home certificate to the partner server |
|
56 |
6. Update your Metacat database |
|
57 |
|
|
58 |
Each step is discussed in more detail in the following sections. |
|
59 |
|
|
60 |
Using the Replication Control Panel |
|
61 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
|
62 |
To add, remove, or alter servers on your home server's Replication list, or to |
|
63 |
activate and customize the Delta-T handler, use the Replication control panel, |
|
64 |
which is accessed at the following URL:: |
|
65 |
|
|
66 |
http://somehost.somelocation.edu/context/style/skins/dev/replControl.html |
|
67 |
|
|
68 |
"http://somehost.somelocation.edu/context" should be replaced with the name |
|
69 |
of your Metacat server and context (e.g., http://knb.ecoinformatics.org/knb/). |
|
70 |
You must be logged in to Metacat as an administrator. |
|
71 |
|
|
72 |
.. figure:: images/screenshots/image061.jpg |
|
73 |
:align: center |
|
74 |
|
|
75 |
Replication control panel. |
|
76 |
|
|
77 |
Note that currently, you cannot use the Replication Control Panel to remove a |
|
78 |
server after a replication has occurred. At this point in time, the only way to |
|
79 |
remove a replication server after replication has occurred is to remove the |
|
80 |
certificates. |
|
81 |
|
|
82 |
Also note that you must SCP partner certificates to your machine; you cannot |
|
83 |
use the "Download Certificate from" option on the Control Panel. For more |
|
84 |
information about creating and installing certificates, please see Generating |
|
85 |
and Exchanging Security Certificates. |
|
86 |
|
|
87 |
Generating and Exchanging Security Certificates |
|
88 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
|
89 |
Before you can take advantage of Metacat's replication feature, you must |
|
90 |
generate security certificates on both the replication partner and home servers. |
|
91 |
The certificates will be exchanged so that each machine understands that the |
|
92 |
other has replication access. |
|
93 |
|
|
94 |
The process for generating certificates is different for Metacat servlets |
|
95 |
running under Tomcat and those under Tomcat/Apache (the recommended configuration). |
|
96 |
For instructions on generating and exchanging certificates on systems running |
|
97 |
only Tomcat (and Java 6), see Generating a Certificate for Tomcat standalone |
|
98 |
(no Apache). |
|
99 |
|
|
100 |
Generate Certificates for Metacat running under Apache/Tomcat |
|
101 |
............................................................. |
|
102 |
Note: Instructions are for Ubuntu/Debian systems. |
|
103 |
|
|
104 |
1. Generate a certificate key using openssl. The key will be named |
|
105 |
``<hostname>-apache.key``, where ``<hostname>`` is the name of your Metacat |
|
106 |
server. Example values for the individual key fields are included in the |
|
107 |
table below. |
|
108 |
|
|
109 |
:: |
|
110 |
|
|
111 |
openssl req -new -out REQ.pem -keyout <hostname>-apache.key |
|
112 |
|
|
113 |
+--------------------------+-------------------------------------------------------------------------+ |
|
114 |
| Key Field | Description and Example Value | |
|
115 |
+==========================+=========================================================================+ |
|
116 |
| Country Name | Two letter country code (e.g., US) | |
|
117 |
+--------------------------+-------------------------------------------------------------------------+ |
|
118 |
| State or Province Name | The name of your state or province spelled in full (e.g., California) | |
|
119 |
+--------------------------+-------------------------------------------------------------------------+ |
|
120 |
| Locality Name | The name of your city (e.g., Santa Barbara) | |
|
121 |
+--------------------------+-------------------------------------------------------------------------+ |
|
122 |
| Organization Name | The company or organization name (e.g., UCSB) | |
|
123 |
+--------------------------+-------------------------------------------------------------------------+ |
|
124 |
| Organizational Unit Name | The department or section name (e.g., NCEAS) | |
|
125 |
+--------------------------+-------------------------------------------------------------------------+ |
|
126 |
| Common Name | The host server name without port numbers (e.g., myserver.mydomain.edu) | |
|
127 |
+--------------------------+-------------------------------------------------------------------------+ |
|
128 |
| Email Address | Administrator's contact email (e.g., administrator@mydomain.edu) | |
|
129 |
+--------------------------+-------------------------------------------------------------------------+ |
|
130 |
| A challenge password | --leave this field blank-- | |
|
131 |
+--------------------------+-------------------------------------------------------------------------+ |
|
132 |
| An optional company name | --leave this field blank-- | |
|
133 |
+--------------------------+-------------------------------------------------------------------------+ |
|
134 |
|
|
135 |
2. Create the local certificate file by running the command: |
|
136 |
|
|
137 |
:: |
|
138 |
|
|
139 |
openssl req -x509 -days 800 -in REQ.pem -key <hostname>-apache.key -out <hostname>-apache.crt |
|
140 |
|
|
141 |
Use the same ``<hostname>`` you used when you generated the key. A file named |
|
142 |
``<hostname>-apache.crt`` will be created in the directory from which you |
|
143 |
ran the openssl command. Note: You can name the certificate file anything |
|
144 |
you'd like, but keep in mind that the file will be sent to the partner |
|
145 |
machine used for replication. The certificate name should have enough |
|
146 |
meaning that someone who sees it on that machine can figure out where it |
|
147 |
came from. |
|
148 |
|
|
149 |
3. Enter the certificate into Apache's security configuration. You must |
|
150 |
register the certificate in the local Apache instance. Note that the |
|
151 |
security files may be in a different directory from the one used in the |
|
152 |
instructions depending on how you installed Apache. Copy the certificate and |
|
153 |
key file using the following commands: |
|
154 |
|
|
155 |
:: |
|
156 |
|
|
157 |
sudo cp <hostname>-apache.crt /etc/ssl/certs |
|
158 |
sudo cp <hostname>-apache.key /etc/ssl/private |
|
159 |
|
|
160 |
4. Apache needs to know about Metacat SSL. The helper file named "knb-ssl" has |
|
161 |
rules that tell Apache which traffic to route to the Metacat SSL port. Set up |
|
162 |
SSL by dropping the knb-ssl file into the sites-available directory and |
|
163 |
running ``a2ensite`` to enable the site: |
|
164 |
|
|
165 |
:: |
|
166 |
|
|
167 |
sudo cp <metacat_helper_dir>/knb-ssl <apache_install_dir>/sites-available |
|
168 |
sudo a2ensite knb-ssl |
|
169 |
|
|
170 |
5. Restart Apache to bring in changes by typing: |
|
171 |
|
|
172 |
:: |
|
173 |
|
|
174 |
sudo /etc/init.d/apache2 restart |
|
175 |
|
|
176 |
6. SCP ``<hostname>-apache.crt`` to the replication partner machine. |
|
177 |
|
|
178 |
Generating a Certificate for Tomcat standalone (no Apache) |
|
179 |
.......................................................... |
|
180 |
If you are running Metacat under Tomcat (no Apache), generate keys in the Java |
|
181 |
default key store. The generated key is placed into the binary certificate's |
|
182 |
file located at ``/etc/java-1.5.0-sun/security/cacerts``. |
|
183 |
|
|
184 |
1. Generate the key by running the following command (note that you must be |
|
185 |
logged in as the root user to use the keytool): |
|
186 |
|
|
187 |
:: |
|
188 |
|
|
189 |
keytool -genkey -alias <aliasname> -keyalg RSA -validity 800 -keystore /etc/java-1.6.0-sun/security/cacerts |
|
190 |
|
|
191 |
``<aliasname>`` is a unique name that you choose for this key. Something |
|
192 |
like "<hostname-tomcat>" might be appropriate, where ``<hostname-tomcat>`` |
|
193 |
is the name of the Metacat host. |
|
194 |
|
|
195 |
2. The Password-keytool will ask for a password. If writing to a pre-existing |
|
196 |
keystore, you must know the password. If you are creating a new keystore, |
|
197 |
the password you enter will become the keystore password. |
|
198 |
|
|
199 |
Sample values when creating certificate: |
|
200 |
|
|
201 |
:: |
|
202 |
|
|
203 |
What is your first and last name? myserver.nceas.ucsb.edu (note: use the host name without port number) |
|
204 |
What is the name of your organizional unit? NCEAS |
|
205 |
What is the name of your organizional unit? UCSB |
|
206 |
What is the name of your City or Locality? Santa Barbara |
|
207 |
What is the name of your State or Province? California (note: this is spelled in full) |
|
208 |
What is the two-letter country code for this unit? US |
|
209 |
|
|
210 |
3. Create a certificate by running the command: |
|
211 |
|
|
212 |
:: |
|
213 |
|
|
214 |
keytool -export -alias <aliasname> -file <outputfile>.cert -keystore /etc/java-1.6.0-sun/security/cacerts |
|
215 |
|
|
216 |
``<aliasname>`` is the same name you used when you created the key file. A |
|
217 |
file named ``<outputfile>.cert`` will be created in the directory from which |
|
218 |
you ran the keytool command. You can name the output file anything you like, |
|
219 |
but keep in mind that it will be sent to the partner machine used for |
|
220 |
replication. The filename should have enough meaning that someone who sees |
|
221 |
it on that machine can figure out where it came from. Again, something like |
|
222 |
"<hostname>-tomcat.cert" will suffice. |
|
223 |
|
|
224 |
4. Edit the Tomcat server file at ``$TOMCAT_HOME/conf/server.xml`` to enable |
|
225 |
SSL in Tomcat. |
|
226 |
|
|
227 |
* Uncomment the section that starts with "<Connector port="8443" ... |
|
228 |
(Note: Databased Information comments start with <!-- and end with -->). |
|
229 |
|
|
230 |
* Add two attribute to that section: |
|
231 |
|
|
232 |
:: |
|
233 |
|
|
234 |
keystoreFile="/etc/java-1.6.0-sun/security/cacerts" |
|
235 |
keystorePass="<keystore_password>" |
|
236 |
|
|
237 |
where ``<keystore_password>`` is the password you used when you created |
|
238 |
or accessed the keystore. |
|
239 |
|
|
240 |
5. SCP the certificate to the partner server. |
|
241 |
|
|
242 |
To import a certificate |
|
243 |
....................... |
|
244 |
1. Log in as a root user (the keytool must run as a root user) |
|
245 |
|
|
246 |
:: |
|
247 |
|
|
248 |
sudo su – |
|
249 |
|
|
250 |
2. Import the remote certificate by running: |
|
251 |
|
|
252 |
:: |
|
253 |
|
|
254 |
keytool -import -alias <remotehostalias> -file <remotehostfilename>.crt -keystore /etc/java-1.6.0-sun/security/cacerts |
|
255 |
|
|
256 |
where the ``<remotehostfilename>`` is the name of the certificate file |
|
257 |
created on the remote partner machine and SCP'd to the home machine. |
|
258 |
The ``<remotehostalias>`` is the name the certificate will use in the |
|
259 |
keystore. The name should identify the remote host. |
|
260 |
|
|
261 |
Update your Metacat database |
|
262 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
|
263 |
The simplest way to update the Metacat database to use replication is to use |
|
264 |
the Replication Control Panel. You can also update the database u |
|
265 |
sing SQL. Instructions for both options are included in this section. |
|
266 |
|
|
267 |
.. figure:: images/screenshots/image063.jpg |
|
268 |
:align: center |
|
269 |
|
|
270 |
Using the Replication Control Panel to update the Metacat database. |
|
271 |
|
|
272 |
To update your Metacat database to use replication, select the "Add this server" |
|
273 |
radio button from the Replication Control Panel, enter the partner server name, |
|
274 |
and specify how the replication should occur (whether to replicate xml, data, |
|
275 |
or use the local machine as a hub). Note that you cannot download certificates |
|
276 |
using this interface. |
|
277 |
|
|
278 |
To update the database using SQL |
|
279 |
................................ |
|
280 |
|
|
281 |
1. Log in to the database |
|
282 |
|
|
283 |
:: |
|
284 |
|
|
285 |
psql -U metacat -W -h localhost metacat |
|
286 |
|
|
287 |
2. Select all rows from the replication table |
|
288 |
|
|
289 |
:: |
|
290 |
|
|
291 |
select * from xml_replication; |
|
292 |
|
|
293 |
3. Insert the partner server. |
|
294 |
|
|
295 |
:: |
|
296 |
|
|
297 |
INSERT INTO xml_replication (server,last_checked,replicate,datareplicate,hub) VALUES ('<partner.server/context>/servlet/replication',NULL,1,1,0); |
|
298 |
|
|
299 |
Where ``<partner.server/context>`` is the name of the partner server and |
|
300 |
context. The values 'NULL, 1,1,0' indicate (respectively) the last time |
|
301 |
replication occurred, that XML docs should be replicated to the partner |
|
302 |
server, that data files should be replicated to the partner server, and |
|
303 |
that the local server should not act as a hub. Set a value of 'NULL,0,0,0' |
|
304 |
if your Metacat is only receiving documents from the partner site and not |
|
305 |
replicating to that site. |
|
306 |
|
|
307 |
4. Exit the database |
|
308 |
5. Restart Apache and Tomcat on both home and partner replication machines |
Also available in: Unified diff
Fixed table layout on geoserver and submission chapters. Converted Replication
chapter to RST.