Bug #4616
closedTimed Replication takes many hours and drives the load up on KNB
0%
Description
Timed replication can more than 12 hours lter and almost that long for PISCO.
During that time, the load on the machine can surpass 10 and java can use over 200% cpu.
lter has over 80K records and pisco over 60K. Only a handful of those actually need to be replicated. We need to see why it's taking so long and using so many resources.
Updated by Michael Daigle over 14 years ago
After raising the log level on KNB to INFO and tracing through the logs, it was clear that the delay was happening when converting the document StringReader back to a string so the file could be written to disk.
Note, the reason this was done in the first place was that the StringReader for the doc was closed during the xml parsing. The write to disk should only take place after the parse is successfully completed. So the doc was extracted to a string and then written to disk after parse.
Since the doc is currently read into a string when it is pulled from the request object, I changed the code to send that string to DocumentImpl.write and .writeReplication. In that way, a StringReader can be created for the parsing, and another for the write to disk.