Metacat: Issueshttps://projects.ecoinformatics.org/ecoinfo/https://projects.ecoinformatics.org/ecoinfo/ecoinfo/favicon.ico?14691340362017-05-24T23:53:24ZEcoinformatics Redmine
Redmine Bug #7194 (Resolved): MN.publish method can't generate a new ore document when the metadata is pu...https://projects.ecoinformatics.org/ecoinfo/issues/71942017-05-24T23:53:24ZJing Taotao@nceas.ucsb.edu
<p>After publish a metadata object created from the R client, the package information was missing.</p> Bug #7193 (Resolved): MN.publish fails because of the mismatched checksumhttps://projects.ecoinformatics.org/ecoinfo/issues/71932017-05-22T15:55:00ZJing Taotao@nceas.ucsb.edu
<p>The MN.publish method will modify the eml document's content, but it doesn't compute the new checksum and only uses the old one to call the mn.update method. The mn.update didn't check the checksum ((see ticket <a class="external" href="https://projects.ecoinformatics.org/ecoinfo/issues/7192">https://projects.ecoinformatics.org/ecoinfo/issues/7192</a>), so it worked. However, when we check the checksum in the mn.update method, it fails.</p> Bug #7192 (Resolved): MN.update method doesn't check the checksumhttps://projects.ecoinformatics.org/ecoinfo/issues/71922017-05-22T15:49:43ZJing Taotao@nceas.ucsb.edu
<p>The MN.update doesn't check the checksum - it doesn't compute the checksum of the coming object and compare the computed value with the one in the system metadata. We need to check as the create method does.</p> Bug #7188 (Resolved): MNodeService.replicate() is failinghttps://projects.ecoinformatics.org/ecoinfo/issues/71882017-05-11T14:19:25ZChris Jonescjones@nceas.ucsb.edu
<p>Laura Moyers reported that she is seeing many failed replication attempts in the Coordinating Node index. In particular, KNB, GOA, UIC, ARCTIC, mnUCSB1, and mnORC1 are all affected, and are all running Metacat.</p>
<p>After looking at catalina.out on the MNs, we're seeing errors in <code>MNodeService.replicate()</code>:<br /><pre>
20170508-06:59:14: [ERROR]: Error computing checksum on replica: mark/reset not supported [edu.ucsb.nceas.metacat.dataone.MNodeService]
</pre></p>
<p>Here's the number of requests and failures<br /><pre>
host requests failures failures_since
-----------------------------------------------------------
mn-orc-1 145 25 20170511-01:23:48
mn-ucsb-1 105 56 20170508-06:59:14
mn-unm-1 0 0 -
knb 71 28 20170509-16:57:53
uic no log access
</pre></p>
<p>I'm pretty sure the failures represent 100% of the requests since the failures began, but we'd need to confirm this. Basically, MN replication looks to be entirely broken in Metacat.</p>
<p>The error reported above comes from line 866 of <code>MNodeService.java</code>, where the checksum of the bytes of the object from the source MN (to be replicated) is calculated. Once the checksum is calculated, we call <code>object.reset()</code> on the input stream so it can be read again when writing to disk. This is throwing the exception above.</p>
<p>So what's changed? The last changes regarding the <code>InputStream</code> was that Jing wrapped the calls in a <code>try{ } finally { }</code> block in order to ensure the input stream gets closed after use to prevent memory leaks. This doesn't seem like an issue at all, although the <code>finally{ }</code> block could have been used in the existing <code>try { }</code> block instead of having three levels of <code>try</code> nesting. This seems inconsequential though.</p>
<p>The other change is that <code>d1_libclient_java</code> is now using the Apache Commons IO <code>AutoCloseInputStream</code>. Looking at the documentation there, it seems to delegate to the underlying input stream implementation. We know that not all input streams support the <code>mark()</code> method and therefore can't be <code>reset()</code>, which is why we call <code>markSupported()</code> before attempting to calculate the checksum. So why is <code>markSupported()</code> succeeding, but then <code>reset()</code> is failing after reading the input stream? It seems like we need to track this down between the interaction of <code>MNodeService</code> and <code>MultipartMNode.getReplica()</code>.</p> Bug #7187 (Resolved): Set file names correctly when reading objects from Metacathttps://projects.ecoinformatics.org/ecoinfo/issues/71872017-05-10T15:05:57ZChris Jonescjones@nceas.ucsb.edu
<p>We now store file names in <code>SystemMetadata.fileName</code>. In <code>MetacatHandler.readFromMetacat()</code>, we are currently generating a file name based on the <code>docid</code>, and setting it into the <code>Content-Disposition</code> header. Change <code>generateOutputName()</code> to use <code>SystemMetadata.fileName</code> when it's available, and fall back to generating a file name.</p>