Metacat: Issueshttps://projects.ecoinformatics.org/ecoinfo/https://projects.ecoinformatics.org/ecoinfo/ecoinfo/favicon.ico?14691340362017-10-06T19:27:14ZEcoinformatics Redmine
Redmine Bug #7215 (New): Metacat produces an invalid ZIP archive when a package member has an invalid for...https://projects.ecoinformatics.org/ecoinfo/issues/72152017-10-06T19:27:14ZBryce Mecummecum@nceas.ucsb.edu
<p>I submitted a new Data Package and went to download it via the Download All button In MetacatUI which triggers the /packages route in Metacat. I then tried to unzip it and couldn't get any of my zip extraction tools to do it. Then I hex dumped it:</p>
<pre><code class="text syntaxhl">bryce@mbp ~/Downloads> hexdump -C resource_map_urn-uuid-13c3000d-09b1-453b-86d7-e852d147fb81.rdf7b2b468205c3.zip
00000000 3c 3f 78 6d 6c 20 76 65 72 73 69 6f 6e 3d 22 31 |<?xml version="1|
00000010 2e 30 22 20 65 6e 63 6f 64 69 6e 67 3d 22 55 54 |.0" encoding="UT|
00000020 46 2d 38 22 3f 3e 3c 65 72 72 6f 72 20 64 65 74 |F-8"?><error det|
00000030 61 69 6c 43 6f 64 65 3d 22 30 30 30 30 22 20 65 |ailCode="0000" e|
00000040 72 72 6f 72 43 6f 64 65 3d 22 34 30 34 22 20 6e |rrorCode="404" n|
00000050 61 6d 65 3d 22 4e 6f 74 46 6f 75 6e 64 22 3e 0a |ame="NotFound">.|
00000060 20 20 20 20 3c 64 65 73 63 72 69 70 74 69 6f 6e | <description|
00000070 3e 54 68 65 20 66 6f 72 6d 61 74 20 73 70 65 63 |>The format spec|
00000080 69 66 69 65 64 20 62 79 20 4e 41 20 77 61 73 20 |ified by NA was |
00000090 6e 6f 74 20 66 6f 75 6e 64 20 61 66 74 65 72 20 |not found after |
000000a0 72 65 66 72 65 73 68 69 6e 67 20 74 68 65 20 63 |refreshing the c|
000000b0 61 63 68 65 2e 3c 2f 64 65 73 63 72 69 70 74 69 |ache.</descripti|
000000c0 6f 6e 3e 0a 3c 2f 65 72 72 6f 72 3e 0a |on>.</error>.|
000000cd
</code></pre>
<p>Which looks like Metacat wrote out an XML file and called it a .ZIP. After seeing this particular error message, I realized that Metacat was choking on the "NA" formatId on the Data Object I put in this package. Fair enough I guess.</p>
<p>I think Metacat did two things that were surprising to me:</p>
<p>1. Didn't produce what I asked for even though it reasonably could have (Metacat doesn't need to know the formatID to send me the file)<br />2. Didn't produce any error message (e.g., a non-200 HTTP status) and instead sent me an invalid ZIP file</p> Bug #7022 (New): Fatal processing error when updating an object with incorrect sysmetahttps://projects.ecoinformatics.org/ecoinfo/issues/70222016-05-06T20:02:36ZBryce Mecummecum@nceas.ucsb.edu
<p>Jessica used the R package today to update an object with new bytes via the D1 REST API call for MNStorage.update(). She received this error back from Tomcat:</p>
<pre>
<?xml version="1.0"?>
<error>Fatal processing error.</error>
</pre>
<p>She showed me what she had for her input to the call and I found that the sysmeta formatId was EML 2.1.1 instead of text/csv which was the correct format ID for the object. This was a bug in my code that she was running that I've since addressed.</p>
<p>I expected a more useful error that directly addressed the mismatch in format ID and the file being uploaded.</p>
<p>Why did this action cause a "Fatal processing error"? It seems like the sysmeta format ID being set to EML should've triggered the EML validation routine which should have returned a validation error.</p> Task #6995 (New): Error message doesn't provide cause of errorhttps://projects.ecoinformatics.org/ecoinfo/issues/69952016-03-23T21:43:52ZPeter Slaughterslaughter@nceas.ucsb.edu
<p>Several services that require authentication return error descriptions that don't indicate the cause of the error, for example for <br />MNStorage.generateIdentifier(), if the token is expired or corrupted, etc:</p>
<p><?xml version="1.0" encoding="UTF-8"?><error detailCode="2190" errorCode="401" name="InvalidToken"><br /> <description>Session is required to generate an Identifier at this Node.</description><br /></error></p>
<p>The description shown in the D1 API docs does provide enough info for the user to troubleshoot the problem:</p>
<p>"The supplied authentication token is not a proper certificate, or missing required fields, or otherwise proves invalid."</p>
<p>It would be helpful if this description is used instead of the "Seesion is required" message.</p>
<p>Other services return similar "Session..." messages, such as MNStorage.create(), MNStorage.update()</p> Bug #6994 (New): Bad call to MNStorage.update() via REST API can result in bad state and StackOve...https://projects.ecoinformatics.org/ecoinfo/issues/69942016-03-23T20:27:30ZBryce Mecummecum@nceas.ucsb.edu
<p>This all happened on arcticdata.io production over the last couple of days.</p>
<p>I was attempting to update an object and forgot the {PID} part of the REST API URL: PUT /object/{pid}. This resulted in unexpected behavior and an unexpected state.</p>
<p>- The request returned a ServiceError (HTTP Status 500) of "StackOverflowError", this was unexpected.<br />- The sysmeta for the PID I was updating changed: The PID became obsoleted and obseletedBy the new PID I chose. This was expected.<br />- Calls to /meta and /object for the new PID failed, this was unexpected.</p>
<p>It appears that the new PID was reserved but never assigned sysmeta or object bytes, resulting in an unexpected system state.</p>
<p>I then set about a path of archiving the PID by first removing public read access, which resulted in another StackOverflowError but public read access was revoked as expected. In the end, I had Chris Jones do an administrative delete on the object.</p>
<p>I see two things here:</p>
<p>1. The requests returned StackOverflowErrors. It seems like a stack overflow shouldn't be possible. The requests returning this error took ~10+ seconds to return which would imply this is a great attack vector.<br />2. An invalid REST API call was not rejected immediately (the call where I was missing the {PID} part of the URL</p> Bug #6952 (New): error trying updating metadata on https://knb.ecoinformatics.org/#view/knb.894.1https://projects.ecoinformatics.org/ecoinfo/issues/69522016-01-28T04:48:05ZJulien Brunbrun@nceas.ucsb.edu
<p>Hi guys,</p>
<p>When we are trying to edit this package metadata <a class="external" href="https://knb.ecoinformatics.org/#view/knb.894.1">https://knb.ecoinformatics.org/#view/knb.894.1</a> using the KNB website (uid=pellis_knb), we are getting this error:</p>
<p>_Whoops!<br />An error occurred. Please check the list of errors below:</p>
<p>cvc-pattern-valid: Value '' is not facet-valid with respect to pattern '[\s]*[\S][\s\S]*' for type 'NonEmptyStringType'.<br />Failed while updating._</p>
<p>Thank you for looking into this!<br />Julien</p> Feature #6951 (New): Develop XSLT for ISO-19119 for Member Node Serviceshttps://projects.ecoinformatics.org/ecoinfo/issues/69512016-01-20T22:46:13ZLauren Walkerwalker@nceas.ucsb.edu
<p>There are details in the ISO-19119 XSLT that may be useful to expose to users through the D1 view service / MetacatUI Metadata View.</p> Bug #6516 (New): Solr index doesn't work in the remote host in the replication when a host is ubu...https://projects.ecoinformatics.org/ecoinfo/issues/65162014-04-15T22:56:02ZJing Taotao@nceas.ucsb.edu
<p>When I use morpho 1.10 branch to insert a package into a local Metacat, which is ubuntu 14.04, the solr index works fine. However, the remote server, which is ubuntu 12.04, got the error:<br />metacat 20140415-14:31:01: [WARN]: XMLService.populateRegisteredSchemaList - Schema file: /var/lib/tomcat7/webapps/knb/schema/fgdc-std-001/fgdc-std-001-1998.xsd is registered in the database but does not exist on the file system. [edu.ucsb.nceas.metacat.service.XMLSchemaService]<br />edu.ucsb.nceas.metacat.McdbDocNotFoundException: the requested docid 'tao.13002' does not exist<br /> at edu.ucsb.nceas.metacat.DocumentImpl.isRevisionOnly(DocumentImpl.java:2159)<br /> at edu.ucsb.nceas.metacat.DocumentImpl.getDocumentInfo(DocumentImpl.java:2190)<br /> at edu.ucsb.nceas.metacat.DocumentImpl.<init>(DocumentImpl.java:228)<br /> at edu.ucsb.nceas.metacat.DocumentImpl.<init>(DocumentImpl.java:257)<br /> at edu.ucsb.nceas.metacat.replication.ReplicationService.getDocumentInfo(ReplicationService.java:1097)<br /> at edu.ucsb.nceas.metacat.replication.ReplicationService.getDocumentInfoMap(ReplicationService.java:1073)<br /> at edu.ucsb.nceas.metacat.dataone.hazelcast.ObjectPathMap.pathToDocid(ObjectPathMap.java:76)<br /> at edu.ucsb.nceas.metacat.dataone.hazelcast.ObjectPathMap.load(ObjectPathMap.java:99)<br /> at edu.ucsb.nceas.metacat.dataone.hazelcast.ObjectPathMap.load(ObjectPathMap.java:43)<br /> at com.hazelcast.impl.concurrentmap.MapStoreWrapper.load(MapStoreWrapper.java:143)<br /> at com.hazelcast.impl.ConcurrentMapManager$GetOperationHandler$GetLoader.doMapStoreOperation(ConcurrentMapManager.java:3785)<br /> at com.hazelcast.impl.ConcurrentMapManager$AbstractMapStoreOperation.run(ConcurrentMapManager.java:3860)<br /> at com.hazelcast.impl.executor.ParallelExecutorService$ParallelExecutorImpl$ExecutionSegment.run(ParallelExecutorService.java:212)<br /> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)<br /> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)<br /> at java.lang.Thread.run(Thread.java:744)<br /> at com.hazelcast.impl.ExecutorThreadFactory$1.run(ExecutorThreadFactory.java:38)<br />metacat-index 20140415-14:31:01: [ERROR]: SolrIndex.update - could not update the solr index since null [edu.ucsb.nceas.metacat.index.SolrIndex]<br />java.lang.NullPointerException<br /> at java.io.FileInputStream.<init>(FileInputStream.java:138)<br /> at java.io.FileInputStream.<init>(FileInputStream.java:101)<br /> at edu.ucsb.nceas.metacat.index.SolrIndex.update(SolrIndex.java:513)<br /> at edu.ucsb.nceas.metacat.index.SystemMetadataEventListener.entryUpdated(SystemMetadataEventListener.java:146)<br /> at edu.ucsb.nceas.metacat.index.SystemMetadataEventListener.entryAdded(SystemMetadataEventListener.java:119)<br /> at com.hazelcast.client.impl.EntryListenerManager.notifyListeners(EntryListenerManager.java:148)<br /> at com.hazelcast.client.impl.EntryListenerManager.notifyListeners(EntryListenerManager.java:130)<br /> at com.hazelcast.client.impl.ListenerManager.customRun(ListenerManager.java:88)<br /> at com.hazelcast.client.ClientRunnable.run(ClientRunnable.java:30)<br /> at java.lang.Thread.run(Thread.java:744)</p>
<pre><code>I used morpho to insert the package to Ubuntu 12.04. The metacat with 14.04 got the error as well.</code></pre>
<p>But the timed solr index process can pick up those failed documents.</p> Feature #6362 (New): Add an id to the year element in the citation in eml-indentifier.xslhttps://projects.ecoinformatics.org/ecoinfo/issues/63622014-01-06T19:30:51ZLauren Walkerwalker@nceas.ucsb.eduFeature #6355 (New): Print "N/A" or similar when EML fields don't exist rather than print a blank...https://projects.ecoinformatics.org/ecoinfo/issues/63552014-01-03T22:35:02ZLauren Walkerwalker@nceas.ucsb.eduBug #6225 (New): Need a link for account skin to come back the home pagehttps://projects.ecoinformatics.org/ecoinfo/issues/62252013-11-15T18:20:24ZJing Taotao@nceas.ucsb.edu
<p>In the account skin, a user can't find a link to come back the main page (home) after does an action, such as create an account, reset password and et al.</p>
<p>We need to add a link on the either footer or header.</p> Bug #6185 (New): Chinese characters are sorted before alphanumeric characters https://projects.ecoinformatics.org/ecoinfo/issues/61852013-10-31T00:22:00ZLauren Walkerwalker@nceas.ucsb.edu
<p>Not sure if this is a MetacatUI or Metacat bug?</p>
<p>It may be technically correct, but it is not very user friendly to see Chinese characters first on a MetacatUI results listing, especially if those comprise of a page or more.</p> Bug #6038 (New): In the systemmetadata table on the dev2 machine, some guids have archvied=false ...https://projects.ecoinformatics.org/ecoinfo/issues/60382013-07-20T00:22:02ZJing Taotao@nceas.ucsb.edu
<p>knb=> select archived, obsoleted_by from systemmetadata where guid='nikkis.27.1';<br /> archived | obsoleted_by <br />----------+--------------<br /> f | <br />(1 row)</p>
<p>This means the guid nikkis.27.1 is not archived.</p>
<p>However,</p>
<p>knb=> select guid, obsoletes from systemmetadata where obsoletes='nikkis.27.1';<br /> guid | obsoletes <br />-------------+-------------<br /> nikkis.27.5 | nikkis.27.1<br />(1 row)</p>
<p>This means that the nikkis.27.1 is archived.</p>
<p>Those guids with the issues are:<br />peggym.36.15<br />judithk.1014.1<br />judithk.1018.2<br />judithk.1000.10<br />peggym.82.4<br />peggym.1048.6<br />nikkis.7.2<br />judithk.1034.1<br />nikkis.169.2<br />judithk.332.1<br />nikkis.27.1</p>
<p>In them n-demo-4, there are three ids with the issue:<br />doi:10.5072/FK2/LTER/knb-lter-gce.333.5<br />doi:10.5072/FK2/LTER/knb-lter-gce.331.12<br />doi:10.5072/FK2/LTER/knb-lter-gce.335.6</p>
<p>We need to figure out why this happened.</p> Task #5994 (New): Create REST API for accessing statisticshttps://projects.ecoinformatics.org/ecoinfo/issues/59942013-05-25T01:16:03ZMatt Jonesjones@nceas.ucsb.edu
<p>For objects, users, packages, nodes, etc.</p> Task #5993 (New): Summarize and index statistics for fast accesshttps://projects.ecoinformatics.org/ecoinfo/issues/59932013-05-25T01:15:33ZMatt Jonesjones@nceas.ucsb.eduTask #5992 (New): Track citationshttps://projects.ecoinformatics.org/ecoinfo/issues/59922013-05-25T01:15:07ZMatt Jonesjones@nceas.ucsb.edu
<p>Or interface with Impact Story</p>