Metacat: Issueshttps://projects.ecoinformatics.org/ecoinfo/https://projects.ecoinformatics.org/ecoinfo/ecoinfo/favicon.ico?14691340362017-11-17T21:19:00ZEcoinformatics Redmine
Redmine Bug #7229 (New): Mis-Formatting of Data Package Contentshttps://projects.ecoinformatics.org/ecoinfo/issues/72292017-11-17T21:19:00ZThomas Thelen
<p>In MetacatUI we're getting a slight mis-formatting when displaying data package contents. This can be seen in the attached images. The issue was initially reported in MetacatUI as issue 379.</p>
<p><a class="external" href="https://github.com/NCEAS/metacatui/issues/379">https://github.com/NCEAS/metacatui/issues/379</a></p>
<p>From Bryce,</p>
<p>"The HTML in question is actually produced by Metacat and MetcatUI is just rendering it without modification from Metacat's View Service. ... The fix would involve changing the underlying eml-2 XSLT."</p> Bug #7228 (New): Error in sorting data-sets based on title in MetaCatUIhttps://projects.ecoinformatics.org/ecoinfo/issues/72282017-11-15T17:35:09ZRushiraj Nenuji
<p>Reference: Issue <a class="external" href="https://github.com/NCEAS/metacatui/issues/350">https://github.com/NCEAS/metacatui/issues/350</a> MetaCatUI<br /><br />The datasets when sorted based on the title(a-z) results into sorting based on the upper case and lower case separately.</p>
<p>Please find attached image for results of sorting along with this email.</p>
<p>Possible Solution: <a class="external" href="https://stackoverflow.com/questions/2053214/how-to-create-a-case-insensitive-copy-of-a-string-field-in-solr">https://stackoverflow.com/questions/2053214/how-to-create-a-case-insensitive-copy-of-a-string-field-in-solr</a></p>
<p>Thanks!</p>
<p>Best,<br />Rushiraj Nenuji.</p> Bug #7226 (New): Perl Registry hangs on uploadshttps://projects.ecoinformatics.org/ecoinfo/issues/72262017-11-10T20:56:18ZChris Jonescjones@nceas.ucsb.edu
<p>Jeanette has reported an issue where the KNB data registry is hanging when a small file is being uploaded (see attached screenshots).</p>
<p>In the JavaScript console, she saw a <code>Bad Gateway</code> error. With a quick web search on this, it looks like the <code>Bad Gateway</code> response is usually thrown when Apache calls a sub processor like the perl interpreter and doesn't get a response back (timeout), or some other non-response error. We need to track down what is happening in the Perl code to induce this. Here's the Slack thread:<br /><pre>
Feeling the "hanging submission page" pain right now. What do I do? Abandon? Wait and hope?
According to my local file these files should only be 831KB so I'm not sure why it thinks they are so huge
chris [1:15 PM]
@jeanette which server is that?
KNB?
jeanette [1:16 PM]
yeah, production
chris [1:16 PM]
ok
jeanette [1:16 PM]
I bailed out after seeing the timeout error
chris [1:16 PM]
that’s an odd one. i’ll look
jeanette [1:16 PM]
but still not the easiest thing to figure out from a user perspective
chris [1:16 PM]
totally
jeanette [1:17 PM]
I think this has happened to me before, and I think it is triggered by
writing metadata ->
add a file ->
accidentally submit before adding all files ->
go back and edit record ->
add more files ->
submit again ->
page hangs forever
I also think I have heard of a similar workflow causing an error for ADC users
</pre></p> Bug #7224 (New): Document the procedure of registering a new schema in Metacathttps://projects.ecoinformatics.org/ecoinfo/issues/72242017-11-07T20:31:45ZJing Taotao@nceas.ucsb.edu
<p>Metacat will NOT automatically download and register a new schema since 2.8 release. Before we have an admin page to help administrator to register a new schema, at least we need to provide a document to guid the administrator to do it manually.</p> Bug #7217 (New): Report on metadata creation date in metadata quality summarieshttps://projects.ecoinformatics.org/ecoinfo/issues/72172017-10-11T18:42:38ZPeter Slaughterslaughter@nceas.ucsb.edu
<p>Indexing fields for metadata quality reports do not include the upload date of the metadata they are reporting on. Therefor, summaries that are created, i.e. mean score for a user over time, currently show the time of the creation of the quality report, not the metadata.</p>
<p>Add the field 'mdq.metadata.timestamp' to application-context-mdq.xml to hold the metadata creation or update time.</p>
<p>Each quality suite will be responsible for making this information available in the quality report, so that MDQClient.saveRun<br />can record it.</p> Bug #7216 (New): MDQClient.saveRun doesn't obsolete existing quality documentshttps://projects.ecoinformatics.org/ecoinfo/issues/72162017-10-11T17:30:14ZPeter Slaughterslaughter@nceas.ucsb.edu
<p>MDQClient.saveRun is called to upload a newly created quality document, in response to a metadata quality document being uploaded or updated.</p>
<p>When MDQClient.saveRun is called by MNodeService.update, it does not check if a quality document has already been created<br />for the metadata document. saveRun should check if a previous quality document has been created for the metadata, and obsolete it<br />with the new quality document. This will ensure that quality statistics are accurate, as obsoleted quality reports will not be<br />included in statistical calculations, as they are essentially duplicates.</p> Bug #7212 (New): metacat-index missing metadata quality fieldshttps://projects.ecoinformatics.org/ecoinfo/issues/72122017-10-03T18:59:38ZPeter Slaughterslaughter@nceas.ucsb.edu
<p>The Spring context file ./metacat-index/src/main/resources/application-context-mdq.xml doesn't contain a bean definition for the quality check types 'congruency' or 'dataFormats',<br />although these are check types that we should record results for. There is a bean for check type 'other', but this isn't sufficient.</p> Bug #7208 (Rejected): Metacat is double-decoding incoming urls on the CNshttps://projects.ecoinformatics.org/ecoinfo/issues/72082017-09-06T21:33:25ZJing Taotao@nceas.ucsb.edu
<p>Details see <br /><a class="external" href="https://redmine.dataone.org/issues/8122">https://redmine.dataone.org/issues/8122</a></p> Task #7204 (New): Use the ldaps protocal to connect the ldap serverhttps://projects.ecoinformatics.org/ecoinfo/issues/72042017-07-31T23:41:44ZJing Taotao@nceas.ucsb.edu
<p>Current we use the StartTlsRequest to secure the connection between the Metacat and ldap server. The url for this connection is ldap://ldap.ecoinformatics.org:389/. Nick is thinking to close the port 389 and leave port 636 open, which is used for the ldaps protocol. So we may need to change the url to ldaps and some code as well. We also need to take a look at the ldapweb code as well.</p> Bug #7203 (In Progress): Improve D1NodeService.isAuthorized() performancehttps://projects.ecoinformatics.org/ecoinfo/issues/72032017-07-22T23:14:48ZChris Jonescjones@nceas.ucsb.edu
<p>We're seeing poor performance in calls to <code>D1NodeService.isAuthorized()</code> on <a class="external" href="https://arcticdata.io">https://arcticdata.io</a>. When the system Metacat is under light load (< 10 requests per second), calls to <code>isAuthorized()</code> are taking up to 35 seconds to return either an <code>HTTP 200</code> response or a <code>HTTP 403</code> exception.</p>
<p>Change <code>isAuthorized()</code> to prioritize user-based authorization first, and then CN or MN authorization last. This should increase performance for end users, whereas MN to MN replication calls and CN-administrative calls will be slightly less prioritized.</p>
<p>Note that calls to <code>userHasPermission()</code> involve token verification using the <code>PortalCertificateManager</code> and the <code>TokenGenerator</code>. These calls may be repeatedly making a call to the CN to get the SSL certificate for verification if it is not cached. If this change doesn't significantly improve performance, look into refactoring those classes in <code>d1_portal</code> to cache and use the certificate, unless there is a verification exception, in which case we make the call to <code>fetchCertificate()</code> again, re-cache it, and attempt to re-verify the token. If it still fails, throw <code>NotAuthorized</code>.</p> Task #7202 (New): Add documentation to indicate and help users Metacat only runs under Java 1.8https://projects.ecoinformatics.org/ecoinfo/issues/72022017-06-28T00:38:49ZJing Taotao@nceas.ucsb.edu
<p>We decided that this 2.9.0 release will only be running under Java 1.8. We need to make clear to the user in our documentation. We also need to help user to install open jdk in Ubuntu 14.</p> Bug #7199 (New): Upgrade postgresql jdbc jar file on Metacathttps://projects.ecoinformatics.org/ecoinfo/issues/71992017-06-02T21:03:49ZJing Taotao@nceas.ucsb.edu
<p>Please see detail on this ticket:<br /><a class="external" href="https://redmine.dataone.org/issues/8104">https://redmine.dataone.org/issues/8104</a></p> Task #7177 (New): Ensure Metacat runs on Tomcat 8https://projects.ecoinformatics.org/ecoinfo/issues/71772017-03-24T19:23:06Zben leinfelderleinfelder@nceas.ucsb.edu
<p>I was working with <a class="email" href="mailto:Eva-Maria.Gerstner@senckenberg.de">Eva-Maria.Gerstner@senckenberg.de</a> on an installation of Metacat 2.8.1 and we could not get Metacat to fully initialize after configuring it in the admin screens. On restart, we would get as far as the</p>
<pre><code class="text syntaxhl">metacat 20170324-18:24:19: [DEBUG]: XMLService.doRefresh - refreshing the schema service. [edu.ucsb.nceas.metacat.service.XMLSchemaService]
</code></pre>
<p>and then it would never get beyond that because of an unexpected exception in that routine:</p>
<pre><code class="text syntaxhl">24-Mar-2017 18:15:38.600 SCHWERWIEGEND [localhost-startStop-1] org.apache.catalina.core.StandardContext.loadOnStartup Servlet [metacat] in web application [/metacat] threw load() exception
java.lang.NullPointerException
at edu.ucsb.nceas.metacat.service.XMLSchemaService.populateRegisteredSchemaList(XMLSchemaService.java:331)
at edu.ucsb.nceas.metacat.service.XMLSchemaService.doRefresh(XMLSchemaService.java:129)
at edu.ucsb.nceas.metacat.service.XMLSchemaService.<init>(XMLSchemaService.java:101)
at edu.ucsb.nceas.metacat.service.XMLSchemaService.getInstance(XMLSchemaService.java:114)
at edu.ucsb.nceas.metacat.MetaCatServlet.initSecondHalf(MetaCatServlet.java:350)
at edu.ucsb.nceas.metacat.MetaCatServlet.init(MetaCatServlet.java:315)
at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1183)
at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1099)
at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:989)
at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:4913)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5223)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:752)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:728)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:734)
at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:952)
at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1823)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
</code></pre> Feature #7171 (Closed): Run MDQ suite on insert/updatehttps://projects.ecoinformatics.org/ecoinfo/issues/71712017-01-05T00:40:26Zben leinfelderleinfelder@nceas.ucsb.edu
<p>We are looking to have Metacat handle quality reports by submitting new and updated metadata records to the MDQ engine and then storing the results (which will be indexed by existing Metacat index code).</p> Bug #7170 (New): Ignore the access part change in the EML sax parser classes when users use datao...https://projects.ecoinformatics.org/ecoinfo/issues/71702016-12-21T01:04:51ZJing Taotao@nceas.ucsb.edu
<p>In old Metacat api, an eml document contains the access control rules of both itself and the data files which are referenced. So our EML sax parser has the mechanism to check if the access control part of data files in the EML has changed when a user is trying to update an eml document. The reason is to avoid a scenario that a user, who only has the ALL permission on the eml but doesn't have the ALL permission to the data files, can update the access rule of the data file. I believe those rules are still needed when a client to use the old Metacat api to update a document.<br />However, to the dataone api, the access control rules are specified by the system metadata and the access rules on the EML object will be ignored. So we don't need the check if the EML sax parser is called by the dataone api.</p>