Metacat: Issueshttps://projects.ecoinformatics.org/ecoinfo/https://projects.ecoinformatics.org/ecoinfo/ecoinfo/favicon.ico?14691340362017-12-14T23:15:43ZEcoinformatics Redmine
Redmine Task #7233 (Resolved): D1ResourceHandler.serializeException should not log all exceptions returne...https://projects.ecoinformatics.org/ecoinfo/issues/72332017-12-14T23:15:43ZJing Taotao@nceas.ucsb.edu
<p>Details see <a class="external" href="https://redmine.dataone.org/issues/8111">https://redmine.dataone.org/issues/8111</a></p> Bug #7232 (Resolved): Invalidly formatted dates in listObjects fromDate / toDate parameters do no...https://projects.ecoinformatics.org/ecoinfo/issues/72322017-12-08T22:15:11ZJing Taotao@nceas.ucsb.edu
<p>Details see:<br /><a class="external" href="https://redmine.dataone.org/issues/8130">https://redmine.dataone.org/issues/8130</a></p> Bug #7231 (Resolved): Support the formatID : http://www.isotc211.org/2005/gmd-pangaeahttps://projects.ecoinformatics.org/ecoinfo/issues/72312017-12-08T00:27:12ZJing Taotao@nceas.ucsb.edu
<p>The schema of the format id has been added and the db script to modify the xml_catalog table has been added also.<br />In metacat-index, the subporcessor for this format id has been added.</p>
<p>The change was committed to both trunk and 2.8 branch.</p> Feature #7230 (Resolved): Metacat index error message doesn't have the pid informationhttps://projects.ecoinformatics.org/ecoinfo/issues/72302017-11-18T02:21:19ZJing Taotao@nceas.ucsb.edu
<p>This will cause the difficulty to debug the metacat-index issue.</p> Task #7227 (Resolved): metacat.EventLog.getReport should downgrade a log levelhttps://projects.ecoinformatics.org/ecoinfo/issues/72272017-11-14T00:24:20ZJing Taotao@nceas.ucsb.edu
<p>Details should be found at:<br /><a class="external" href="https://redmine.dataone.org/issues/8110">https://redmine.dataone.org/issues/8110</a></p> Bug #7225 (Resolved): /object/id command get a null exception if there the object format is not f...https://projects.ecoinformatics.org/ecoinfo/issues/72252017-11-07T22:07:08ZJing Taotao@nceas.ucsb.edu
<p>Val from ess-dive reported:<br />```mn_1 | In D1URLFilter.<br />mn_1 | HTTP Verb: GET<br />mn_1 | original pathInfo: /object<br />mn_1 | original requestURI: /metacat/d1/mn/v2/object<br />mn_1 | stripping /metacat/d1/mn/v2 from requestURI<br />mn_1 | new pathinfo: /object<br />mn_1 | In D1URLFilter.<br />mn_1 | HTTP Verb: GET<br />mn_1 | original pathInfo: /object/10.5072/D3P26Q35R-Test2<br />mn_1 | original requestURI: /metacat/d1/mn/v2/object/10.5072/D3P26Q35R-Test2<br />mn_1 | stripping /metacat/d1/mn/v2 from requestURI<br />mn_1 | new pathinfo: /object/10.5072/D3P26Q35R-Test2<br />mn_1 | After decoded: 10.5072/D3P26Q35R-Test2<br />mn_1 | metacat 20171107-20:49:17: [DEBUG]: In isAdminAuthorized(), session is null [edu.ucsb.nceas.metacat.dataone.D1NodeService:isAdminAuthorized:1010]<br />mn_1 | metacat 20171107-20:49:17: [DEBUG]: D1NodeService.isAuthoritativeMNodeAdmin - the session object is null and return false. [edu.ucsb.nceas.metacat.dataone.D1NodeService:isAuthoritativeMNodeAdmin:924]<br />mn_1 | metacat 20171107-20:49:17: [DEBUG]: Comparing the rights holder in the system metadata 0000-0001-9061-8952 against one of the client's subject public [edu.ucsb.nceas.metacat.dataone.D1NodeService:userHasPermission:1248]<br />mn_1 | metacat 20171107-20:49:17: [DEBUG]: D1NodeService.expandRightHolder - at the start of method: after getting the cn node and cn node is <a class="external" href="https://cn-stage-2.test.dataone.org/cn/v2">https://cn-stage-2.test.dataone.org/cn/v2</a> [edu.ucsb.nceas.metacat.dataone.D1NodeService:expandRightsHolder:1312]<br />mn_1 | metacat 20171107-20:49:17: [DEBUG]: D1NodeService.expandRightHolder - search the subject 0000-0001-9061-8952 in the cn and the returned result is not null [edu.ucsb.nceas.metacat.dataone.D1NodeService:expandRightsHolder:1321]<br />mn_1 | metacat 20171107-20:49:17: [DEBUG]: D1NodeService.isInGroups - the given groups' (the returned result including groups) size is 0 [edu.ucsb.nceas.metacat.dataone.D1NodeService:isInGroups:1374]<br />mn_1 | metacat 20171107-20:49:17: [DEBUG]: D1NodeService.expandRightHolder - we are already at the end of the returned restult since the size of returned results 1 is less than the count 200. So we have to break the loop and finish the try. [edu.ucsb.nceas.metacat.dataone.D1NodeService:expandRightsHolder:1346]<br />mn_1 | metacat 20171107-20:49:17: [DEBUG]: D1NodeService.expandRightHolder - We can NOT find any member in the group 0000-0001-9061-8952 (if it is a group) matches the user public [edu.ucsb.nceas.metacat.dataone.D1NodeService:expandRightsHolder:1359]<br />mn_1 | metacat 20171107-20:49:17: [DEBUG]: Checking allow access rule for subject: public [edu.ucsb.nceas.metacat.dataone.D1NodeService:userHasPermission:1270]<br />mn_1 | metacat 20171107-20:49:17: [DEBUG]: Access rule contains subject: public [edu.ucsb.nceas.metacat.dataone.D1NodeService:userHasPermission:1272]<br />mn_1 | metacat 20171107-20:49:17: [DEBUG]: Checking permission: read [edu.ucsb.nceas.metacat.dataone.D1NodeService:userHasPermission:1274]<br />mn_1 | metacat 20171107-20:49:17: [INFO]: Permission granted: read to public [edu.ucsb.nceas.metacat.dataone.D1NodeService:userHasPermission:1278]<br />mn_1 | metacat 20171107-20:49:17: [ERROR]: class java.lang.NullPointerException: null [edu.ucsb.nceas.metacat.restservice.v2.MNResourceHandler:handle:526]<br />mn_1 | java.lang.NullPointerException<br />mn_1 | at edu.ucsb.nceas.metacat.restservice.v2.MNResourceHandler.getObject(MNResourceHandler.java:1271)<br />mn_1 | at edu.ucsb.nceas.metacat.restservice.v2.MNResourceHandler.handle(MNResourceHandler.java:285)<br />mn_1 | at edu.ucsb.nceas.metacat.restservice.D1RestServlet.doGet(D1RestServlet.java:76)<br />mn_1 | at javax.servlet.http.HttpServlet.service(HttpServlet.java:624)<br />mn_1 | at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)<br />mn_1 | at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)<br />mn_1 | at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)<br />mn_1 | at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)<br />mn_1 | at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)<br />mn_1 | at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)<br />mn_1 | at edu.ucsb.nceas.metacat.restservice.D1URLFilter.doFilter(D1URLFilter.java:48)<br />mn_1 | at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)<br />mn_1 | at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)<br />mn_1 | at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219)<br />mn_1 | at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:110)<br />mn_1 | at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:506)<br />mn_1 | at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)<br />mn_1 | at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)<br />mn_1 | at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:962)<br />mn_1 | at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)<br />mn_1 | at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:445)<br />mn_1 | at org.apache.coyote.ajp.AjpAprProcessor.process(AjpAprProcessor.java:188)<br />mn_1 | at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:637)<br />mn_1 | at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.doRun(AprEndpoint.java:2549)<br />mn_1 | at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.run(AprEndpoint.java:2538)<br />mn_1 | at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)<br />mn_1 | at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)<br />mn_1 | at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)<br />mn_1 | at java.lang.Thread.run(Thread.java:748)<br />mn_1 | metacat 20171107-20:49:17: [ERROR]: D1ResourceHandler: Serializing exception with code 500: null [edu.ucsb.nceas.metacat.restservice.D1ResourceHandler:serializeException:533]<br />mn_1 | org.dataone.service.exceptions.ServiceFailure<br />mn_1 | at edu.ucsb.nceas.metacat.restservice.v2.MNResourceHandler.handle(MNResourceHandler.java:533)<br />mn_1 | at edu.ucsb.nceas.metacat.restservice.D1RestServlet.doGet(D1RestServlet.java:76)<br />mn_1 | at javax.servlet.http.HttpServlet.service(HttpServlet.java:624)<br />mn_1 | at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)<br />mn_1 | at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)<br />mn_1 | at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)<br />mn_1 | at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)<br />mn_1 | at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)<br />mn_1 | at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)<br />mn_1 | at edu.ucsb.nceas.metacat.restservice.D1URLFilter.doFilter(D1URLFilter.java:48)<br />mn_1 | at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)<br />mn_1 | at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)<br />mn_1 | at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219)<br />mn_1 | at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:110)<br />mn_1 | at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:506)<br />mn_1 | at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)<br />mn_1 | at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)<br />mn_1 | at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:962)<br />mn_1 | at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)<br />mn_1 | at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:445)<br />mn_1 | at org.apache.coyote.ajp.AjpAprProcessor.process(AjpAprProcessor.java:188)<br />mn_1 | at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:637)<br />mn_1 | at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.doRun(AprEndpoint.java:2549)<br />mn_1 | at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.run(AprEndpoint.java:2538)<br />mn_1 | at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)<br />mn_1 | at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)<br />mn_1 | at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)<br />mn_1 | at java.lang.Thread.run(Thread.java:748)<br />```</p> Bug #7221 (Resolved): Remove the duplicated the log4j file in Metacathttps://projects.ecoinformatics.org/ecoinfo/issues/72212017-10-20T22:36:49ZJing Taotao@nceas.ucsb.edu
<p>Currently we have to copies of log4j files in Metacat. It causes the problem in tomcat 8 in linux. We need to remove the old version one.</p> Task #7218 (Closed): Merge revision 10194 into the next release branchhttps://projects.ecoinformatics.org/ecoinfo/issues/72182017-10-13T00:36:17ZBryce Mecummecum@nceas.ucsb.edu
<p>I could just Slack this but I thought I'd do an Issue since it's after-hours and I didn't want to send an email. I made a change, revision 10194 back in March and it looks like it never got merged onto a branch for release so the bug it fixed is still affecting Metacat installs. I think the reason it didn't get merged is because I didn't know we cherry-picked commits onto branches to manage release but I know this now!</p>
<p>Jing, could you please merge this sometime so it gets deployed during the next deploy cycle?</p> Bug #7214 (Closed): Metacat overwrites the deployDir propertyhttps://projects.ecoinformatics.org/ecoinfo/issues/72142017-10-05T00:51:02ZChris Jonescjones@nceas.ucsb.edu
<p>On a fresh installation of Metacat, the <code>application.deployDir</code> property value gets overwritten. This is undesirable for Docker deployments where the Metacat Admin UI is not used for configuration, but rather the <code>metacat.properties</code> file is pre-configured for deployment. Fix this. It looks to be happening in the <code>PropertiesAdmin.configureProperties()</code> method:<br /><pre>
PropertyService.setPropertyNoPersist("application.deployDir",
SystemUtil.discoverDeployDir(request));
</pre></p>
<p>Look at the consequences of not calling <code>discoverDeployDir()</code> and perhaps remove this. Needs review.</p> Bug #7211 (Closed): Metacat sets a wrong default value for the deploy path during the totally fre...https://projects.ecoinformatics.org/ecoinfo/issues/72112017-10-03T16:01:43ZJing Taotao@nceas.ucsb.edu
<p>Matt reported: I’ve noticed that when run from scratch, the metacat /admin utility seems to tack the /webapps/metacat onto the deployDir property, and I always have to change it. SO I think that is an admin utility bug on a totally clean deployment (i.e., with no backup directory in place).<br />so, this arises with clean deploys from docker and the like.</p> Bug #7208 (Rejected): Metacat is double-decoding incoming urls on the CNshttps://projects.ecoinformatics.org/ecoinfo/issues/72082017-09-06T21:33:25ZJing Taotao@nceas.ucsb.edu
<p>Details see <br /><a class="external" href="https://redmine.dataone.org/issues/8122">https://redmine.dataone.org/issues/8122</a></p> Task #7206 (Resolved): Change the behavior to archive the older version of the resource map objec...https://projects.ecoinformatics.org/ecoinfo/issues/72062017-08-17T05:15:31ZJing Taotao@nceas.ucsb.edu
<p>Do you know if Morpho is by default archiving the resource map when a new version of a data package is pushed to KNB? <br />I am asking because I realized that when I try to go back to previous versions of our SNAPP packages, it seems that all the previous resourceMaps were archived (thanks for @peter for helping me to understand that) restricting the access to previous versions to the EML only. Here as example: <a class="external" href="https://knb.ecoinformatics.org/#view/karakoenig.46.14">https://knb.ecoinformatics.org/#view/karakoenig.46.14</a> as the lateset and <a class="external" href="https://knb.ecoinformatics.org/#view/karakoenig.46.12">https://knb.ecoinformatics.org/#view/karakoenig.46.12</a> as a previously shared version with the PIs.</p> Bug #7201 (Closed): Some DataONE service packages not being reportedhttps://projects.ecoinformatics.org/ecoinfo/issues/72012017-06-27T21:11:47ZPeter Slaughterslaughter@nceas.ucsb.edu
<p>The DataONE 'node' service requests a member node to provide a list of service packages that it supports, for example <a class="external" href="https://knb.ecoinformatics.org/knb/d1/mn/v2/node">https://knb.ecoinformatics.org/knb/d1/mn/v2/node</a><br />will list the implemented packages:<br />...<br /><services><br /><service name="MNCore" version="v1" available="true"/><br /><service name="MNCore" version="v2" available="true"/><br /><service name="MNRead" version="v1" available="true"/><br /><service name="MNRead" version="v2" available="true"/><br /><service name="MNAuthorization" version="v1" available="true"/><br /><service name="MNAuthorization" version="v2" available="true"/><br /><service name="MNStorage" version="v1" available="true"/><br /><service name="MNStorage" version="v2" available="true"/><br /><service name="MNReplication" version="v1" available="true"/><br /><service name="MNReplication" version="v2" available="true"/><br /></services><br />...</p>
<p>However, it appears that not all service packages are being reported, as KNB supports the 'getPackage' service that is part of "MNPackage", which does not appear.<br />Also, "MNView" and "MNQuery" do not appear.</p>
<p>The full list of course, is here: <a class="external" href="https://purl.dataone.org/architecture-dev/apis/MN_APIs.html">https://purl.dataone.org/architecture-dev/apis/MN_APIs.html</a></p> Bug #7196 (Resolved): MN.Delete should continue even though it can't find the local filehttps://projects.ecoinformatics.org/ecoinfo/issues/71962017-05-30T20:32:56ZJing Taotao@nceas.ucsb.edu
<p>The MN.delete method stops and throws an exception when it can't find the local file. Somehow this scenario, which is not normal, happens - only system metadata exists and we can't delete it since the local file doesn't exist. But users really want to delete it. So the MN.delete method should continue even though it can't find the local file.</p> Bug #7195 (Resolved): LDAP-based group authorization is failinghttps://projects.ecoinformatics.org/ecoinfo/issues/71952017-05-26T17:44:46ZChris Jonescjones@nceas.ucsb.edu
<p>In calls to <code>MNRead.getSystemMetadata()</code> where the user logged in is a member of a group, and the group is listed in the <code>SystemMetadata.AccessPolicy</code> with read, write, and changePermission permissions, authorization to read the private object is not allowed. In <code>D1NodeService.getSystemMetadata()</code>, we call <code>isAuthorized()</code>, which in turn calls <code>userHasPermission()</code>. Recent changes to this code allows for DataONE group authorization to work. However, group DNs defined in the <code>dc=ecoinformatics,dc=org</code> LDAP tree that are listed in the <code>AccessPolicy</code> and stored Metacat look to not be expanded to their individual group members when comparing DNs. On lines 1229 to 1255 of <code>D1NodeService</code>, we call:<br /><pre>
if (accessRule.getSubjectList().contains(s)) {
logMetacat.debug("Access rule contains subject: " + s.getValue());
for (Permission p: accessRule.getPermissionList()) {
logMetacat.debug("Checking permission: " + p.xmlValue());
expandedPermissions = expandPermissions(p);
allowed = expandedPermissions.contains(permission);
if (allowed) {
logMetacat.info("Permission granted: " + p.xmlValue() + " to " + s.getValue());
break search; //label break
}
}
}
</pre><br />It looks like the call to <code>accessRule.getSubjectList()</code> is not expanding the the list of subjects if the subject DN is a group from the locally configured Metacat auth store (LDAP or file store).</p>
<p>An example is in the KNB: <code>kengmiller.14.15</code>. This object has an access rule allowing <code>cn=snapp,o=NCEAS,dc=ecoinformatics,dc=org</code> to r/w/chP. Members of the group can not read the object, much less modify it. Jing, I've added you to the SNAPP group for troubleshooting this.</p>
<p>Change <code>userHasPermission()</code> to correctly expand all groups, and write a unit test to exercise group-based authorization from auth file, LDAP, or DataONE group definitions.</p>