Metacat: Issueshttps://projects.ecoinformatics.org/ecoinfo/https://projects.ecoinformatics.org/ecoinfo/ecoinfo/favicon.ico?14691340362017-10-05T00:43:02ZEcoinformatics Redmine
Redmine Bug #7213 (New): Document the EZID landing page template propertyhttps://projects.ecoinformatics.org/ecoinfo/issues/72132017-10-05T00:43:02ZChris Jonescjones@nceas.ucsb.edu
<p>Mike Frenock pointed out some issues with the <code>guid.ezid.uritemplate.metadata</code> property. We need to add this to the documentation. Also, consider if we should make this configurable to an external server instad of the local Metacat server.</p> Bug #7203 (In Progress): Improve D1NodeService.isAuthorized() performancehttps://projects.ecoinformatics.org/ecoinfo/issues/72032017-07-22T23:14:48ZChris Jonescjones@nceas.ucsb.edu
<p>We're seeing poor performance in calls to <code>D1NodeService.isAuthorized()</code> on <a class="external" href="https://arcticdata.io">https://arcticdata.io</a>. When the system Metacat is under light load (< 10 requests per second), calls to <code>isAuthorized()</code> are taking up to 35 seconds to return either an <code>HTTP 200</code> response or a <code>HTTP 403</code> exception.</p>
<p>Change <code>isAuthorized()</code> to prioritize user-based authorization first, and then CN or MN authorization last. This should increase performance for end users, whereas MN to MN replication calls and CN-administrative calls will be slightly less prioritized.</p>
<p>Note that calls to <code>userHasPermission()</code> involve token verification using the <code>PortalCertificateManager</code> and the <code>TokenGenerator</code>. These calls may be repeatedly making a call to the CN to get the SSL certificate for verification if it is not cached. If this change doesn't significantly improve performance, look into refactoring those classes in <code>d1_portal</code> to cache and use the certificate, unless there is a verification exception, in which case we make the call to <code>fetchCertificate()</code> again, re-cache it, and attempt to re-verify the token. If it still fails, throw <code>NotAuthorized</code>.</p> Bug #7200 (Resolved): Remove redundant alternate identifier form in register-dataset.cgi templatehttps://projects.ecoinformatics.org/ecoinfo/issues/72002017-06-19T15:55:13ZChris Jonescjones@nceas.ucsb.edu
<p>We somehow have duplicated the <code>alternateIdentifier</code> section of the <code>entryForm.tmpl</code> template, which is causing display issues when viewing metadata documents. After looking at the commit history, it looks like a merge issue. Remove the duplicate form.</p> Bug #7187 (Resolved): Set file names correctly when reading objects from Metacathttps://projects.ecoinformatics.org/ecoinfo/issues/71872017-05-10T15:05:57ZChris Jonescjones@nceas.ucsb.edu
<p>We now store file names in <code>SystemMetadata.fileName</code>. In <code>MetacatHandler.readFromMetacat()</code>, we are currently generating a file name based on the <code>docid</code>, and setting it into the <code>Content-Disposition</code> header. Change <code>generateOutputName()</code> to use <code>SystemMetadata.fileName</code> when it's available, and fall back to generating a file name.</p> Bug #7178 (New): MNodeService.getPackage() takes too long for large packageshttps://projects.ecoinformatics.org/ecoinfo/issues/71782017-03-29T14:22:46ZChris Jonescjones@nceas.ucsb.edu
<p>When users click on the <code>Download All</code> button in MetacatUI, we call <code>MN.getPackage()</code> to zip up the members, create HTML and PDF metadata, etc. For very large packages, the packaging time is far too long for a decent user experience. To address this, we may need to provide some sort of progress API call that allows the client to get estimated packaging time and provide a progress bar for the user. Also, <code>getPackage()</code> uses <code>File.createTempFile()</code> to copy contents into a single directory tree for zipping (really BagIt bagging). This doesn't scale well for large packages (GBs). We can explore a few strategies to mitigate this. One that comes to mind is using hard symbolic links to the original data files in the directory tree rather than copying them. This needs some thought, but ultimately we need to speed up the packaging process for large packages.</p> Bug #7018 (Closed): MNodeService().updateSystemMetadata() doesn't honor the MN certificatehttps://projects.ecoinformatics.org/ecoinfo/issues/70182016-04-30T17:18:22ZChris Jonescjones@nceas.ucsb.edu
<p>When calling MNStorage.updateSystemMetadata() with the Member Node cert (like CN=urn:node:ARCTIC, DC=dataone,DC=org), Metacat throws a NotAuthorized exception. It should honor the certificate for pids where this MN is authoritative.</p> Bug #6619 (Closed): Add support for ORNL Mercury metadata schema to the Metacat schema cataloghttps://projects.ecoinformatics.org/ecoinfo/issues/66192014-11-13T19:50:11ZChris Jonescjones@nceas.ucsb.edu
<p>We cache a number of schemas in Metacat, and need to add support for the Oak Ridge National Lab Mercury schema. Add this to the schema/ornl directory, and modify the xml_catalog SQL population scripts to register the schema on install or upgrade. See the DataONE ticket: <a class="external" href="https://redmine.dataone.org/issues/6353">https://redmine.dataone.org/issues/6353</a>.</p> Feature #6419 (Closed): Add Dryad and DataONE schemas to Metacat cataloghttps://projects.ecoinformatics.org/ecoinfo/issues/64192014-02-14T19:23:04Zben leinfelderleinfelder@nceas.ucsb.edu
<p>Rather than rely on xsi:schemaLocation being accessible and valid, we should incorporate released schemas from DataONE and Dryad to support validation of their content - both for MN and CN use of Metacat.</p> Bug #6323 (Closed): CNodeService.archive() doesn't increment serialVersion correctly https://projects.ecoinformatics.org/ecoinfo/issues/63232013-12-20T15:31:25ZChris Jonescjones@nceas.ucsb.edu
<p>In calling CNodeService.archive(), and perhaps delete() too, the serialVersion isn't set correctly. Check this in DocumentImpl too. Related to <a class="external" href="https://redmine.dataone.org/issues/4210">https://redmine.dataone.org/issues/4210</a></p> Bug #6322 (Rejected): Ensure lock is obtained prior to calls to Hazelcast IMap.put() https://projects.ecoinformatics.org/ecoinfo/issues/63222013-12-20T15:28:46ZChris Jonescjones@nceas.ucsb.edu
<p>When modifying the system metadata table, we call systemMetadataMap.put(). since this is a distributed map in the HZ cluster, we need to call lock() first on the pid. This particularly needs to be fixed in DocumentImpl, but we should audit this call in other classes to make sure we lock and unlock correctly.</p> Bug #6315 (Closed): Needs to work for DATA objects as wellhttps://projects.ecoinformatics.org/ecoinfo/issues/63152013-12-19T00:33:37Zben leinfelderleinfelder@nceas.ucsb.edu
<p>Found that calls need to work on data objects where the CN does not actually store the object.<br />See DataONE tracker: <a class="external" href="https://redmine.dataone.org/issues/4204">https://redmine.dataone.org/issues/4204</a></p> Feature #5939 (Closed): Create view service that can render metadata documents at REST URLhttps://projects.ecoinformatics.org/ecoinfo/issues/59392013-05-23T23:44:25ZMatt Jonesjones@nceas.ucsb.edu
<p>We need a 'landing page' for metadata views that can be referenced as REST URLs and that show an HTML'ised version of an object using its PID. A service might have a REST url of the form:</p>
<p><del><a class="external" href="https://metacat.someplace.org/knb/d1/mn/v1/view/{pid">https://metacat.someplace.org/knb/d1/mn/v1/view/{pid</a>}</del><br /><a class="external" href="https://data.somplace.org/metacat/d1/mn/v1/views/{format}/{pid">https://data.somplace.org/metacat/d1/mn/v1/views/{format}/{pid</a>}</p>
<p>where {pid} is the permenant identifier for the metadata document. Such a URL landing page would prbably deliver an HTML version of the metadata. An optional ?format=knb parameter might be used to control which CSS file is linked into the page, or maybe we just omit the CSS altogether and assume we are delivering just an HTML fragment to a client? Needs more discussion.</p>
<p>This service would be used in several places, including the landing page URL for the new backbone based UI, and as the URL that is written into sitemaps that are published to google and elsewhere for search engines to index.</p> Bug #5353 (Resolved): EcogridWriter doesn't work with the metacat built from trunk.https://projects.ecoinformatics.org/ecoinfo/issues/53532011-03-21T22:20:16ZJing Taotao@nceas.ucsb.edu
<p>When I used EcogridWriter to write metadata and data into dev.nceas' metadata, it worked file. I believe the version of the metadata is 1.9.3</p>
<p>I built my local metacat by revision 6013 from the svn trunk.<br />But when i used EcogridWriter to write metadata and data file into it, it gave me the following error:</p>
<p>EML document type: eml://ecoinformatics.org/eml-2.0.0<br />edu.ucsb.nceas.metacat.McdbDocNotFoundException: 2Could not find document doc.130074542695043<br /> at edu.ucsb.nceas.metacat.IdentifierManager.getDocumentInfo(IdentifierManager.java:246)<br /> at edu.ucsb.nceas.metacat.MetacatHandler.createSystemMetadata(MetacatHandler.java:3477)<br /> at edu.ucsb.nceas.metacat.MetacatHandler.createSystemMetadata(MetacatHandler.java:3618)<br /> at edu.ucsb.nceas.metacat.MetacatHandler.handleInsertOrUpdateAction(MetacatHandler.java:1862)<br /> at edu.ucsb.nceas.metacat.MetaCatServlet.handleGetOrPost(MetaCatServlet.java:885)<br /> at edu.ucsb.nceas.metacat.MetaCatServlet.doPost(MetaCatServlet.java:493)<br /> at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)<br /> at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)<br /> at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)<br /> at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)<br /> at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)<br /> at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)<br /> at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:470)<br /> at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)<br /> at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)<br /> at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)<br /> at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)<br /> at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:190)<br /> at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:291)<br /> at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:776)<br /> at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:705)<br /> at org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:898)<br /> at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:690)<br /> at java.lang.Thread.run(Thread.java:619)</p> Bug #5288 (Resolved): Metacat got "java.lang.OutOfMemoryError: PermGen space" after new geoserver...https://projects.ecoinformatics.org/ecoinfo/issues/52882011-02-02T01:13:25ZJing Taotao@nceas.ucsb.edu
<p>Today, I updated my metacat source tree to get the changes on the geoserver part.</p>
<p>After installing the new Metacat, the catalina.out was very easy (it happened 4 times in one hour) to get the message even I didn't have many activities on the Metacat:</p>
<p>java.lang.OutOfMemoryError: PermGen space</p>
<p>Before this updating, I hadn't seen the error message for a long time.</p> Bug #5244 (Resolved): ldapweb.cgi shouldn't report ou=Account accounts since they're unusablehttps://projects.ecoinformatics.org/ecoinfo/issues/52442010-11-17T02:09:47ZShaun Walbridgewalbridge@nceas.ucsb.edu
<p>Because of security considerations, we can't allow users to reuse their credentials from ou=Account, which is where we create the majority of accounts for Plone and SVN. However, when trying to show users other accounts which are the same, we show these accounts, confusing new users.</p>
<p>As a temporary fix, filter out the ou=Account from accounts when making suggestions.</p>