Metacat: Issueshttps://projects.ecoinformatics.org/ecoinfo/https://projects.ecoinformatics.org/ecoinfo/ecoinfo/favicon.ico?14691340362015-08-28T23:41:47ZEcoinformatics Redmine
Redmine Support #6838 (In Progress): LTER user can't log inhttps://projects.ecoinformatics.org/ecoinfo/issues/68382015-08-28T23:41:47ZJing Taotao@nceas.ucsb.edu
<p>marco: ldap.lternet.edu should still work<br />[4:32pm] Jing: but why the search doesn’t work?<br />[4:32pm] Jing: and i can’t log in it from knb web page.<br />[4:34pm] marco: my guess is that the connection is trying to connect to 389, which IIRC is where startTLS initiates<br />[4:34pm] marco: port 389 is now blocked - not my decision<br />[4:34pm] Jing: aha.<br />[4:35pm] Jing: thanks, marco<br />[4:35pm] marco: if necessary, 389 can be opened for a specific IP or range<br />[4:35pm] marco: and startTLS enabled<br />[4:37pm] marco: we'll work with mark schildhauer next week to figure out the disposition of LDAP</p> Bug #5836 (New): Logshttps://projects.ecoinformatics.org/ecoinfo/issues/58362013-01-30T18:16:16ZBrendan Hahnhahn@nceas.ucsb.edu
<p>Logs are kind of a mess, mixing multiple logging object types along with System.* output. Metacat has its own log directory, but only replication events are recorded there. Default build/distribution has uneven log levels set.</p> Bug #5835 (New): Leakshttps://projects.ecoinformatics.org/ecoinfo/issues/58352013-01-30T17:44:53ZBrendan Hahnhahn@nceas.ucsb.edu
<p>There are significant memory leaks that will eventually hang or kill the application server if metacat is restarted without terminating the entire server process. E.g. use of log4j dynamic configuration leaves orphan threads about (not really metacat's fault -- I'm amazed this is <strong>still</strong> a problem). Other threads, classloaders, etc along with the associated resources get left behind as well.</p>
<p>In short: should clean up better.</p> Bug #5828 (New): docids not validatedhttps://projects.ecoinformatics.org/ecoinfo/issues/58282013-01-29T22:58:48ZBrendan Hahnhahn@nceas.ucsb.edu
<p>Externally provided document identifiers are not always checked for syntax (e.g. multipart-form submission) and produce strange errors (e.g. string index out of range) if invalid.</p> Bug #5698 (New): datapackage uploads to dev2 metacat are currently slowhttps://projects.ecoinformatics.org/ecoinfo/issues/56982012-08-25T02:43:20ZDerik Barseghianbarseghian@nceas.ucsb.edu
<p>When using the sensor-view archiveDataturbineDataToMetacat.kar to archive data from my laptop at ucsb to dev2's metacat, the upload sections of the workflow takes at minimim ~125s to complete, even when the data is small (~30 timestamp/datapoint pairs). A tomcat restart didn't help. When archiving roughly the same amounts of data to Jing's chico1 metacat, the upload sections ran about 4 times faster (30-40s). For more details see:<br /><a class="external" href="http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5403#c9">http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5403#c9</a></p> Bug #5608 (New): Enable all FK constraints in Metacat production [copies]https://projects.ecoinformatics.org/ecoinfo/issues/56082012-05-14T19:14:31Zben leinfelderleinfelder@nceas.ucsb.edu
<p>Looks like the FK constraints have been removed from the production knb database.</p>
<blockquote>
<p>select conname, contype, conkey, confkey from pg_constraint;</p>
</blockquote>
<p>Need to see what FKs are no long satisfied and potentially fix them as best we can so that the constraints can be re enabled.</p> Bug #5599 (New): absence of line feeds in eml causes pathQuery to not find some elementshttps://projects.ecoinformatics.org/ecoinfo/issues/55992012-05-07T22:54:42Zgastil gastilmarygastil@yahoo.com
<p>Presence of line feeds seems to be needed for an eml doc to get loaded properly so pathQuery can find attributeList or attribute. Not just one line feed at the end.<br />We detected this on metacat 1.9.5 at metacat.lternet but tested it on metacat 2.0 (lava.lternet)</p>
<p>Evidence:<br />in lava.lternet.edu</p>
<p>knb-lter-kbs.10.19 has no line feeds at all in the document.</p>
<p>revision 20 is same as 19 except stmml-1.1 is spelled right.<br />revision 21 is same as 20 except it has one line feed at the end of the file.<br />(so revision 21 has one line)</p>
<p>revisions 19 thru 21, while they were the last revision, did not have their attributeList found by pathQuery.</p>
<p>revision 22, with 165 lines feeds DOES have its attributeList seen by pathQuery.</p>
<p>wc -l knb-lter-kbs.10.*<br /> 0 knb-lter-kbs.10.19.xml<br /> 0 knb-lter-kbs.10.20.xml<br /> 1 knb-lter-kbs.10.21.xml<br /> 165 knb-lter-kbs.10.22.xml</p>
<p>pathQuery result snippets from two separate queries (when two different revisions were the last revision):</p>
<p><document><br /><docid>knb-lter-kbs.10.22</docid><br /><docname>eml</docname><br /><doctype>eml://ecoinformatics.org/eml-2.1.0</doctype><br /><createdate>2012-05-07</createdate><br /><updatedate>2012-05-07</updatedate><br /><param name="attributeList"></param><br /><param name="@packageId">knb-lter-kbs.10.22</param><br /></document></p>
<p>older query:</p>
<p><document><br /><docid>knb-lter-kbs.10.21</docid><br /><docname>eml</docname><br /><doctype>eml://ecoinformatics.org/eml-2.1.0</doctype><br /><createdate>2012-05-07</createdate><br /><updatedate>2012-05-07</updatedate><br /><param name="@packageId">knb-lter-kbs.10.22</param><br /></document></p> Bug #5553 (New): setaccess action may have deleting access rule functionalityhttps://projects.ecoinformatics.org/ecoinfo/issues/55532011-11-23T22:55:24ZJing Taotao@nceas.ucsb.edu
<p>Currently, setaccess action can only add access rules to the metacat. There is a limitation.</p>
<p>Sometimes, we want to keep granting or revoking a allow public read access rules for a document.</p>
<p>If we choose "denyFirst" as the order type(metacat can't change order type when a document set it), we can't revoke the public readable access if we granted it.</p>
<p>If we choose "allowFirst" as the order type, we can't regrant it if we denied it.</p>
<p>So if setaccess action (or we can have another action deleteaccess), this scenario can be avoided.</p> Bug #5548 (New): Replace COS Mime multipart libraryhttps://projects.ecoinformatics.org/ecoinfo/issues/55482011-11-21T22:22:38ZJing Taotao@nceas.ucsb.edu
<p>Here is a comment from Matt in bug 5543:</p>
<p>The COS Mime multipart library is incredibly old, and we should<br />seriously consider replacing it. On the client side for DataONE, we've<br />switched to using the nicely maintained Apache HttpComponents implementation,<br />particularly the MIME MultipartEntity class for representing our mime messages<br />(<a class="external" href="http://hc.apache.org/httpcomponents-client-ga/httpmime/apidocs/org/apache/http/entity/mime/MultipartEntity.html">http://hc.apache.org/httpcomponents-client-ga/httpmime/apidocs/org/apache/http/entity/mime/MultipartEntity.html</a>).</p> Bug #5535 (New): The 611 CDR documents replicated to KNB need to be repaired by fixing their inli...https://projects.ecoinformatics.org/ecoinfo/issues/55352011-11-09T18:31:39ZJing Taotao@nceas.ucsb.edu
<p>This bug is the section 4 of the ongoing concerns:<br /><a class="external" href="http://bugzilla.ecoinformatics.org/show_bug.cgi?id=3296">http://bugzilla.ecoinformatics.org/show_bug.cgi?id=3296</a></p>
<p>During the replication, the "&" and "<" in the inline data section of the 611 CDR document in LTER were changed to "&" and "<" in knb site.</p>
<p>We need to fix it. Duane proposed a solution on the bug 3296.</p> Bug #5239 (New): Difficulties configuring and viewing Metacat replication log filehttps://projects.ecoinformatics.org/ecoinfo/issues/52392010-11-12T17:31:01ZDuane Costadcosta@lternet.edu
<p>While working with Chad to restore replication between KNB and LTER, I encountered issues with configuring and viewing the replication log file. As detailed in the email exchanges below, I configured a replication log directory and file name in the following three places:</p>
<p>1. Metacat administrative interface, 'Metacat Properties Configuration' page:</p>
<pre><code>Replication Log Directory /home/tcat/local/metacat_data/metacat/logs<br /> [The directory where replication log should be located.]</code></pre>
<p>2. metacat.properties file</p>
<pre><code>replication.logdir=/home/tcat/local/metacat_data/metacat/logs</code></pre>
<p>3. log4j.properties file<br /> log4j.appender.replication.File=/home/tcat/local/metacat_data/metacat/logs/metacatreplication.log</p>
<p>However, Metacat seems to ignore these configuration values and instead sends the output to the Tomcat log file, 'catalina.out'.</p>
<hr />
<p>In an email to Chad Berkley on 2010-11-12, Duane Costa wrote:</p>
<pre><code>Hi Chad,<br /> .<br /> .<br /> .<br /> There's still a problem with the replication log file. I configured it<br /> exactly as described below, but the file has still not been touched since<br /> 4/26/2010. There is, however, a bunch of replication output in the Tomcat<br /> catalina.out file. Apparently, Metacat is ignoring my configuration<br /> settings and sending the output to the default location of catalina.out.<br /> This is a fairly minor issue, but I'll open a Bugzilla bug report on it so<br /> that it can be recorded.</code></pre>
<pre><code>Thanks,<br /> Duane</code></pre>
<hr />
<p>In an email to Duane Costa on 2010-11-10, Chad Berkley wrote:</p>
<p>On 11/10/10 2:19 PM, Duane Costa wrote:</p>
<blockquote>
<p>Ours is configured as follows:</p>
<p>log4j.appender.replication.File = ${replication.logfile.name}</p>
<p>On the other hand, the metacat.properties file has:</p>
<p>replication.logdir=/home/tcat/local/metacat_data/metacat/logs</p>
<p>And the Metacat administrative interface displays the following:</p>
<p>Replication Log Directory /home/tcat/local/metacat_data/metacat/logs<br />[The directory where replication log should be located.]</p>
<p>This raises a few questions. You don't have to try to answer them. I'm<br />just trying to point out that the current situation is confusing, and<br />hopefully this can be simplified in future versions of Metacat.</p>
<p>1. Do we configure a directory name or a file name? (A file name<br />according to the first setting; a directory name according to the next<br />two.)</p>
</blockquote>
<p>I'm not sure of the original intent here. It would be nice if we just gave it a directory and all of the log files are just put there.</p>
<blockquote>
<p>2. There used to be two different replication output files:<br />'metacatreplicationerror.log', 'metacatreplication.log'. Is there now<br />only one?</p>
</blockquote>
<p>Again, not sure. Seems like there should really only be one.</p>
<blockquote>
<p>3. If log4j.properties now controls where the replication output is<br />sent, are the other two settings no longer used by the system?</p>
</blockquote>
<p>It's always controlled the output, I just think that there's something wrong with the token system that isn't updating the log4j file. I think there's a bug in the admin interface where it's not updating the log4j.properties file as well as the metacat.properties file. I think that token is supposed to be replaced by the actual path to the file.</p>
<blockquote>
<p>4. Should '${replication.logfile.name}' in log4j.properties be manually<br />edited, or is 'replication.logfile.name' the name of a property that is<br />being set somewhere else? If the latter, where is it being controlled? I<br />don't see it in either build.properties or metacat.properties.</p>
</blockquote>
<p>It's safe to manually edit it. I just did and it worked fine.</p>
<blockquote>
<p>For now I will explicitly set the value in log4j.properties to the<br />following:</p>
<p>log4j.appender.replication.File=/home/tcat/local/metacat_data/metacat/logs/metacatreplication.log</p>
</blockquote>
<p>Yep, that should work.</p>
<p>I'm seeing a bunchof files coming from lter now, unfortunately there are also a lot of errors, but I'm not sure if they're critical. Here's an example:</p>
<p>knb 20101110-14:42:45: [ERROR]: ReplicationHandler.handleSingleXMLDocument - Failed to write doc knb-lter-gce.113.11 into db because The file you are trying to write already exists in metacat. Please update your version number. [ReplicationLogging]</p>
<p>It might just be trying to get all of your files and using that exception to handle the "already exists" case, but I'm not exactly sure. Lets see if it mirrors correctly when it's done.</p>
<p>chad</p>
<blockquote>
<p>Thanks,<br />Duane</p>
<p>On 11/10/2010 4:43 PM, Chad Berkley wrote:</p>
<blockquote>
<p>Check your webapps/knb/WEB-INF/log4j.properties file. It should show<br />you where the output for replication is going. I've attached the<br />log4j.properties file from knb for reference. By default, all logs<br />will be sent to catalina.out in tomcat/logs. If you configure the<br />replication log, it will go to whatever file you configure<br />(/var/metacat/logs/metacatreplication.log in the attached file).</p>
<p>chad</p>
</blockquote></blockquote> Bug #5224 (New): Scheduler part of SanParks skin needs to be updatedhttps://projects.ecoinformatics.org/ecoinfo/issues/52242010-10-23T01:24:47ZJing Taotao@nceas.ucsb.edu
<p>Workflow run engine and scheduler service were updated recently. The main change is add more parameteres, such as the user can specify workflow run engine and destination repository name.</p>
<p>So the scheduler part in sanParks need to be updated as well.</p> Bug #4892 (New): ESA registry shouldn't allow accepted docs to be modifiedhttps://projects.ecoinformatics.org/ecoinfo/issues/48922010-03-18T18:33:02ZJim Regetzregetz@nceas.ucsb.edu
<p>As submitter of an ESA document, I was able to modify the doc <strong>after</strong> it was accepted by the moderator, which I believe shouldn't happen. There is no GUI element to do it, so it's not as though this is likely to happen, but as either the submitter or the moderator, I can edit an accepted doc by entering a URL like the following:</p>
<p><a class="external" href="http://data.esa.org/esa/cgi-bin/register-dataset.cgi?stage=modify&cfg=esa&docid=esa.70">http://data.esa.org/esa/cgi-bin/register-dataset.cgi?stage=modify&cfg=esa&docid=esa.70</a></p>
<p>To reiterate, this doc (esa.70) was already accepted when I did this, but I was nevertheless able to edit it and submit a modification. This triggered a 'doc has been revised' email to the ESA moderator list, and threw the doc back into the moderation queue.</p> Bug #4843 (New): ESA registry: Remove 'station' default value from Organization Namehttps://projects.ecoinformatics.org/ecoinfo/issues/48432010-02-25T00:53:13ZJim Regetzregetz@nceas.ucsb.edu
<p>In the ESA registration form, the Organization Name field under Basic Information is prepopulated with the value 'station'. This shouldn't be in there.</p> Bug #4840 (New): When registering, U.S. State/Territory of owner is also applied to contacthttps://projects.ecoinformatics.org/ecoinfo/issues/48402010-02-24T23:50:25ZJim Regetzregetz@nceas.ucsb.edu
<p>When registering a new data set, dropdown menus for "U.S. State or Territory" appear both in the Principal Data Set Owner section and in the Data Set Contact section. However, whatever value is submitted under Owner is evidently getting applied to Contact as well, and the value (if any) selected under Contact is ignored. This is true even if no state was selected under Owner, in which case no state is recorded for Contact.</p>
<p>This was observed both for the ESA and NCEAS registries.</p>