Metacat: Issueshttps://projects.ecoinformatics.org/ecoinfo/https://projects.ecoinformatics.org/ecoinfo/ecoinfo/favicon.ico?14691340362011-05-05T00:11:34ZEcoinformatics Redmine
Redmine Bug #5393 (Resolved): SANParks FGDC skin issueshttps://projects.ecoinformatics.org/ecoinfo/issues/53932011-05-05T00:11:34Zben leinfelderleinfelder@nceas.ucsb.edu
<p>1. the generated ID somehow uses my name (from LDAP?) rather than just my uid. It used to be "leinfelder"but now i get a doc id like: "ben leinfelder.1.1" which I believe can cause problems (the white space, that is).</p>
<p>2. when I attempt to insert a second package, it tries to reuse the same ID and never seems to increment it's counter (not sure where it's getting the last id from, but it could be related the the whitespace in the ID's scope).</p> Bug #5316 (Resolved): Interactive map doesn't work with firefoxhttps://projects.ecoinformatics.org/ecoinfo/issues/53162011-02-19T14:57:48ZEva-Maria Gerstnerevamaria.gerstner@googlemail.com
<p>The interactive map, for example</p>
<p><a class="external" href="http://knb.ecoinformatics.org/knb/style/skins/nceas/map.jsp">http://knb.ecoinformatics.org/knb/style/skins/nceas/map.jsp</a></p>
<p>doesn't work with firefox. The "loading map" text is displayed forever without any map appearing.</p> Bug #5239 (New): Difficulties configuring and viewing Metacat replication log filehttps://projects.ecoinformatics.org/ecoinfo/issues/52392010-11-12T17:31:01ZDuane Costadcosta@lternet.edu
<p>While working with Chad to restore replication between KNB and LTER, I encountered issues with configuring and viewing the replication log file. As detailed in the email exchanges below, I configured a replication log directory and file name in the following three places:</p>
<p>1. Metacat administrative interface, 'Metacat Properties Configuration' page:</p>
<pre><code>Replication Log Directory /home/tcat/local/metacat_data/metacat/logs<br /> [The directory where replication log should be located.]</code></pre>
<p>2. metacat.properties file</p>
<pre><code>replication.logdir=/home/tcat/local/metacat_data/metacat/logs</code></pre>
<p>3. log4j.properties file<br /> log4j.appender.replication.File=/home/tcat/local/metacat_data/metacat/logs/metacatreplication.log</p>
<p>However, Metacat seems to ignore these configuration values and instead sends the output to the Tomcat log file, 'catalina.out'.</p>
<hr />
<p>In an email to Chad Berkley on 2010-11-12, Duane Costa wrote:</p>
<pre><code>Hi Chad,<br /> .<br /> .<br /> .<br /> There's still a problem with the replication log file. I configured it<br /> exactly as described below, but the file has still not been touched since<br /> 4/26/2010. There is, however, a bunch of replication output in the Tomcat<br /> catalina.out file. Apparently, Metacat is ignoring my configuration<br /> settings and sending the output to the default location of catalina.out.<br /> This is a fairly minor issue, but I'll open a Bugzilla bug report on it so<br /> that it can be recorded.</code></pre>
<pre><code>Thanks,<br /> Duane</code></pre>
<hr />
<p>In an email to Duane Costa on 2010-11-10, Chad Berkley wrote:</p>
<p>On 11/10/10 2:19 PM, Duane Costa wrote:</p>
<blockquote>
<p>Ours is configured as follows:</p>
<p>log4j.appender.replication.File = ${replication.logfile.name}</p>
<p>On the other hand, the metacat.properties file has:</p>
<p>replication.logdir=/home/tcat/local/metacat_data/metacat/logs</p>
<p>And the Metacat administrative interface displays the following:</p>
<p>Replication Log Directory /home/tcat/local/metacat_data/metacat/logs<br />[The directory where replication log should be located.]</p>
<p>This raises a few questions. You don't have to try to answer them. I'm<br />just trying to point out that the current situation is confusing, and<br />hopefully this can be simplified in future versions of Metacat.</p>
<p>1. Do we configure a directory name or a file name? (A file name<br />according to the first setting; a directory name according to the next<br />two.)</p>
</blockquote>
<p>I'm not sure of the original intent here. It would be nice if we just gave it a directory and all of the log files are just put there.</p>
<blockquote>
<p>2. There used to be two different replication output files:<br />'metacatreplicationerror.log', 'metacatreplication.log'. Is there now<br />only one?</p>
</blockquote>
<p>Again, not sure. Seems like there should really only be one.</p>
<blockquote>
<p>3. If log4j.properties now controls where the replication output is<br />sent, are the other two settings no longer used by the system?</p>
</blockquote>
<p>It's always controlled the output, I just think that there's something wrong with the token system that isn't updating the log4j file. I think there's a bug in the admin interface where it's not updating the log4j.properties file as well as the metacat.properties file. I think that token is supposed to be replaced by the actual path to the file.</p>
<blockquote>
<p>4. Should '${replication.logfile.name}' in log4j.properties be manually<br />edited, or is 'replication.logfile.name' the name of a property that is<br />being set somewhere else? If the latter, where is it being controlled? I<br />don't see it in either build.properties or metacat.properties.</p>
</blockquote>
<p>It's safe to manually edit it. I just did and it worked fine.</p>
<blockquote>
<p>For now I will explicitly set the value in log4j.properties to the<br />following:</p>
<p>log4j.appender.replication.File=/home/tcat/local/metacat_data/metacat/logs/metacatreplication.log</p>
</blockquote>
<p>Yep, that should work.</p>
<p>I'm seeing a bunchof files coming from lter now, unfortunately there are also a lot of errors, but I'm not sure if they're critical. Here's an example:</p>
<p>knb 20101110-14:42:45: [ERROR]: ReplicationHandler.handleSingleXMLDocument - Failed to write doc knb-lter-gce.113.11 into db because The file you are trying to write already exists in metacat. Please update your version number. [ReplicationLogging]</p>
<p>It might just be trying to get all of your files and using that exception to handle the "already exists" case, but I'm not exactly sure. Lets see if it mirrors correctly when it's done.</p>
<p>chad</p>
<blockquote>
<p>Thanks,<br />Duane</p>
<p>On 11/10/2010 4:43 PM, Chad Berkley wrote:</p>
<blockquote>
<p>Check your webapps/knb/WEB-INF/log4j.properties file. It should show<br />you where the output for replication is going. I've attached the<br />log4j.properties file from knb for reference. By default, all logs<br />will be sent to catalina.out in tomcat/logs. If you configure the<br />replication log, it will go to whatever file you configure<br />(/var/metacat/logs/metacatreplication.log in the attached file).</p>
<p>chad</p>
</blockquote></blockquote> Bug #5224 (New): Scheduler part of SanParks skin needs to be updatedhttps://projects.ecoinformatics.org/ecoinfo/issues/52242010-10-23T01:24:47ZJing Taotao@nceas.ucsb.edu
<p>Workflow run engine and scheduler service were updated recently. The main change is add more parameteres, such as the user can specify workflow run engine and destination repository name.</p>
<p>So the scheduler part in sanParks need to be updated as well.</p> Bug #5199 (New): OAI-PMH: Improve memory management of data provider catalog metadatahttps://projects.ecoinformatics.org/ecoinfo/issues/51992010-10-11T16:16:44ZDuane Costadcosta@lternet.edu
<p>On October 6, 2010, Marco Fahmi wrote:</p>
<p>What scares me about this code is that it stores the whole metadata catalog<br />in memory (static member fields docTypeMap and dateMap) at class-load time,<br />which is on the first call to the OAIHandler servlet.</p>
<p>So that check for the refreshDate is checking whether the entire catalog<br />needs to be reloaded into memory! I can't see that scaling to any extent.</p>
<hr />
<p>On October 11, 2010, Duane Costa and Mark Servilla wrote in a reply to Marco Fahmi:</p>
<p>With regard to the memory management issue -- the OAI-PMH data provider code has been tested on a repository of 375 EML documents with no apparent problem regarding memory. We will soon be applying the code to a much larger repository (~10,000 documents) so we'll have a better sense of whether the current implementation scales. One distinction we'd like to make is that when you state that "the entire catalog" is loaded into memory, please note that only the documents' identifiers, their EML versions, and their revision dates are loaded into memory; the full contents of the documents themselves are not stored in memory but are instead retrieved from the Metacat database only as needed. In any case, we agree that it's likely that a better memory management solution should ultimately be implemented for storing the subset of catalog metadata that is currently held in memory.</p> Bug #5198 (New): OAI-PMH: Data provider may fail to trigger reharvest of documentshttps://projects.ecoinformatics.org/ecoinfo/issues/51982010-10-11T16:11:47ZDuane Costadcosta@lternet.edu
<p>On Oct 6, 2010, Marco Fahmi (<a class="email" href="mailto:hani.fahmi@qut.edu.au">hani.fahmi@qut.edu.au</a>) wrote:</p>
<p>From my tests I found out that the metacat OAI PMH Provider stores something<br />called "refreshDate" which is the last time the xml is generated and<br />compares it to maxDateUpdated (date that the data package records table had<br />last been modified). If the maxDateUpdated is newer than the refreshDate,<br />OAI-PMH XML will be refreshed. I'm sure you can see the problem with this<br />immediately but I'll just elaborate for our documentations about morpho and<br />metacat.</p>
<p>Sample Scenario:</p>
<p>a. 1st October 10 AM - Harvester/User accesses OAI-PMH Provider URL and an<br />XML is returned for the first time, refreshDate is set at 1st October<br />b. 1st October 11 AM - Researcher delete/update/add a data package through<br />Morpho to metacat, and the maxDateUpdated = '1st October'<br />c. 1st October 12 AM - Harvester/user accesses the provider URL again and<br />refreshDate is compared with maxDateUpdated. There is no indication of time,<br />so there will be no refreshing of the OAI-PMH XML since the date is the<br />same.<br />d. 2nd October 12 AM - Harvester/user accesses the provider URL again and<br />refreshDate is compared with maxDateUpdated. There is still no difference<br />between the dates and the bug is not resolved until someone decided to<br />update on this day or future days or tomcat is refreshed.</p>
<p>Conclusion: Any updates on the same day with the refreshDate will not be<br />reflected and may return the NullPointerException error in the case of data<br />package deletion until a data package is modified/added"</p>
<p>"If you change the String comparison in MetacatCatalog.shouldRefeshCatalog()<br />to be:</p>
<pre><code>else if (maxDateUpdated.compareTo(refreshDate) >= 0) {</code></pre>
<p>Then you will get fresh data when the refreshDate is equal to<br />maxDateUpdated. Not an ideal fix, but should get you past that bug.</p>
<p>--</p>
<p>On October 11, 2010, Duane Costa and Mark Servilla wrote in a reply to Marco Fahmi:</p>
<p>The bug relating to the sample scenario you outline is a known limitation of the OAI-PMH data provider code. It originates from the fact that Metacat stores the 'date_created' and 'date_updated' fields as type 'date', rather than type 'timestamp', in the database table that stores metadata about EML documents. This makes it difficult for the data provider to know when to re-harvest based on anything more fine-grained than the current date. We're not certain that the change you suggest would actually resolve the issue: we think it would re-load the data catalog into memory, but it wouldn't necessarily trigger the re-harvest of a document. We agree that this is a significant limitation of the data provider code and we think further analysis will be needed before a good solution can be implemented.</p> Bug #5176 (Resolved): Configuration to disable writing/updating/deleting function on metacat, but...https://projects.ecoinformatics.org/ecoinfo/issues/51762010-09-15T22:01:39ZJing Taotao@nceas.ucsb.edu
<p>This scenario happened couple times:</p>
<p>An administrator tried to use a new machine to replace the old one. Eventually the new one will use the DNS name of the old one and will have all documents/data from the old one.</p>
<p>The administrator always dumps the old db to a file and run this file on the new machine. For a big metacat server, the transformation may take a while. So administrator want users still can read/query documents/data from the old hardware, but they can't write/update documents/data to the old one. This can make sure that the old metacat and new metacat have the identical documents/data.</p>
<p>We may add a new property on the metacat.properies file, see disableWriting. Its default value is false. When the property is set to be true, users can read/query documents, but can't write/update/delete documents (include replication).</p> Bug #4044 (New): start time replication command doesn't work on replControl.html of dev skinhttps://projects.ecoinformatics.org/ecoinfo/issues/40442009-04-30T22:41:08ZJing Taotao@nceas.ucsb.edu
<p>Today, Chris Jones reported that the timed replication doesn't work between knb and psico metacats. I checked the metacat.properties and found the timed replication is set as false. So I logined as user tao, who is the knb metacat administrator, and submitted the request to turn on the timed replication. However, I checked the metacat.properties again and found the replication.timedreplication is still false.</p> Bug #3403 (Resolved): xmlns without a : is serialized incorrectlyhttps://projects.ecoinformatics.org/ecoinfo/issues/34032008-06-19T18:54:10ZChad Berkleyberkley@nceas.ucsb.edu
<p>if you upload a document with an xmlns=xxx instead of xmlns:y=xxx to metacat, it uploads correctly, but when you read the document, metacat erroneously puts a ":" into the xmlns, which makes the document invalid.</p> Bug #3397 (Resolved): metacat needs server side sort mechanismhttps://projects.ecoinformatics.org/ecoinfo/issues/33972008-06-16T23:38:50ZChad Berkleyberkley@nceas.ucsb.edu
<p>in the past, we've been using xslt to sort search results going to a browser or client. This works fine until you use paged search results when you need the results sorted before the paging takes place. we need to add functionality to give the server an attribute to sort on before it pages the results. this should probably work similar to the returnfields mechanism (or with the returnfields mechanism).</p> Bug #2418 (Resolved): New docid functionshttps://projects.ecoinformatics.org/ecoinfo/issues/24182006-04-14T17:59:58ZChad Berkleyberkley@nceas.ucsb.edu
<p>Users need to be able to ask metacat what the latest revision number for a document is. We currently have the getLastId function which returns the last used id number for a scope. We also need to be able to find the last used revision for a scope and an object number. For instance, if I sent in doc.1.1 and the last id used was doc.1.3, the function should return 4.</p>
<p>Both of these functions need to be exposed via the ecogrid-identifier EcoGrid interface.</p> Bug #1127 (Resolved): marine web site enhancementshttps://projects.ecoinformatics.org/ecoinfo/issues/11272003-08-06T21:19:25ZChad Berkleyberkley@nceas.ucsb.edu
<p>The following list was compiled when matt was at nceas in july 2003:</p>
<p>-make the login more friendly<br />-say "repository" instead of "registry" <br />-fix bullet alignment on index page<br />-main search page<br /> -move icon legend to a link<br /> -change "data search and processing interface" to "search and summarize data" <br /> -move radio buttons to the right of the text box<br /> -add submit button to search field<br /> -change the wording of the 2nd paragraph<br /> -"to get a graphical or statistical summary of a dataset either click on a<br />summary statistics icon or select a dataset and click the 'process' button" <br /> -"summary statistics" should be changed to "quick stats" <br /> -put link to the key of the icons in the summary header<br /> -summary icons<br /> -scatterplot<br /> -univariate<br /> -north/south plot<br /> -histogram<br /> -change "data search" to "new data search" <br /> -chris will redo the icons.</p> Bug #309 (Resolved): metacat does not follow LDAP referralshttps://projects.ecoinformatics.org/ecoinfo/issues/3092001-10-26T00:45:28ZMatt Jonesjones@nceas.ucsb.edu
<p>Metacat can't authenticate against the LTER ldap db because it doesn't need to<br />follow referrals properly. It was never implemented this way. Need to modify<br />the AuthLdap class to do so. This can be done by following the examples for<br />JNDI at ther following url:</p>
<p><a class="external" href="http://java.sun.com/products/jndi/tutorial/ldap/referral/follow.html">http://java.sun.com/products/jndi/tutorial/ldap/referral/follow.html</a></p> Bug #298 (Resolved): Problem with Export functionhttps://projects.ecoinformatics.org/ecoinfo/issues/2982001-10-16T18:29:20ZDan Higginshiggins@nceas.ucsb.edu
<p>If a dataset which includes a data document is inserted into production Metacat<br />context and then 'Exported' the export of the data document appears to work<br />(because there is also a local copy?) However, if local dataset is deleted and<br />then one 'Exports' from Metacat, the data document contains an error message<br />rather than the data!</p>
<p>It is currently not clear whether this is a Morpho or a Metacat problem. Thus,<br />this error has been entered to both.</p> Bug #271 (Resolved): production metacat installhttps://projects.ecoinformatics.org/ecoinfo/issues/2712001-08-31T17:06:01ZMatt Jonesjones@nceas.ucsb.edu
<p>Need to take the current release of metacat and install a 'production' version<br />on ecoinfo so that it is completely distinct from our development versions<br />(located on dev). This production version should be integrated with a<br />production version at LTERnet and should replicate all info between the two systems.</p>