Metacat: Issueshttps://projects.ecoinformatics.org/ecoinfo/https://projects.ecoinformatics.org/ecoinfo/ecoinfo/favicon.ico?14691340362006-06-21T18:01:03ZEcoinformatics Redmine
Redmine Bug #2467 (Resolved): IE auto-stretch does not display correctly with eml-2.0.0 data packageshttps://projects.ecoinformatics.org/ecoinfo/issues/24672006-06-21T18:01:03ZCallie Bowdishbowdish@nceas.ucsb.edu
<p>Data Packages created with eml.2.0.0 do not display correcly when using the ESA, KNB, NRS, OBFS and NCEAS skins on Internet Explorer.</p>
<p>knb: <a class="external" href="http://knb.ecoinformatics.org/knb/metacat?action=read&qformat=knb&docid=nceas.146">http://knb.ecoinformatics.org/knb/metacat?action=read&qformat=knb&docid=nceas.146</a></p>
<p>nceas: <a class="external" href="http://knb.ecoinformatics.org/knb/metacat?action=read&qformat=nceas&docid=nceas.146">http://knb.ecoinformatics.org/knb/metacat?action=read&qformat=nceas&docid=nceas.146</a></p>
<p>Data packages that contain this in the eml file have problems transforming correctly. <eml:eml xmlns:eml="eml://ecoinformatics.org/eml-2.0.0" xmlns:stmml="http://www.xml-cml.org/schema/stmml" xmlns:ds="eml://ecoinformatics.org/dataset-2.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" packageId="nceas.146.5" scope="system" system="knb" xsi:schemaLocation="eml://ecoinformatics.org/eml-2.0.0 eml.xsd"></p>
<p>Data Package nceas.146 is the first one in the NCEAS list of data packages when Browse existing NCEAS data sets is chosen.</p> Bug #2466 (Resolved): knb skin does not auto-stretch correctly on IEhttps://projects.ecoinformatics.org/ecoinfo/issues/24662006-06-21T17:33:46ZCallie Bowdishbowdish@nceas.ucsb.edu
<p>Internet Explorer does not auto-stretch correctly using the knb skin.</p>
<p><a class="external" href="http://knb.ecoinformatics.org/knb/metacat?action=read&qformat=knb&docid=nceas.289.1">http://knb.ecoinformatics.org/knb/metacat?action=read&qformat=knb&docid=nceas.289.1</a> or nceas.289.2 do not display correctly when using the knb skin. The NCEAS skin auto-stretch works with nceas skin. <a class="external" href="http://knb.ecoinformatics.org/knb/metacat?action=read&qformat=nceas&docid=nceas.289.2">http://knb.ecoinformatics.org/knb/metacat?action=read&qformat=nceas&docid=nceas.289.2</a></p>
<p>Note that this document has large paragraphs.</p> Bug #2312 (Resolved): finalize header graphic for ESA registryhttps://projects.ecoinformatics.org/ecoinfo/issues/23122005-12-08T18:43:06ZMatt Jonesjones@nceas.ucsb.edu
<p>The header is almost complete except for final wording changes on buttons and<br />final detail on the moderation list button. This button needs to be part of the<br />main toolbar if possible. Othersise, it probably needs to be set off more.</p>
<p>Deadline: Dec 14</p> Bug #2222 (Resolved): Bug in squery when using not-containshttps://projects.ecoinformatics.org/ecoinfo/issues/22222005-10-05T20:40:37ZSaurabh Gargsgarg@nceas.ucsb.edu
<p>For the following squery:<br /><pathquery version="1.2"><br /><querytitle>Moderator-Search</querytitle><br /><returndoctype>eml://ecoinformatics.org/eml-2.0.1</returndoctype><br /><returndoctype>eml://ecoinformatics.org/eml-2.0.0</returndoctype><br /><returndoctype>-//ecoinformatics.org//eml-dataset-2.0.0beta6//EN</returndoctype><br /><returndoctype>-//ecoinformatics.org//eml-dataset-2.0.0beta4//EN</returndoctype><br /><returndoctype>-//NCEAS//resource//EN</returndoctype><br /><returndoctype>-//NCEAS//eml-dataset//EN</returndoctype><br /><returnfield>originator/individualName/surName</returnfield><br /><returnfield>originator/individualName/givenName</returnfield><br /><returnfield>creator/individualName/surName</returnfield><br /><returnfield>creator/individualName/givenName</returnfield><br /><returnfield>originator/organizationName</returnfield><br /><returnfield>creator/organizationName</returnfield><br /><returnfield>dataset/title</returnfield><br /><returnfield>keyword</returnfield><br /><querygroup operator="INTERSECT"><br /><queryterm searchmode="not-contains" <br />casesensitive="false"><value>public</value><pathexpr>dataset/access/allow/princi<br />pal</pathexpr></queryterm><br /></querygroup></pathquery></p>
<p>The key part here is:<br /><queryterm searchmode="not-contains" <br />casesensitive="false"><value>public</value><pathexpr>dataset/access/allow/princi<br />pal</pathexpr></queryterm></p>
<p>i.e. dataset/access/allow/principal should not have 'public' value.</p>
<p>This request is resulting in the following query:<br />SELECT docid,docname,doctype,date_created, date_updated, rev FROM <br />xml_documents WHERE docid IN ((SELECT DISTINCT docid FROM xml_path_index <br />WHERE <abbr title="nodedata">UPPER</abbr> NOT LIKE '%PUBLIC%' AND path LIKE <br />'dataset/access/allow/principal' )) AND (docid IN(SELECT docid FROM <br />xml_documents WHERE lower(user_owner) ='public') OR (docid IN (SELECT <br />docid from xml_access <abbr title=" (lower(principal_name">WHERE</abbr> = 'public' AND <br />perm_type = 'allow' AND (permission='4' OR permission='7'))OR <br />(lower(principal_name) = 'public' AND perm_type = 'allow' AND <br />(permission='4' OR permission='7'))) AND subtreeid IS NULL) AND docid <br />NOT IN (SELECT docid from xml_access <abbr title=" (lower(principal_name">WHERE</abbr> = <br />'public' AND perm_type = 'deny' AND perm_order ='allowFirst' AND <br />(permission='4' OR permission='7'))OR (lower(principal_name) = 'public' <br />AND perm_type = 'deny' AND perm_order ='allowFirst' AND (permission='4' <br />OR permission='7'))) AND subtreeid IS NULL )))</p>
<p>This is not right as it still gives me back documents which have <br />'public' value in dataset/access/allow/principal. The part which is <br />wrong is:<br />SELECT DISTINCT docid FROM xml_path_index WHERE <abbr title="nodedata">UPPER</abbr> NOT LIKE <br />'%PUBLIC%' AND path LIKE 'dataset/access/allow/principal'</p>
<p>The query should be something like this: (maybe a simpler version of this)<br />SELECT DISTINCT docid from xml_path_index where docid NOT IN (Select <br />docid FROM xml_path_index WHERE <abbr title="nodedata">UPPER</abbr> LIKE '%PUBLIC%' AND path <br />LIKE 'dataset/access/allow/principal') ;</p> Bug #2218 (Resolved): Data medium info not saved in editing modehttps://projects.ecoinformatics.org/ecoinfo/issues/22182005-09-30T20:26:27ZVeronique Connollyconnolly@nceas.ucsb.edu
<p>When you go back to the data registry form (this was for the NCEAS skin) after<br />getting a failure message (by using the link in "Click here to return to the<br />form, fill in the required fields, and submit the data set description again"),<br />the option that was previously selected under "Data medium" doesn't appear,<br />i.e., the info previously submitted wasn't saved.</p> Bug #2217 (Resolved): Change output text for Usage rightshttps://projects.ecoinformatics.org/ecoinfo/issues/22172005-09-30T20:08:09ZVeronique Connollyconnolly@nceas.ucsb.edu
<p>Under the "Usage Rights" field, when you select "Obtain permission from data set<br />owner(s)" the output (what appears in the data package) is simply "permissions".<br />The output should be "Obtain permission from data set owner(s)".</p> Bug #2211 (Resolved): automatically insert pubDate on registry submissionhttps://projects.ecoinformatics.org/ecoinfo/issues/22112005-09-28T23:08:40ZCallie Bowdishbowdish@nceas.ucsb.edu
<p>We need to automatically insert pubDate on registry submission. The 'pubDate'<br />field represents the date that the resource was published. The format should be<br />represented as: CCYY, which represents a 4 digit year, or as CCYY-MM-DD, which<br />denotes the full year, month, and day. Note that month and day are optional<br />components. Formats must conform to ISO 8601</p>
<p>Updated and new version of data packages that have been submitted to the<br />registry will have a new pubDate.</p>
<p>IRQ conversation, " yeah, an updated document is really a new version with a new<br />date the (modified) resource was published <matt> so it should proabbly be<br />updated <matt> the original revision will still have the original pubDate<br /><matt> so nothing is lost"</p>
<p>Note: This will be used for citations and will be used with citation information<br />in stylesheets along with the lsid on the ESA stylesheet.</p> Bug #2207 (Resolved): Advanced Search integrationhttps://projects.ecoinformatics.org/ecoinfo/issues/22072005-09-26T22:20:40ZDuane Costadcosta@lternet.edu
<p>Over the past year, the LTER Network Office has developed an Advanced Search web<br />application that uses the Metacat client to run an advanced search on criteria<br />such as subject, author, spatial, and taxon. In its current form, the Advanced<br />Search interface exists as a separate web application, outside of the Metacat<br />code base. The goal of this task is the integrate the Advanced Search web<br />application with the Metacat code base, as described in more detail below.</p>
<hr />
<p>Proposal to Integrate Advanced Search Capability <br />with Metacat Distribution</p>
<p>The goal is to refactor the Query application so that major parts<br />of it would be integrated with Metacat, while other parts of it<br />could be customized for LTER-specific needs and maintained<br />independent of Metacat.</p>
<p>1. Query Engine</p>
<p>The back-end Query Engine can be fully integrated with the Metacat<br />code base. It contains search engine functionality that is<br />generic to Metacat and should be relatively easy to factor out of <br />the Query application. Once it is part of Metacat, it can be<br />packaged as a library that can be distributed with the Query<br />application. After it is integrated with Metacat, the Query Engine <br />code can be maintained by all Metacat developers. If logical <br />improvements or performance optimizations are made to the Query <br />Engine code by the Metacat community, the LTER Query application<br />will benefit from these improvements and optimizations because <br />it will utilize the same code that Metacat utilizes.</p>
<p>2. Advanced Search Form</p>
<p>The Advanced Search Form is implemented as a JSP. It can be <br />reimplemented to eliminate the Struts-based custom tags that it <br />currently uses. JavaScript could be added to replace the <br />Struts-generated JavaScript for client-side form validation; <br />alternatively, form validation could be moved to the server side.</p>
<p>A LTER-customized version of the Advanced Search Form could be <br />maintained in the Query Application. It would be nearly identical<br />to the Metacat version, but it would contain additional input fields,<br />such as a drop-down list that allows the user to restrict the search<br />results to a particular LTER site.</p>
<p>If possible, a mechanism would be worked out to minimize the<br />duplication of effort that would be required to maintain both the <br />Metacat form and the custom LTER form and to keep them consistent.</p>
<p>3. Login Page, Simple Search Page, and Browse Page</p>
<p>These pages in the Query Application would not be integrated with <br />Metacat, since equivalent functionality already exists in the <br />default Metacat skin. These pages would continue to be part of the <br />Query application. Since the Query application uses Struts to manage <br />the functionality of these pages, we could continue to use Struts <br />in the Query application, though we would not need Struts in Metacat.</p>
<p>We may be able to utilize the Metacat skin to provide this <br />functionality for the Query application. Eventually, we may be<br />able to fully migrate all of the functionality of the Query application<br />to Metacat, though we would need to provide a way to extend the<br />Metacat skin with LTER customizations.</p>
<p>The current browse capability of the Query Application is intended<br />to be replaced by a browsable hierarchy of terms based on the work<br />that is being initiated in the LTER working groups for Ontologies<br />and Controlled Vocabularies. Since this work would be a valuable <br />contribution to Metacat as well, it would be useful to integrate <br />these new browse capabilities in the Metacat skin rather than <br />restrict them to just the Query application.</p>
<p>Work Estimates:</p>
<p>Task Effort (Weeks)<br />---- --------------</p>
<p>Phase One:</p>
<p>These tasks would fulfill Matt's requirements<br />to integrate the Advanced Search capability with<br />Metacat, while retaining the Query application as a<br />LTER custom application that shares some of its<br />software components with Metacat.</p>
<pre><code>Refactor Query Engine. Add code 1<br /> to Metacat. Refactor Query Application<br /> to use Metacat library.</code></pre>
<pre><code>Refactor Advanced Search Form to eliminate 1<br /> Struts custom tags.</code></pre>
<pre><code>Implement generic Advanced Search Form and 1<br /> integrate it with Metacat Skin. Maintain custom<br /> LTER form in the Query Application as an<br /> extension to the Metacat form.</code></pre>
<p>Phase Two:</p>
<p>This optional phase would deprecate the Query application<br />as a separate entity, eliminating the duplication of <br />effort needed to keep its advanced search functionality <br />consistent with Metacat's. The time estimates for these<br />tasks should be adjusted after Phase One is completed,<br />since we will have a better understanding of the effort<br />required at that point.</p>
<p>Fully migrate the Query Application to Metacat, 2<br />allowing for LTER customizations within Metacat.<br />Other organizations could use the LTER customizations<br />in Metacat as a model for their own customizations.</p>
<hr />
<p>On 9/6/2005, Mark Servilla wrote:</p>
<p>Hi Matt,</p>
<p>Attached, please find a brief statement of work proposed by Duane Costa<br />regarding the Advanced Query Interface for metacat. We consider the<br />re-unification of the two applications to be a high-priority to the LTER, NCEAS,<br />and the eco-community, and will begin the planning/work effort immediately. At<br />your earliest convenience, please review the SOW and let us know if this is<br />acceptable and/or if you have any questions/comments.</p>
<p>Sincerely,<br />Mark</p>
<hr />
<p>On 9/6/2005, Matt Jones wrote:</p>
<p>Hi Mark,</p>
<p>Thanks. In general this statement of work looks great -- no real modifications<br />on my part. It will be a great time-saver for many people who want an advanced<br />search in their metacat installation, so I appreciate it.</p>
<p>A couple of brief comments for context:<br />1) We need to revise our login infrastructure. Right now metacat uses cookies<br />for session state. That has worked well, and is pretty robust.<br /> We also have a more recent javascript login for managing the cookies on the KNB<br />and default skins -- this is incomplete and therefore broken, although it is<br />what is used on the main KNB page. The problem is that the session information<br />does not propagate across pages as one navigates through the app, especially in<br />the EML pages. We know why, but haven't fixed it. It would be a good time to do<br />so, so let us know if you have particular needs in your part of the login<br />infrastructure.<br />2) We've wanted an effective browsable hierarchy of terms for KNB datasets, but<br />there just is't consensus on a controlled vocabulary. The one on the KNB skin is<br />one I made up with feedback from Mark S and a few others. If you get consensus<br />with LTER, we'd probably want to switch the KNB in general over to using it and<br />creating a browsing interface that allows one to navigate it. So that might be<br />another area for shared work.<br />3) I think the advanced search page needs a map to draw a box for spatial<br />searches. We've found users simply don't know the lat/lon for their area of<br />interest.<br />4) We are working on a spatial option for metacat for Kruger that allows the<br />locations of data to be plotted against other GIS layers in a map and searched<br />using spatial queries. This is in prototype now, but will be released hopefully<br />with the next metacat release. Just a FYI in case a similar request has come<br />your way.<br />5) The new SEEK web developer located at LNO (in process of hiring replacement<br />for Tekell) will be working on a portal for access to EcoGrid data (both EML and<br />DarwinCore). I hope we will be able to adapt your client interface so that it<br />can be applied to the web services backend in EcoGrid, as I think the<br />requirements are essentially the same.</p>
<p>Thanks. Looking forward to working on this with you.</p>
<p>Matt</p>
<hr />
<p>On 9/23/05, Mark Servilla wrote:</p>
<p>Hi Matt,</p>
<p>Sorry for the delay in getting back to this matter. After discussing your<br />comments with Duane, the only item that impacts our involvement directly is<br />number 1 - the session management. It will be critical to work this problem to<br />completion. With respect to number 3, the map UI, we would appreciate any<br />suggestions in this area; short of installing a full-up ArcIMS/MapServer type of<br />application. Is there a simple javascript version for such a map? A first<br />version of this would require only the bounding lat/lon for the query. Thanks!</p>
<p>Sincerely,<br />Mark</p>
<hr />
<p>On 9/23/05, Matt Jones wrote:</p>
<p>Mark,</p>
<p>Thanks for the followthrough. I agree that (1) needs to be worked out, and I'd<br />like to see the GT4 GSI certificate stuff that NCSA did before we decide on a<br />solution. We should probably take an approach that accomodates that if we're<br />delving into the auth infrastructure. <br />Regarding (3), I think there are several open-source geospatial libs for doing<br />this (we use one of them in Morpho). Maybe Duane could talk to John Harris (who<br />is working on this stuff in metacat now) and Dan Higgins (who did the geospatial<br />map in morpho) and come up with a proposal? My impression is that the java lib<br />Dan used was pretty effective and easy to plug into morpho, and I think others<br />have come about. GEON has one in their portal search client, so we might be<br />able to borrow code or ideas from them. We definitely don't want an IMS for<br />this part -- our needs are much simpler.</p>
<p>Matt</p> Bug #2172 (Resolved): & is replaced by &amp;https://projects.ecoinformatics.org/ecoinfo/issues/21722005-08-16T22:31:24ZSaurabh Gargsgarg@nceas.ucsb.edu
<p>Need to check for &XXX; in MetaCatUtil.normalize() function. This problem was <br />reported by Sven. Below are the emails describing the bug.</p>
<p>----------------------------------------------------<br />Saurabh Garg wrote:</p>
<blockquote>
<p>Ah ok. I looked into the code and the patch from Johnoel only takes care <br />of cases which has &#xxxx; <br />It doesnt take care of & situation.</p>
<p>I did a lot of testing to check the normalize() function before Metacat <br />1.5 release - I am not sure why I didnt think about this case. I will <br />add a bug and fix it for the next release.</p>
<p>Sid</p>
<p>Duane Costa wrote:</p>
<blockquote>
<p>Sid,</p>
<p>To see &amp; you have to view the page source in your browser. The browser</p>
</blockquote></blockquote>
<p>itself will not show it.</p>
<blockquote><blockquote>
<p>Duane</p>
<blockquote>
<p>-----Original Message-----<br />From: Saurabh Garg [mailto:<a class="email" href="mailto:sgarg@nceas.ucsb.edu">sgarg@nceas.ucsb.edu</a>] <br />Sent: Tuesday, August 16, 2005 3:57 PM<br />To: Matt Jones<br />Cc: Duane Costa; 'Sven Bohm'; <a class="email" href="mailto:metacat-dev@ecoinformatics.org">metacat-dev@ecoinformatics.org</a><br />Subject: Re: [metacat-dev] & and & in metacat</p>
<p>Hi,</p>
<p>Metacat 1.5 does the escaping for you even if you have <br />already done it. <br />The patch submitted by Johnoel fixes this and will be part of <br />the next Metacat release.</p>
<p>However the normalize function is called during document insertion. <br />Hence the document inserted already has &amp; in it. I <br />think Duane's first idea might be better for the time being.</p>
<p>Though I didnt see &amp; in the document which Duane sent. <br />Maybe I missed something or maybe it is already fixed.</p>
<p>Sid</p>
<p>Matt Jones wrote:</p>
<blockquote>
<p>I think this should be fixed in Metacat's normalize()</p>
</blockquote>
<p>method. Sid --</p>
<blockquote>
<p>will you file a bug please? Thanks.</p>
<p>Matt</p>
<p>Duane Costa wrote:</p>
<blockquote>
<p>Sven,</p>
<p>Yes, it looks like the ampersand in the online URL does get</p>
</blockquote></blockquote>
<p>escaped by</p>
<blockquote><blockquote>
<p>Metacat. I found a Metacat method, MetacatUtil.normalize(),</p>
</blockquote></blockquote>
<p>that does</p>
<blockquote><blockquote>
<p>this replacement, though I'm not sure at what point in the process <br />it's happending: during document insertion or during</p>
</blockquote></blockquote>
<p>document retrieval. I think the assumption Metacat is making <br />is that your document will not have the escaped characters in <br />the URL already, so it happily does the service of escaping <br />them for you.</p>
<blockquote><blockquote>
<p>There are two possible solutions I can think of (though, of</p>
</blockquote></blockquote>
<p>course, there are probably better ones I haven't thought of):</p>
<blockquote><blockquote>
<p>1. Remove the escaped ampersands from the EML and let</p>
</blockquote></blockquote>
<p>Metacat do the</p>
<blockquote><blockquote>
<p>escaping for you. However, if you need to keep the</p>
</blockquote></blockquote>
<p>ampersands escaped</p>
<blockquote><blockquote>
<p>because you are using the documents for purposes other than</p>
</blockquote></blockquote>
<p>just Metacat, you might need to maintain a separate <br />stylesheet just for Metacat harvests that does not escape the <br />ampersands.</p>
<blockquote><blockquote>
<p>2. MetacatUtil.normalize() might be modified to check for an <br />already-escaped ampersand before it decides to escape an</p>
</blockquote></blockquote>
<p>ampersand. I noticed that a patch had already been added to <br />check for character references:</p>
<blockquote><blockquote>
case '&': {<br />/*
<ul>
<li>patch provided by Johnoel</li>
</ul>
</blockquote></blockquote>
<p>Ancheta from U of Hawaii</p>
<blockquote><blockquote>
<p>*/<br />// check if & is for a character reference <br />&#xnnnn;</p>
<p>I am cc:ing metacat-dev for their thoughts.</p>
<p>To see an example of a document that exhibits this behavior:<br /><a class="external" href="http://prairie.lternet.edu:8080/knb/metacat?action=read&qfor">http://prairie.lternet.edu:8080/knb/metacat?action=read&qfor</a></p>
</blockquote></blockquote>
<p>mat=xml&do</p>
<blockquote><blockquote>
<p>cid=knb-lter-kbs.1.5 Look at the <distribution><online><url> value.</p>
<p>Duane</p>
<blockquote>
<p>-----Original Message-----<br />From: Sven Bohm [mailto:<a class="email" href="mailto:bohms@msu.edu">bohms@msu.edu</a>]<br />Sent: Tuesday, August 16, 2005 1:03 PM<br />To: duane Costa<br />Subject: & and & in metacat</p>
</blockquote>
<p>Hi Duane</p>
<p>In my metacat stylesheet<br /><a class="external" href="http://lter.kbs.msu.edu/Data/LTER_Metadata.jsp?Dataset=KBS002x%x">http://lter.kbs.msu.edu/Data/LTER_Metadata.jsp?Dataset=KBS002x%x</a>%<br />stylesheet=LTER_Metadata-eml.xsl</p>
<p>I set the data url to something like this:<br /><a class="external" href="http://lter.kbs.msu.edu/Data/taple.jsp?Product=KBS002-006&amp;">http://lter.kbs.msu.edu/Data/taple.jsp?Product=KBS002-006&amp;</a></p>
<blockquote>
<p>format=csv&stylesheet=data.xsl</p>
</blockquote>
<p>It comes to this in the browser:<br /><a class="external" href="http://lter.kbs.msu.edu/Data/table.jsp?Product=KBS002-006&form">http://lter.kbs.msu.edu/Data/table.jsp?Product=KBS002-006&form</a><br />at=csv&stylesheet=data.xsl</p>
<p>Yet in the online distribution line on the lter metacat it gives me <br />this line (notice the double amp;'s:<br /><a class="external" href="http://lter.kbs.msu.edu/Data/table.jsp?Product=KBS002-006&amp;">http://lter.kbs.msu.edu/Data/table.jsp?Product=KBS002-006&amp;</a></p>
<blockquote>
<p>amp;format=csv&amp;stylesheet=data.xsl</p>
</blockquote>
<p>It looks like metacat bindly escapes &'s even if they are</p>
</blockquote></blockquote>
<p>part of an</p>
<blockquote><blockquote>
<p>escape sequence. Any thoughts?<br />--<br />Sven Bohm <del>.- ..</del>. ---.. .-<br />KBS-LTER Information Manager <a class="external" href="http://www.msu.edu/~bohms/key">http://www.msu.edu/~bohms/key</a></p>
</blockquote>
<blockquote>
<p>_<em><i></em></i>__<em>_</em>___________________________________<br />Metacat-dev mailing list<br /><a class="email" href="mailto:Metacat-dev@ecoinformatics.org">Metacat-dev@ecoinformatics.org</a><br /><a class="external" href="http://mercury.nceas.ucsb.edu/ecoinformatics/mailman/listinf">http://mercury.nceas.ucsb.edu/ecoinformatics/mailman/listinf</a></p>
</blockquote></blockquote>
<p>o/metacat-</p>
<blockquote><blockquote>
<p>dev</p>
</blockquote>
<p>_<em><i></em></i>__<em>_</em>___________________________________<br />Metacat-dev mailing list<br /><a class="email" href="mailto:Metacat-dev@ecoinformatics.org">Metacat-dev@ecoinformatics.org</a><br /><a class="external" href="http://mercury.nceas.ucsb.edu/ecoinformatics/mailman/listinfo">http://mercury.nceas.ucsb.edu/ecoinformatics/mailman/listinfo</a></p>
</blockquote>
<p>/metacat-d</p>
<blockquote>
<p>ev</p>
</blockquote>
</blockquote>
<p>_<em><i></em></i>__<em>_</em>___________________________________<br />Metacat-dev mailing list<br /><a class="email" href="mailto:Metacat-dev@ecoinformatics.org">Metacat-dev@ecoinformatics.org</a><br /><a class="external" href="http://mercury.nceas.ucsb.edu/ecoinformatics/mailman/listinfo/metacat-dev">http://mercury.nceas.ucsb.edu/ecoinformatics/mailman/listinfo/metacat-dev</a></p>
</blockquote>
<p>_<em><i></em></i>__<em>_</em>___________________________________<br />Metacat-dev mailing list<br /><a class="email" href="mailto:Metacat-dev@ecoinformatics.org">Metacat-dev@ecoinformatics.org</a><br /><a class="external" href="http://mercury.nceas.ucsb.edu/ecoinformatics/mailman/listinfo/metacat-dev">http://mercury.nceas.ucsb.edu/ecoinformatics/mailman/listinfo/metacat-dev</a></p>
</blockquote> Bug #2154 (Resolved): Metacat Performace: Configurable path condition indiceshttps://projects.ecoinformatics.org/ecoinfo/issues/21542005-07-14T01:16:31ZSaurabh Gargsgarg@nceas.ucsb.edu
<p>From Matt's email...</p>
<p>Configurable path condition indices -- this would allow admins to<br />configure specific XML paths in metacat to be replicated to their own<br />table in the DB (which would be dynamically created) to make searching<br />fast for those paths. This is effective because clients (e.g., Morpho,<br />web) tend to use only a limited set of where clause restrictions (e.g.,<br />on title, surname, keyword, ...). In a typical EML document with 5000<br />nodes, our current search has to hit many records when searching for a<br />path like "dataset/title". Given that each document has only one or a<br />few titles, a dedicated table for title that is indexed might only have<br />3500 records in the "dataset/title" table (compared to ~106 in<br />xml_nodes), which would be a significantly faster query. Also, by<br />creating temporary tables for all commonly searched fields, we would<br />possibly avoid searchcing the xml_nodes table altogether, and therefore<br />avoid the whole recursive query issue. See Sid's point <a class="issue tracker-1 status-3 priority-2 priority-default closed" title="Bug: attribute editing for EMPTY elements doesn't work (Resolved)" href="https://projects.ecoinformatics.org/ecoinfo/issues/2">#2</a> below for<br />more details. This has a lot of potential for speeding up structured<br />queries (e.g., spatial), but xml_nodes would still be used for<br />unstructured (anyfield) queries.</p> Bug #2152 (Resolved): Metacat Performace: Reduce size of xml_nodes tablehttps://projects.ecoinformatics.org/ecoinfo/issues/21522005-07-14T01:11:27ZSaurabh Gargsgarg@nceas.ucsb.edu
<p>xml_nodes is huge (6.8x106 records), and grows with every insert and<br />update by many thousands of records. By design we don't want to search<br />deleted docs nor old revisions anyway, so these could and should be<br />moved to their own table (xml_nodes_revisions) that parallels the<br />xml_revisions table. This would mean a massive reduction in the<br />xml_nodes table size (xml_nodes has 6,885,702 records now, would reduce<br />to 2,861,435 records) and a much slower growth rate for that<br />table. This alone could speed up query time substantially.</p> Bug #2091 (Resolved): Don't determine document namespace by "schemaLocation" attributehttps://projects.ecoinformatics.org/ecoinfo/issues/20912005-05-25T23:08:03ZJing Taotao@nceas.ucsb.edu
<p>In MetacatServlet class, before inserting document into db, we need to <br />determine which parser should be initialized. We get the namespace <br />information from the attribute "schemaLocation" and if namespace is <br />eml200 document the eml parser200 will be initialized. If it is not <br />eml200 or eml201, the generate parser will be initialized(which wouldn't <br />handle writing access rule). Since the document doesn't has <br />"schemaLocation" attribute, the generate parser will be used and there <br />won't have any access rules will be created.</p>
<p>Using schemaLocation to determine the namespace is defintiely<br />a bug. There is never any requirement in XML Schema that someone<br />provides a schemaLocation -- its only used to suggest a location as a<br />hint so the .xsd file can be tracked down if the parser doesn't have a<br />cached copy already. The prefix of the root element should tell you the<br />xmlns, which is then present in the xmlns:prefix element in the header.<br />There are routines in the XML parsers for finding this stuff out during<br />the parse.</p> Bug #2060 (Resolved): Documents not indexed because of error generated during indexing of documentshttps://projects.ecoinformatics.org/ecoinfo/issues/20602005-04-05T00:20:51ZSaurabh Gargsgarg@nceas.ucsb.edu
<p>Sometimes given below errors have been seen in the log files. These are <br />generated because of lack of coordination between the thread updating the <br />xml_nodes and the thread updating the xml_index table. The result is that <br />sometimes you have documents which are not indexed.</p>
<p>MetaCat: Error in DBSAXHandler.checkDocumentTable Couldn't find the docid for <br />index build in reseaonable time!<br />MetaCat: SQL Exception while inserting path index in DocumentImpl.buildIndex <br />for document test.200594171957<br />MetaCat: ORA-02291: integrity constraint (SGARG.XML_INDEX_DOCID_FK) violated - <br />parent key not found</p>
<p>java.sql.SQLException: ORA-02291: integrity constraint <br />(SGARG.XML_INDEX_DOCID_FK) violated - parent key not found</p>
<pre><code>at oracle.jdbc.driver.DatabaseError.throwSqlException<br />(DatabaseError.java:125)<br /> at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:305)<br /> at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:272)<br /> at oracle.jdbc.driver.T4C8Oall.receive(T4C8Oall.java:623)<br /> at oracle.jdbc.driver.T4CPreparedStatement.doOall8<br />(T4CPreparedStatement.java:181)<br /> at oracle.jdbc.driver.T4CPreparedStatement.execute_for_rows<br />(T4CPreparedStatement.java:543)<br /> at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout<br />(OracleStatement.java:1028)<br /> at oracle.jdbc.driver.OraclePreparedStatement.executeInternal<br />(OraclePreparedStatement.java:2888)<br /> at oracle.jdbc.driver.OraclePreparedStatement.executeUpdate<br />(OraclePreparedStatement.java:2960)<br /> at edu.ucsb.nceas.metacat.DocumentImpl.updateNodeIndex<br />(DocumentImpl.java:1345)<br /> at edu.ucsb.nceas.metacat.DocumentImpl.buildIndex<br />(DocumentImpl.java:1214)<br /> at edu.ucsb.nceas.metacat.DBSAXHandler.run(DBSAXHandler.java:444)<br /> at java.lang.Thread.run(Thread.java:534)</code></pre>
<p>MetaCat: SQL Exception while inserting path index in DocumentImpl.buildIndex <br />for document test.20059417224<br />MetaCat: ORA-00001: unique constraint (SGARG.XML_INDEX_PK) violated</p>
<p>java.sql.SQLException: ORA-00001: unique constraint (SGARG.XML_INDEX_PK) <br />violated</p>
<pre><code>at oracle.jdbc.driver.DatabaseError.throwSqlException<br />(DatabaseError.java:125)<br /> at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:305)<br /> at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:272)<br /> at oracle.jdbc.driver.T4C8Oall.receive(T4C8Oall.java:623)<br /> at oracle.jdbc.driver.T4CPreparedStatement.doOall8<br />(T4CPreparedStatement.java:181)<br /> at oracle.jdbc.driver.T4CPreparedStatement.execute_for_rows<br />(T4CPreparedStatement.java:543)<br /> at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout<br />(OracleStatement.java:1028)<br /> at oracle.jdbc.driver.OraclePreparedStatement.executeInternal<br />(OraclePreparedStatement.java:2888)<br /> at oracle.jdbc.driver.OraclePreparedStatement.executeUpdate<br />(OraclePreparedStatement.java:2960)<br /> at edu.ucsb.nceas.metacat.DocumentImpl.updateNodeIndex<br />(DocumentImpl.java:1345)<br /> at edu.ucsb.nceas.metacat.DocumentImpl.buildIndex<br />(DocumentImpl.java:1214)<br /> at edu.ucsb.nceas.metacat.DBSAXHandler.run(DBSAXHandler.java:444)<br /> at java.lang.Thread.run(Thread.java:534)</code></pre> Bug #1302 (Resolved): Changes in links to the guide and updating the guidehttps://projects.ecoinformatics.org/ecoinfo/issues/13022004-02-02T23:13:45ZSaurabh Gargsgarg@nceas.ucsb.edu
<p>-> The linsk to the guide need to be changed so that it is easier to understand <br />that they are links. Or a help icon can be added next to the heading and make <br />the icon a link instead of making the heading a link</p>
<p>-> Also the guide is out of date and needs to be updated for eml2.</p> Bug #1236 (Resolved): enhancement to metacat: overall access controlshttps://projects.ecoinformatics.org/ecoinfo/issues/12362003-12-11T00:54:26ZDavid Kaplandmkaplan@ucdavis.edu
<p>I am submitting this bug to archive discussions we have been having regarding<br />access priviledges to the entire metacat system. Basically, I would like to (1)<br />allow write access and the ability to create new EML documents to only a subset<br />of the authenticated users, i.e. to an LDAP group/user and (2) provide the name<br />of a LDAP user or group that always has full priviledges on all documents, i.e.<br />a sysadmin type priviledge.</p>
<p>I envision a special access list that would be stored in the database that<br />contained access priviledges for the entire metacat system. This document could<br />set all the same type of priviledges that you can set in a normal ACL document<br />associated with a package, but they would apply to all documents before applying<br />the documents own ACL list. The owner of this special ACL list would be a<br />user/group specified in the config file and this would essentially create an<br />admin user/group.</p>
<p>I haven't looked at the code, but this might not be that difficult to implement<br />considering that you already have to do the same sort of access control on<br />updates with the package ACL document. You could just consider it a<br />concatenation of priviledges, although you might have to take care that the user<br />can never overide overall access controls in his document's ACL list. If not,<br />it could just be another level of checking before taking any action.</p>