Metacat: Issueshttps://projects.ecoinformatics.org/ecoinfo/https://projects.ecoinformatics.org/ecoinfo/ecoinfo/favicon.ico?14691340362017-10-11T18:42:38ZEcoinformatics Redmine
Redmine Bug #7217 (New): Report on metadata creation date in metadata quality summarieshttps://projects.ecoinformatics.org/ecoinfo/issues/72172017-10-11T18:42:38ZPeter Slaughterslaughter@nceas.ucsb.edu
<p>Indexing fields for metadata quality reports do not include the upload date of the metadata they are reporting on. Therefor, summaries that are created, i.e. mean score for a user over time, currently show the time of the creation of the quality report, not the metadata.</p>
<p>Add the field 'mdq.metadata.timestamp' to application-context-mdq.xml to hold the metadata creation or update time.</p>
<p>Each quality suite will be responsible for making this information available in the quality report, so that MDQClient.saveRun<br />can record it.</p> Bug #7216 (New): MDQClient.saveRun doesn't obsolete existing quality documentshttps://projects.ecoinformatics.org/ecoinfo/issues/72162017-10-11T17:30:14ZPeter Slaughterslaughter@nceas.ucsb.edu
<p>MDQClient.saveRun is called to upload a newly created quality document, in response to a metadata quality document being uploaded or updated.</p>
<p>When MDQClient.saveRun is called by MNodeService.update, it does not check if a quality document has already been created<br />for the metadata document. saveRun should check if a previous quality document has been created for the metadata, and obsolete it<br />with the new quality document. This will ensure that quality statistics are accurate, as obsoleted quality reports will not be<br />included in statistical calculations, as they are essentially duplicates.</p> Bug #7212 (New): metacat-index missing metadata quality fieldshttps://projects.ecoinformatics.org/ecoinfo/issues/72122017-10-03T18:59:38ZPeter Slaughterslaughter@nceas.ucsb.edu
<p>The Spring context file ./metacat-index/src/main/resources/application-context-mdq.xml doesn't contain a bean definition for the quality check types 'congruency' or 'dataFormats',<br />although these are check types that we should record results for. There is a bean for check type 'other', but this isn't sufficient.</p> Feature #7198 (New): Format solr engine description outputhttps://projects.ecoinformatics.org/ecoinfo/issues/71982017-06-02T17:30:11ZPeter Slaughterslaughter@nceas.ucsb.edu
<p>The solr engine description information from DataONE has XSLT formatted output that includes a description of each field: <a class="external" href="https://cn.dataone.org/cn/v2/query/solr">https://cn.dataone.org/cn/v2/query/solr</a>. The corresponding metacat index output does not: <a class="external" href="https://knb.ecoinformatics.org/knb/d1/mn/v2/query/solr">https://knb.ecoinformatics.org/knb/d1/mn/v2/query/solr</a>. It would be very useful to provide users with this info to help them use/learn solr and our index.<br />I wasn't able to find the .xsl file in metacat repo or the DataONE repo, so am not sure how to include this into metacat-index</p> Bug #7181 (New): Verify completeness of unit test MetacatRdfXmlSubprocessorTesthttps://projects.ecoinformatics.org/ecoinfo/issues/71812017-04-11T23:41:43ZPeter Slaughterslaughter@nceas.ucsb.edu
<p>Verify that all prov relationships that are indexed via src/main/resources/application-context-prov-base.xml are inspected by the unit test MetacatRdfXmlSubprocessorTest.java which reads src/test/resources/rdfxml-example.xml.</p> Bug #7150 (New): Do not populate the "documents" Solr index field with a metadata object's own id...https://projects.ecoinformatics.org/ecoinfo/issues/71502016-11-03T21:35:14ZLauren Walkerwalker@nceas.ucsb.edu
<p>If a resource map has a cito:documents relationship that states that a metadata doc cito:documents itself, that should be ignored in the Solr discovery index. We have metadata documents in the Arctic Data Center that now are indexed as documenting themselves, which causes issues with the "Only results with data" filter in Metacat UI.</p>
<p>Here is an example of a metadata doc that documents itself: <a class="external" href="https://arcticdata.io/metacat/d1/mn/v2/query/solr/?q=id:%22urn:uuid:dfb9b407-086d-4270-b872-b43504d5e94a%22&fl=id,documents">https://arcticdata.io/metacat/d1/mn/v2/query/solr/?q=id:%22urn:uuid:dfb9b407-086d-4270-b872-b43504d5e94a%22&fl=id,documents</a></p> Bug #7093 (New): Metacat-index is not indexing all package members correctlyhttps://projects.ecoinformatics.org/ecoinfo/issues/70932016-08-26T17:04:55ZChris Jonescjones@nceas.ucsb.edu
<p>Shirley pointed out that for the <code>urn:uuid:09fd1d96-c6b0-442d-b1cb-e2c4a2e476b6</code> package on arcticdata.io:</p>
<p><a class="external" href="https://arcticdata.io/catalog/#view/urn:uuid:09fd1d96-c6b0-442d-b1cb-e2c4a2e476b6">https://arcticdata.io/catalog/#view/urn:uuid:09fd1d96-c6b0-442d-b1cb-e2c4a2e476b6</a></p>
<p>Notice that the resource map aggregates the following 29 objects:</p>
<pre>
$ curl -s -o - \
"https://arcticdata.io/metacat/d1/mn/v2/object/resource_map_urn:uuid:09fd1d96-c6b0-442d-b1cb-e2c4a2e476b6" | \
xmlstarlet sel -t -v "//ore:aggregates/@rdf:resource" -n | \
cut -d "/" -f7
urn%3Auuid%3A1841cbef-f024-4099-ba2b-ccf89e65d295
urn%3Auuid%3A8fe1169b-5c39-4793-bdc3-a3ced4e321ad
urn%3Auuid%3A355e25c7-9637-42e6-85ea-6d642c02a454
urn%3Auuid%3Ac9d4d530-aafb-4d3b-bef9-9eb9bf4dfba3
urn%3Auuid%3A6ce5b2bd-fcf9-4a4f-bbf4-63337ea0ad16
arctic-data.9492.1
arctic-data.9468.1
urn%3Auuid%3Af6c12e96-95c0-423f-83a0-36c3c0c6abf4
urn%3Auuid%3Ada0f723b-bb38-4437-bee5-56b0b4efcb5d
urn%3Auuid%3A58ba459e-c508-4b74-9f2e-db569cf9d737
urn%3Auuid%3A09a0749e-82dd-48d8-8be2-23a7993db65b
urn%3Auuid%3A7cda7813-1d23-4e57-b0cb-59980bd2dbde
urn%3Auuid%3A9b2d3c3c-d775-4e20-b9e6-896f2e102f0f
urn%3Auuid%3Ab9218df4-5a34-4ddc-8d35-6f7824049b89
urn%3Auuid%3Ac385222e-caf8-4ae4-b3bf-463f2fba3968
urn%3Auuid%3A9a9077ee-011c-44de-8b77-b16280998250
urn%3Auuid%3Ad933a025-8dc0-4933-91af-5b49e1e48079
urn%3Auuid%3Ae2ba7348-1c64-405b-9840-68dc8d776e5c
urn%3Auuid%3Add5925d8-a9a1-48df-9623-3b75b48d0460
urn%3Auuid%3Abb885aa0-6e31-4de4-adaa-b842eaf3a9fa
urn%3Auuid%3Ac1d2ab68-e653-488a-9942-fd1045a7aad6
arctic-data.9469.1
urn%3Auuid%3Ab70e055d-593d-4b3a-8aaf-04574e377e46
urn%3Auuid%3Aa0e2e791-0a07-4213-b53e-ed5f9a16178e
urn%3Auuid%3Aa0769215-4c17-4935-87ed-442d761a4269
urn%3Auuid%3A17c1009b-5e1f-4d2d-b040-b21a6366adda
urn%3Auuid%3A760cc12e-e263-48cd-b337-eeb2f6336702
urn%3Auuid%3A96dc7150-0e19-4647-ae77-d5dda1abedda
urn%3Auuid%3A09fd1d96-c6b0-442d-b1cb-e2c4a2e476b6
</pre>
<p>However, looking at the index, only 26 are listed with a <code>resourceMap</code> field of <code>resource_map_urn:uuid:09fd1d96-c6b0-442d-b1cb-e2c4a2e476b6</code>:</p>
<pre>
$ curl -s -o - 'https://arcticdata.io/metacat/d1/mn/v2/query/solr/q=resourceMap:"resource_map_urn:uuid:09fd1d96-c6b0-442d-b1cb-e2c4a2e476b6"&fl=id&rows=50&wt=csv'
id
urn:uuid:09fd1d96-c6b0-442d-b1cb-e2c4a2e476b6
urn:uuid:96dc7150-0e19-4647-ae77-d5dda1abedda
urn:uuid:6ce5b2bd-fcf9-4a4f-bbf4-63337ea0ad16
urn:uuid:8fe1169b-5c39-4793-bdc3-a3ced4e321ad
urn:uuid:f6c12e96-95c0-423f-83a0-36c3c0c6abf4
urn:uuid:58ba459e-c508-4b74-9f2e-db569cf9d737
urn:uuid:c9d4d530-aafb-4d3b-bef9-9eb9bf4dfba3
urn:uuid:da0f723b-bb38-4437-bee5-56b0b4efcb5d
urn:uuid:355e25c7-9637-42e6-85ea-6d642c02a454
arctic-data.9492.1
arctic-data.9468.1
urn:uuid:17c1009b-5e1f-4d2d-b040-b21a6366adda
urn:uuid:c1d2ab68-e653-488a-9942-fd1045a7aad6
urn:uuid:bb885aa0-6e31-4de4-adaa-b842eaf3a9fa
urn:uuid:b70e055d-593d-4b3a-8aaf-04574e377e46
urn:uuid:a0769215-4c17-4935-87ed-442d761a4269
arctic-data.9469.1
urn:uuid:09a0749e-82dd-48d8-8be2-23a7993db65b
urn:uuid:d933a025-8dc0-4933-91af-5b49e1e48079
urn:uuid:9b2d3c3c-d775-4e20-b9e6-896f2e102f0f
urn:uuid:e2ba7348-1c64-405b-9840-68dc8d776e5c
urn:uuid:b9218df4-5a34-4ddc-8d35-6f7824049b89
urn:uuid:7cda7813-1d23-4e57-b0cb-59980bd2dbde
urn:uuid:9a9077ee-011c-44de-8b77-b16280998250
urn:uuid:c385222e-caf8-4ae4-b3bf-463f2fba3968
urn:uuid:1841cbef-f024-4099-ba2b-ccf89e65d295
</pre>
<p>The following three data objects are missing:</p>
<pre>
urn:uuid:760cc12e-e263-48cd-b337-eeb2f6336702 2016-co2.csv
urn:uuid:dd5925d8-a9a1-48df-9623-3b75b48d0460 2016-gps.csv
urn:uuid:a0e2e791-0a07-4213-b53e-ed5f9a16178e 2016-met.csv
</pre>
<p>I've tried reindexing all of the pids associated with the package, and the package itself, with no luck. I've turned on DEBUG level logging in <code>/var/lib/tomcat7/webapps/metacat/WEB-INF/lib/classes/log4j.properties</code> to get more logging information from metacat-index into catalina.out. I restarted Tomcat to enable the logging changes.</p>
<p>We need to check to see why these members aren't being indexed correctly. It's also affecting many other packages according to the graduate interns on the data team.</p> Task #6040 (New): Metacat-index does not handle <references>https://projects.ecoinformatics.org/ecoinfo/issues/60402013-07-26T00:01:50Zben leinfelderleinfelder@nceas.ucsb.edu
<p>I indexed a document from EVOS that uses a reference for a creator rather than the details of the person:<br /><pre>
<creator><references>1359152217358</references></creator>
</pre><br />But in the index it shows up as "||" instead of following the reference back the the id where it was declared:<br /><pre>
<associatedParty id="1359152217358">...
</pre></p>
<p><a class="external" href="http://evos.nceas.ucsb.edu/evos/metacat/df35c.9.14/default">http://evos.nceas.ucsb.edu/evos/metacat/df35c.9.14/default</a></p>