Project

General

Profile

Actions

Bug #7093

open

Metacat-index is not indexing all package members correctly

Added by Chris Jones over 8 years ago. Updated over 8 years ago.

Status:
New
Priority:
Normal
Assignee:
Category:
index
Target version:
-
Start date:
08/26/2016
Due date:
% Done:

0%

Estimated time:
Bugzilla-Id:

Description

Shirley pointed out that for the urn:uuid:09fd1d96-c6b0-442d-b1cb-e2c4a2e476b6 package on arcticdata.io:

https://arcticdata.io/catalog/#view/urn:uuid:09fd1d96-c6b0-442d-b1cb-e2c4a2e476b6

Notice that the resource map aggregates the following 29 objects:

$ curl -s -o - \
"https://arcticdata.io/metacat/d1/mn/v2/object/resource_map_urn:uuid:09fd1d96-c6b0-442d-b1cb-e2c4a2e476b6" | \
xmlstarlet sel -t -v "//ore:aggregates/@rdf:resource" -n | \
cut -d "/" -f7

urn%3Auuid%3A1841cbef-f024-4099-ba2b-ccf89e65d295
urn%3Auuid%3A8fe1169b-5c39-4793-bdc3-a3ced4e321ad
urn%3Auuid%3A355e25c7-9637-42e6-85ea-6d642c02a454
urn%3Auuid%3Ac9d4d530-aafb-4d3b-bef9-9eb9bf4dfba3
urn%3Auuid%3A6ce5b2bd-fcf9-4a4f-bbf4-63337ea0ad16
arctic-data.9492.1
arctic-data.9468.1
urn%3Auuid%3Af6c12e96-95c0-423f-83a0-36c3c0c6abf4
urn%3Auuid%3Ada0f723b-bb38-4437-bee5-56b0b4efcb5d
urn%3Auuid%3A58ba459e-c508-4b74-9f2e-db569cf9d737
urn%3Auuid%3A09a0749e-82dd-48d8-8be2-23a7993db65b
urn%3Auuid%3A7cda7813-1d23-4e57-b0cb-59980bd2dbde
urn%3Auuid%3A9b2d3c3c-d775-4e20-b9e6-896f2e102f0f
urn%3Auuid%3Ab9218df4-5a34-4ddc-8d35-6f7824049b89
urn%3Auuid%3Ac385222e-caf8-4ae4-b3bf-463f2fba3968
urn%3Auuid%3A9a9077ee-011c-44de-8b77-b16280998250
urn%3Auuid%3Ad933a025-8dc0-4933-91af-5b49e1e48079
urn%3Auuid%3Ae2ba7348-1c64-405b-9840-68dc8d776e5c
urn%3Auuid%3Add5925d8-a9a1-48df-9623-3b75b48d0460
urn%3Auuid%3Abb885aa0-6e31-4de4-adaa-b842eaf3a9fa
urn%3Auuid%3Ac1d2ab68-e653-488a-9942-fd1045a7aad6
arctic-data.9469.1
urn%3Auuid%3Ab70e055d-593d-4b3a-8aaf-04574e377e46
urn%3Auuid%3Aa0e2e791-0a07-4213-b53e-ed5f9a16178e
urn%3Auuid%3Aa0769215-4c17-4935-87ed-442d761a4269
urn%3Auuid%3A17c1009b-5e1f-4d2d-b040-b21a6366adda
urn%3Auuid%3A760cc12e-e263-48cd-b337-eeb2f6336702
urn%3Auuid%3A96dc7150-0e19-4647-ae77-d5dda1abedda
urn%3Auuid%3A09fd1d96-c6b0-442d-b1cb-e2c4a2e476b6

However, looking at the index, only 26 are listed with a resourceMap field of resource_map_urn:uuid:09fd1d96-c6b0-442d-b1cb-e2c4a2e476b6:

$ curl -s -o - 'https://arcticdata.io/metacat/d1/mn/v2/query/solr/q=resourceMap:"resource_map_urn:uuid:09fd1d96-c6b0-442d-b1cb-e2c4a2e476b6"&fl=id&rows=50&wt=csv'
id
urn:uuid:09fd1d96-c6b0-442d-b1cb-e2c4a2e476b6
urn:uuid:96dc7150-0e19-4647-ae77-d5dda1abedda
urn:uuid:6ce5b2bd-fcf9-4a4f-bbf4-63337ea0ad16
urn:uuid:8fe1169b-5c39-4793-bdc3-a3ced4e321ad
urn:uuid:f6c12e96-95c0-423f-83a0-36c3c0c6abf4
urn:uuid:58ba459e-c508-4b74-9f2e-db569cf9d737
urn:uuid:c9d4d530-aafb-4d3b-bef9-9eb9bf4dfba3
urn:uuid:da0f723b-bb38-4437-bee5-56b0b4efcb5d
urn:uuid:355e25c7-9637-42e6-85ea-6d642c02a454
arctic-data.9492.1
arctic-data.9468.1
urn:uuid:17c1009b-5e1f-4d2d-b040-b21a6366adda
urn:uuid:c1d2ab68-e653-488a-9942-fd1045a7aad6
urn:uuid:bb885aa0-6e31-4de4-adaa-b842eaf3a9fa
urn:uuid:b70e055d-593d-4b3a-8aaf-04574e377e46
urn:uuid:a0769215-4c17-4935-87ed-442d761a4269
arctic-data.9469.1
urn:uuid:09a0749e-82dd-48d8-8be2-23a7993db65b
urn:uuid:d933a025-8dc0-4933-91af-5b49e1e48079
urn:uuid:9b2d3c3c-d775-4e20-b9e6-896f2e102f0f
urn:uuid:e2ba7348-1c64-405b-9840-68dc8d776e5c
urn:uuid:b9218df4-5a34-4ddc-8d35-6f7824049b89
urn:uuid:7cda7813-1d23-4e57-b0cb-59980bd2dbde
urn:uuid:9a9077ee-011c-44de-8b77-b16280998250
urn:uuid:c385222e-caf8-4ae4-b3bf-463f2fba3968
urn:uuid:1841cbef-f024-4099-ba2b-ccf89e65d295

The following three data objects are missing:

urn:uuid:760cc12e-e263-48cd-b337-eeb2f6336702    2016-co2.csv    
urn:uuid:dd5925d8-a9a1-48df-9623-3b75b48d0460    2016-gps.csv    
urn:uuid:a0e2e791-0a07-4213-b53e-ed5f9a16178e    2016-met.csv    

I've tried reindexing all of the pids associated with the package, and the package itself, with no luck. I've turned on DEBUG level logging in /var/lib/tomcat7/webapps/metacat/WEB-INF/lib/classes/log4j.properties to get more logging information from metacat-index into catalina.out. I restarted Tomcat to enable the logging changes.

We need to check to see why these members aren't being indexed correctly. It's also affecting many other packages according to the graduate interns on the data team.


Related issues

Is duplicate of Metacat - Bug #7083: Metadata/data objects which have obsoletedBy field ignore the resource map index ClosedJing Tao08/08/2016

Actions
Actions #1

Updated by Chris Jones over 8 years ago

  • Description updated (diff)
Actions #2

Updated by Chris Jones over 8 years ago

  • Is duplicate of Bug #7083: Metadata/data objects which have obsoletedBy field ignore the resource map index added
Actions

Also available in: Atom PDF