Project

General

Profile

Activity

From 04/22/2014 to 05/21/2014

05/21/2014

06:24 PM Revision 8794 (metacat): use a member instance of ezid service that only logs in every 24 hours (or other time TBD) instead of every time there is an interaction with the service. Saves us many calls when doing batch updates to ezid but keeps us from trying to use expired sessions. Motivated by https://projects.ecoinformatics.org/ecoinfo/issues/6440
ben leinfelder
02:59 PM Revision 8793 (metacat): Fixed an error in the ORE model diagrams
Lauren Walker

05/20/2014

04:11 PM Revision 8792 (metacat): prevent js scriptlets from running when we return error messages to the client by escaping any potentially harmful xml blocks. https://projects.ecoinformatics.org/ecoinfo/issues/6224
ben leinfelder
04:09 PM Revision 8791 (metacat): allow updates to all doi: prefixes - realized we are already restricting to specific replica servers when updating these. worst case is we try to update a registration for which we are not the owner. https://projects.ecoinformatics.org/ecoinfo/issues/6440
ben leinfelder
04:08 PM Revision 8790 (metacat): show the SM and ORE generation buttons even if they have not registered/configured dataone. many potential MNs want to see their generated SM _before_ registering (and we want them to too!).
ben leinfelder
10:56 AM Feature #5989: Track data download, view and citation statistics
I believe this is now all targeted for the CNs. We may consider moving tasks into the DataONE tracker as appropriate... ben leinfelder
10:40 AM Revision 8789 (metacat): restrict DOI updates to DOIs that match our server shoulder -- may consider opening this up to any "doi:" prefix if this is too restrictive. https://projects.ecoinformatics.org/ecoinfo/issues/6440
ben leinfelder
10:35 AM Story #6440 (In Progress): Keep DOI registrations current and resolvable
There is now a section in the metacat admin screen (replication configuration) for updating DOIs. Three input choices... ben leinfelder
10:00 AM Bug #6475 (Closed): CN.create() implementation sets SystemMetadata.archived=false
We need to allow cn.create() calls with archived=true in cases where the CN is harvesting old/archived objects that s... ben leinfelder
09:57 AM Bug #6466 (Closed): Groups Assignment after making some accounts aliases
Jing and I talked about this and I agree with his proposal and implementation. ben leinfelder

05/19/2014

02:21 PM Revision 8788 (metacat): use separate surName and givenNames to lookup ORCIDs.
ben leinfelder
01:17 PM Revision 8787 (metacat): Changed errors in the ORE model expansion metacat documentation diagrams
Lauren Walker

05/16/2014

04:49 PM Feature #6556 (Rejected): Create PROV annotations in ResourceMapFactory
Add methods such as addWasDerivedFrom() to add PROV triples to an RDF Lauren Walker
04:13 PM Task #6555 (Closed): Convert PNG diagrams in ORE Model Expansion doc to plantUML diagrams
Lauren Walker
04:12 PM Story #6548 (In Progress): Expand ORE model to allow relationships for derived datasets
A page in the metacat docs has been added to describe changes to Metacat's ORE model. Lauren Walker
03:13 PM Revision 8786 (metacat): Describe the primary data resource in the derived resource RDF
Lauren Walker
10:03 AM Revision 8785 (metacat): Create a derived data and metadata in the OrePackageTest
Lauren Walker

05/15/2014

03:17 PM Revision 8784 (metacat): all full-text queries for ORCID, but it isn't that great because we might have a"PISCO" creator that shows us in may different orcid profiles...false matches.
ben leinfelder
11:55 AM Revision 8783 (metacat): correct glaring erros -- still needs to be honed, but at least it runs without NPE and Jena/foresite errors
ben leinfelder
11:29 AM Revision 8782 (metacat): stub for testing ORE augmentation - this generates an ORE, adds a "wasDerivedFrom" triple and saves to Metacat MN for indexing. https://projects.ecoinformatics.org/ecoinfo/issues/6548
ben leinfelder
10:27 AM Revision 8781 (metacat): include ORE formatId as handled by the RDF subprocessor and index prov:wasDerivedFrom field where it exists in the RDF model. https://projects.ecoinformatics.org/ecoinfo/issues/6548
ben leinfelder
09:34 AM Revision 8780 (metacat): expand the sparql queries to include dcterms:identifier
ben leinfelder

05/14/2014

02:09 PM Revision 8779 (metacat): Text changes to ORE docs
Lauren Walker
01:22 PM Revision 8778 (metacat): Add documentation for the ORE model expansion for derived data
Lauren Walker
12:04 PM Revision 8777 (metacat): use HttpClient to query orcid so I can easily set headers and such -- getting 503s from their production server when I test on dev.nceas...odd
ben leinfelder
11:43 AM Revision 8776 (metacat): adjust tests for production service -- more "real" information shows additional return values from the query.
ben leinfelder
09:18 AM Revision 8775 (metacat): switch to the production ORCID server for looking up orcid matches for our creators.
add test to summarize how many creator matches we can actually find. https://projects.ecoinformatics.org/ecoinfo/issu... ben leinfelder

05/13/2014

03:36 PM Revision 8774 (metacat): use a non-public rightsHolder for both EML and Annotation test documents now that the RDF subprocessor checks each annotation to see that it came from a user that as write permission for the object being annotated.
ben leinfelder
03:34 PM Revision 8773 (metacat): test for update using the updated EML file, not the original. Also add the SM to the shared map so that the indexing process can consult SM.accessPolicy when indexing annotations that assert things about those test documents.
ben leinfelder
03:32 PM Revision 8772 (metacat): ignore the metacat/solr comparator tests - they are one-offs.
ben leinfelder
03:31 PM Revision 8771 (metacat): change the hazelcast group name to be the default "metacat" instance so that the metacat-index tests pass without additional local configuration, at least when running a default metacat deployment.
ben leinfelder
01:36 PM Feature #6554 (New): Tag the Metacat doc pages to populate the docs index
The Metacat docs index is pretty sparse right now with only 2 terms. It would be pretty simple to go through each pag... Lauren Walker

05/12/2014

12:55 PM Bug #6488 (Closed): Comply with DataONE MN.query() REST endpoint specifications.
been using this in Metacat/MetacatUI for a while now...closing. ben leinfelder
12:50 PM Feature #6416 (In Progress): Do not allow restrictive access control change to content with a DOI
Thinking about this more, we have only CN.setAccessPolicy() in the DataONE API. So any change in the access policy wo... ben leinfelder
11:31 AM Revision 8770 (metacat): do not set archived=false for all CN.create calls. The CN will use create() even harvesting content that is new to it and needs to handle already-archived content. https://projects.ecoinformatics.org/ecoinfo/issues/6475
ben leinfelder
11:29 AM Bug #6475 (Feedback): CN.create() implementation sets SystemMetadata.archived=false
Commenting out this in the Metacat impl -- not sure why this was being set explicitly to begin with. ben leinfelder

05/09/2014

01:48 PM Revision 8769 (metacat): cache the imported models to avoid timeouts from remote hosts (or being locked out for too many requests in a given time period).
ben leinfelder

05/08/2014

04:25 PM Revision 8768 (metacat): process all the returned annotation suggestions until we find one that is appropriately located in the subclass hierarchy for the given superclass.
ben leinfelder
04:23 PM Revision 8767 (metacat): use in-memory TDB dataset for querying annotations for indexing -- this comes with the same reasoning capabilities as the directory-based one, but has the benefit of not filling the directory with triples that will not be used again. prepping for d1 AHM
ben leinfelder

05/07/2014

11:16 PM Revision 8766 (metacat): pass around the object file path rather than the data stream so that multiple subprocessors can index the same object and not consume the stream before it gets to the next one. In preparation for extending the assertions stored in OREs. https://projects.ecoinformatics.org/ecoinfo/issues/6548
ben leinfelder
11:12 PM Revision 8765 (metacat): when indexing annotations directly, just use an in-memory triple store rather than TDB since we remove each graph as it is processed (and my TDB instance would get into the multi-GB range with a few runs, even if I removed the old models)
ben leinfelder
10:42 AM Story #6548: Expand ORE model to allow relationships for derived datasets
Looking at the ORE spec, this is certainly acceptable practice - they just want to make sure that there isn't any orp... ben leinfelder

05/06/2014

04:22 PM Story #6548 (Closed): Expand ORE model to allow relationships for derived datasets
Design a new model for Metacat's OREs where relationships:
- wasGeneratedBy
- derivedFrom
- used
- etc.
are ...
Lauren Walker
02:10 PM Bug #6546 (Closed): old metacat URIs should redirect to #view service
Metacat trunk now has the short form Metatcat read URIs redirecting to the Metacat UI #view using the looked-up pid/i... ben leinfelder

05/05/2014

05:17 PM Revision 8764 (metacat): redirect "short form" metacat read URIs to the the new Metacat UI using the configured UI context. This translates the docid -> pid to use the correct identifier for the correct service. https://projects.ecoinformatics.org/ecoinfo/issues/6546
ben leinfelder
01:39 PM Bug #6546: old metacat URIs should redirect to #view service
It looks like we have decided that we will redirect users to the new KNB /#view service from the /metacat/{docid} KNB... Lauren Walker
11:37 AM Bug #6546: old metacat URIs should redirect to #view service
For the perl registry, I ended up needing another Metacat servlet method to get the docid when given a pid:
https://...
ben leinfelder
11:21 AM Bug #6546: old metacat URIs should redirect to #view service
We could update the Metacat KNB skin with a informational message about the new KNB redesign with a link to the new K... Lauren Walker
10:52 AM Bug #6546: old metacat URIs should redirect to #view service
You are proposing a redirect from a Metacat docid to a DataONE Identifier -- these are not always the same value and ... ben leinfelder
02:25 AM Bug #6546 (Resolved): old metacat URIs should redirect to #view service
Researchers have complained that prior citations on the KNB metacat of the form:
https://knb.ecoinformatics.or...
Matt Jones

05/02/2014

04:39 PM Revision 8763 (metacat): simplify lookup for classes and orcid. remove the "random" annotation code branches -- just too confusing to look at those bogus classes especially now that we have "real" generated annotations.
ben leinfelder

04/30/2014

03:47 PM Revision 8762 (metacat): add 'test' for indexing annotations without actually storing the RDF of the generated annotation.
ben leinfelder
03:46 PM Revision 8761 (metacat): only allow multiple values for multi-valued fields....
ben leinfelder
02:17 PM Feature #6530 (Resolved): Create a new service to update DOI pointers
I checked in the UpdateDOI class today which has the ability to either:
-Update all DOIs for metadata with an EML na...
Lauren Walker
01:33 PM Bug #6531 (Resolved): Make sure that the MN node description wouldn't be overwritten by the default value during a Metacat upgrading
I can't reproduce the bug any more. I also looked the code and the node description shouldn't be overwritten by the d... Jing Tao
01:02 PM Revision 8760 (metacat): Add admin service to update DOI registrations by specifying a list of formatIds or DOIs, or update all.
Lauren Walker

04/29/2014

05:21 PM Revision 8759 (metacat): use new method to override the CN URL when constructing a CNode instance. see https://redmine.dataone.org/issues/5142
ben leinfelder
04:55 PM Revision 8758 (metacat): use newer httpclient library so that Jena's dependency is met - this goes all the way back to d1_common/libclient needing to pull in the newer library.
ben leinfelder
04:54 PM Revision 8757 (metacat): first pass at direct EML->semantic index method. Still produces an RDF model, but does not persist it in Metacat, only in the triplestore. Allows us to re-run without adding stale RDF to the MN store.
ben leinfelder
04:52 PM Revision 8756 (metacat): allow multivalued fields to be indexed using the "fields" pass through.
ben leinfelder
04:47 PM Revision 8755 (metacat): Remove the attribute disable from the update button if it has been submitted.
Jing Tao
03:31 PM Revision 8754 (metacat): The image which has the default values.
Jing Tao
03:13 PM Revision 8753 (metacat): Add a new screen shot which contains the cn url.
Jing Tao

04/28/2014

04:54 PM Revision 8752 (metacat): Localized the file which doesn't have the bean for dataUrl.
Jing Tao
03:38 PM Revision 8751 (metacat): Add the cn url.
Jing Tao
03:37 PM Revision 8750 (metacat): Add the text field for the cn url.
Jing Tao
03:30 PM Revision 8749 (metacat): Store the cn url in the backup.
Jing Tao

04/25/2014

02:25 PM Bug #6542 (Closed): Fix MNodeService.systemMetadataChanged() to be asynchronous
The DataONE MNWebTester will at times show failures against Metacat MNs for calls to MNAuthorization.systemMetadataCh... Chris Jones

04/24/2014

04:20 PM Bug #6517 (Closed): Sanparks FGDC upload does not accept data files
Switched to use Apache's FileUpload library and can now handle the chunked streaming uploads. Added a test for this s... ben leinfelder
01:28 PM Revision 8748 (metacat): switch to use FIleUpload instead of O'Reilly COS library for handling chunked file uploads. https://projects.ecoinformatics.org/ecoinfo/issues/6517
ben leinfelder
01:26 PM Revision 8747 (metacat): forgot to check in the actual class: first pass at allowing admins to update DOI registration. This only acts on EML objects at the moment and is meant to illustrate one mechanism for updating the DOIs. https://projects.ecoinformatics.org/ecoinfo/issues/6530
ben leinfelder
11:50 AM Feature #6512: Add brief explanation to Metacat docs for using custom map markers in a custom theme
Technically this is a part of the metacat project Lauren Walker
11:49 AM Task #6532 (Closed): Explain different metacat and CN endpoints in the MetacatUI docs
Add a section to the Metacat documentation that explains how to create a new AppModel for pointing to a custom endpoi... Lauren Walker
11:22 AM Bug #6531 (Resolved): Make sure that the MN node description wouldn't be overwritten by the default value during a Metacat upgrading
When a Metacat was configured as a DataONE member node, the node description and et al would be stored in the metacat... Jing Tao

04/23/2014

12:35 PM Revision 8746 (metacat): first pass at allowing admins to update DOI registration. This only acts on EML objects at the moment and is meant to illustrate one mechanism for updating the DOIs. https://projects.ecoinformatics.org/ecoinfo/issues/6530
ben leinfelder
12:18 PM Feature #6530 (Resolved): Create a new service to update DOI pointers
Add to the admin interface a way to specify a list of pids or a formatId and update the URLs associated with those DO... Lauren Walker
10:45 AM Bug #6520 (Closed): Metacat indexing the resource map prior to metadata document
Tested package upload from Morpho and the resourceMap was successfully indexed. ben leinfelder
10:20 AM Bug #6529 (Closed): Solr query used for ORE lookup in Metacat has a syntax issue
Corrected the syntax in trunk and added an additional JUnit assertion to the ORE package test that already existed. ben leinfelder
10:19 AM Revision 8745 (metacat): correct the ORE lookup query syntax and add junit assertion to check that it continues to function as expected. https://projects.ecoinformatics.org/ecoinfo/issues/6529
ben leinfelder

04/22/2014

03:39 PM Bug #6520 (In Progress): Metacat indexing the resource map prior to metadata document
Now we will submit the ORE for indexing after we are sure the metadata document has been indexed. ben leinfelder
01:31 PM Bug #6520: Metacat indexing the resource map prior to metadata document
I also saw the error when i used morpho 1.10 to insert a package with data:
[ERROR]: org.apache.solr.common.SolrExce...
Jing Tao
01:30 PM Bug #6520: Metacat indexing the resource map prior to metadata document
Actually, the localhost has the same error:
[ERROR]: Solr index doesn't have the information about the id tao.13010....
Jing Tao
03:38 PM Revision 8744 (metacat): index the ORE after we submit the metadata for indexing. https://projects.ecoinformatics.org/ecoinfo/issues/6520
ben leinfelder
03:33 PM Bug #6529: Solr query used for ORE lookup in Metacat has a syntax issue
I can see this happening on the KNB. Metacat is querying itself (solr index) to find potential OREs and seems there's... ben leinfelder
03:26 PM Bug #6529 (Closed): Solr query used for ORE lookup in Metacat has a syntax issue
I also saw the error when i used morpho 1.10 to insert a package with data:
[ERROR]: org.apache.solr.common.SolrExce...
Jing Tao
01:53 PM Bug #6516: Solr index doesn't work in the remote host in the replication when a host is ubuntu 12.04 and another one is ubuntu 14.04
Metacat indexed a document before it wrote to db. This issue was fixed. Jing Tao
01:51 PM Bug #6518 (Resolved): Authentication service in Ecogrid only access metacat through http connection
Using the SystemUtil.getMetacatServletUrl to replace the hard-coded "http" in authenciation, put, authenticatedQuery ... Jing Tao
11:43 AM Revision 8743 (metacat): include BioPortal lookup for Entity matches using the data table description. TODO: only associate measurements to the entity observation if they apply.
ben leinfelder
 

Also available in: Atom