1. In the identifierExists method, the code to check if the identifier exist on the serial-id fields of the system metadata table.2. Fixed a bug on the identifierExists - it considered an identifier didn't exist when the determining process threw an exception.
Add a method to check if a serial id exists.
Add a method to get the pid of the head version of a sid.
add Annotator Store implementation -- pass through to D1 API for the AnnotatorJS API
include person's full name in the JWT returned from /token
Fixed a bug that the reindex of data objects may index an archived data object.
Fixed a bug that the solr index of data files doesn't reflect the change of access control.
[merge from 2.4 branch] use solr schema upgrader for 2.4.2 release -- need to replace the solr schema to include geohash fields
remove AnnotatorService completely - was moved to cn-index-processor
let metacat-index lookup annotations for indexing rather than the metacat "reindex" action.
Add delete log for data objects on CNs.
only index non-empty comment text
index both tags and text from annotateit.org
query by consumerKey until the pid search facets are fully supported on annotateit.org
During the replication, the remote content will be saved without alteration.
allow per-document reindexing to be initiated by any user (to support third-party annotations)
Write the input stream into the file system without alteration in dataone create and update methods.
look up annotations when reindexing a given pid. still very much a prototype in that we are looking up annotations from an external annotator-store. TODO: add pid filtering to query when annotateit.org supports it (pending upgrade on their site).
use http://tools.ietf.org/rfc/rfc3023 spec for conformsTo property. use the full xpath for EML dataTable and attribute selectors
Added the code to inform users the pid was deleted in the NotFound exception.
Added the code to check if a not-found object was deleted in the isAuthorized method.
Move the code to get the object in front of the method to get the system metadata.
Add a utility method for determine if there is a delete event for a given id.
Removed the method which had the byte array attribute.
Remove the system metadata for data objects.
change the way to delete the solr index of a resource map.
add /token endpoint for annotatorJS/annotateIt.org integration. https://github.com/DataONEorg/sem-prov-design/issues/18
Persitence the system metadata object in the memory before deleting it from hazelcast.
Add the code to handle the delete of the resource map.
Make the delete method work.
Make the deleteSystemmetadata method really roll-backable.
Add the code to delete systemmetadata.
Add a code to check if the pathquery engine is enabled in the checkIndexPaths method.
convert v2 SM to v1 SM for the v1 service call response
update to use v2 types for indexing
remove CN.systemMetadataChanged in favor of the CN.updateSystemMetadata method. Otherwise there's no good way to know where to fetch the auth copy from since the SM change might be to switch the authMN!
add support for v2 DataONE API.
remove dependency on HttpMessage that was in the utilities project but is now removed in favor of newer (standard) http client library code.
Include PDF version of the metadata in the package download. https://projects.ecoinformatics.org/ecoinfo/issues/6053
take advantage of the ezidclient for multi-threaded/asynchronous DOI registration. This will be most useful for doing large batch updates and not so much for the one-at-a-time publish actions but works in either context. https://projects.ecoinformatics.org/ecoinfo/issues/6440
use a member instance of ezid service that only logs in every 24 hours (or other time TBD) instead of every time there is an interaction with the service. Saves us many calls when doing batch updates to ezid but keeps us from trying to use expired sessions. Motivated by https://projects.ecoinformatics.org/ecoinfo/issues/6440
prevent js scriptlets from running when we return error messages to the client by escaping any potentially harmful xml blocks. https://projects.ecoinformatics.org/ecoinfo/issues/6224
allow updates to all doi: prefixes - realized we are already restricting to specific replica servers when updating these. worst case is we try to update a registration for which we are not the owner. https://projects.ecoinformatics.org/ecoinfo/issues/6440
show the SM and ORE generation buttons even if they have not registered/configured dataone. many potential MNs want to see their generated SM before registering (and we want them to too!).
restrict DOI updates to DOIs that match our server shoulder -- may consider opening this up to any "doi:" prefix if this is too restrictive. https://projects.ecoinformatics.org/ecoinfo/issues/6440
use separate surName and givenNames to lookup ORCIDs.
all full-text queries for ORCID, but it isn't that great because we might have a"PISCO" creator that shows us in may different orcid profiles...false matches.
use HttpClient to query orcid so I can easily set headers and such -- getting 503s from their production server when I test on dev.nceas...odd
adjust tests for production service -- more "real" information shows additional return values from the query.
switch to the production ORCID server for looking up orcid matches for our creators.add test to summarize how many creator matches we can actually find. https://projects.ecoinformatics.org/ecoinfo/issues/6423
change the hazelcast group name to be the default "metacat" instance so that the metacat-index tests pass without additional local configuration, at least when running a default metacat deployment.
do not set archived=false for all CN.create calls. The CN will use create() even harvesting content that is new to it and needs to handle already-archived content. https://projects.ecoinformatics.org/ecoinfo/issues/6475
cache the imported models to avoid timeouts from remote hosts (or being locked out for too many requests in a given time period).
process all the returned annotation suggestions until we find one that is appropriately located in the subclass hierarchy for the given superclass.
use in-memory TDB dataset for querying annotations for indexing -- this comes with the same reasoning capabilities as the directory-based one, but has the benefit of not filling the directory with triples that will not be used again. prepping for d1 AHM
when indexing annotations directly, just use an in-memory triple store rather than TDB since we remove each graph as it is processed (and my TDB instance would get into the multi-GB range with a few runs, even if I removed the old models)
redirect "short form" metacat read URIs to the the new Metacat UI using the configured UI context. This translates the docid -> pid to use the correct identifier for the correct service. https://projects.ecoinformatics.org/ecoinfo/issues/6546
simplify lookup for classes and orcid. remove the "random" annotation code branches -- just too confusing to look at those bogus classes especially now that we have "real" generated annotations.
Add admin service to update DOI registrations by specifying a list of formatIds or DOIs, or update all.
use new method to override the CN URL when constructing a CNode instance. see https://redmine.dataone.org/issues/5142
first pass at direct EML->semantic index method. Still produces an RDF model, but does not persist it in Metacat, only in the triplestore. Allows us to re-run without adding stale RDF to the MN store.
Store the cn url in the backup.
switch to use FIleUpload instead of O'Reilly COS library for handling chunked file uploads. https://projects.ecoinformatics.org/ecoinfo/issues/6517
forgot to check in the actual class: first pass at allowing admins to update DOI registration. This only acts on EML objects at the moment and is meant to illustrate one mechanism for updating the DOIs. https://projects.ecoinformatics.org/ecoinfo/issues/6530
first pass at allowing admins to update DOI registration. This only acts on EML objects at the moment and is meant to illustrate one mechanism for updating the DOIs. https://projects.ecoinformatics.org/ecoinfo/issues/6530
correct the ORE lookup query syntax and add junit assertion to check that it continues to function as expected. https://projects.ecoinformatics.org/ecoinfo/issues/6529
index the ORE after we submit the metadata for indexing. https://projects.ecoinformatics.org/ecoinfo/issues/6520
include BioPortal lookup for Entity matches using the data table description. TODO: only associate measurements to the entity observation if they apply.
Index the document after it has been inserted.
Index the document after document is written to the db.
check for null entities and/or attributes (typically when otherEntity is being used in EML).
remove extra space in log message
attribute the datapackage to the creator (using orcid if we can find it). https://projects.ecoinformatics.org/ecoinfo/issues/6267https://projects.ecoinformatics.org/ecoinfo/issues/6423
add test for BioPortal annotator service.
refactor web service calls to bioportal and orcid outside of the annotator class. test with orcid sandbox server. include orcid uri for the annotations being generated (we can index these and drive our searches on these values down the road). related to this: https://projects.ecoinformatics.org/ecoinfo/issues/6423 and also some semtools tasks.
remove leading '?' in the query parameter for MN.query() implementation. We want it to match CN behavior/expectations and comply with the DataONE specification for the interface. https://projects.ecoinformatics.org/ecoinfo/issues/6488
Use OBOE-SBC ontology for looking up concepts (it contains subclasses of our OBOE Characteristic and Standard superclasses). Restrict annotations to only subclasses that fit the OBOE model. Correct the xpointer and individual naming conventions so they are unique, but express the exact entity/attribute being annotated.
remove my api key. oops
add comment/pointer to BioPortal annotation service.
Include method to look up annotation classes from BioPortal. We still have OBOE-SBC in there, and theyhave the SWEET ontology. The suggestions returned are not perfect, but they can be better than nothing. Ideally, we'd only query a few ontologies so we don't end up using terms from medical ontologies that aren't really appropriate for our domain. https://projects.ecoinformatics.org/ecoinfo/issues/6256
Add xpointer FragmentSelectors to each annotation.Split attribute label into tokens to attempt matching to OBOE concepts.
include code to generate random annotations for UI testing. Effective, but can be confusing to see so many unrelated concepts on duplicate EML packages.
Remove the code to lookup alias dn in the getGroups method.
Rather than directly to modify the env, we use context.addToEnv.This fixed a bug in non-tls env, the alias log-in doesn't work.
first pass at generating annotations from EML attribute information. uses the OpenAnnotation model that the metacat-index tests assume which allows us to populate dynamic index fields for the annotation class[es]. There is still much to be done with finding appropriate concepts for each attribute. https://projects.ecoinformatics.org/ecoinfo/issues/6256
support content from all serverLocations when summarizing entity info (semtools)
allow "+" in solr query syntax. https://projects.ecoinformatics.org/ecoinfo/issues/6435
include read events when re-indexing obsoleted objects. https://projects.ecoinformatics.org/ecoinfo/issues/6424
Set the userManagementURL property.
In the authenticate method, if metacat can't get user info, the login still can be successful.
change a log information.
In the getALiasedName method, the referral set to ignore. Since the alias name is the local referral, we need to set it to ignore.
recursively submit obsoleted objects for indexing when instructed. https://projects.ecoinformatics.org/ecoinfo/issues/6424
First pass at a class for summarizing attribute information for analysis. (semtools) https://projects.ecoinformatics.org/ecoinfo/issues/6256
merge recent upgrade changes from 2.4 branch
look up guid when done setting access by docid so we can sync and refresh accesspolicy on MN and CN.
additional logging for set access
get guid from online id for call to SyncAccessPolicy
setAccessAction: get guid from passed in id for calls to SyncAccessPolicy, HazelcastService.refreshSystemMetadataEntry
example of how we can look up pid (guid) given a metacat docid.