Add the class to upgrade the solr schema.
Every time when we modified the metacat.properties file through the PropertyService, the Setting class will be updated as well. Otherwise, the values on the Setting class are still the old one.
Add the code to upgrade the solr schema.
Close the input stream object on the MN.replicate method.
Add the java code to handle the ezid configuration.
merge from 2.5.x branch: neglected to replace solr schema during 2.5.0 upgrade - this will do it for 2.5.1.
Add the keyword "select" into the list.
rollback: set session.subject so libclient can look up certificate. Seems like the default behavior may have changed?
set session.subject so libclient can look up certificate. Seems like the default behavior may have changed?
Add code to update smmediatypeproperties table.
[merge from 2.4 branch] use solr schema upgrader for 2.4.2 release -- need to replace the solr schema to include geohash fields
add support for v2 DataONE API.
allow updates to all doi: prefixes - realized we are already restricting to specific replica servers when updating these. worst case is we try to update a registration for which we are not the owner. https://projects.ecoinformatics.org/ecoinfo/issues/6440
restrict DOI updates to DOIs that match our server shoulder -- may consider opening this up to any "doi:" prefix if this is too restrictive. https://projects.ecoinformatics.org/ecoinfo/issues/6440
change the hazelcast group name to be the default "metacat" instance so that the metacat-index tests pass without additional local configuration, at least when running a default metacat deployment.
Add admin service to update DOI registrations by specifying a list of formatIds or DOIs, or update all.
use new method to override the CN URL when constructing a CNode instance. see https://redmine.dataone.org/issues/5142
Store the cn url in the backup.
forgot to check in the actual class: first pass at allowing admins to update DOI registration. This only acts on EML objects at the moment and is meant to illustrate one mechanism for updating the DOIs. https://projects.ecoinformatics.org/ecoinfo/issues/6530
merge recent upgrade changes from 2.4 branch
allow statements starting with 'WITH'
Add the code to handle a runtime exception.
Add code to persist the properties in the group 3.
If the user doesn't exist in the password file, the auth configuration will give an error.
Add some log information.
set registry permissions when configuring the exploded war on *nix systems. https://projects.ecoinformatics.org/ecoinfo/issues/4690
use 2.3.0 for this next release of metacat.
Add code the display the SchemaModification exception.
Fixed a catch clause syntax which is only compatible with java 1.7.
Add code the overwrite the schema.xml in the solr-home/conf.
correctly configure metacat-index to use metacat context/deployment location. https://projects.ecoinformatics.org/ecoinfo/issues/6138
Reviewed code for all uses of FileInputStream, checking to see if the method should be closing the stream, and if so, closing it in the method as well as in the finally clause to ensure we don't leak file descriptors.
Refer to metacat.war deployments since those are now the default. https://projects.ecoinformatics.org/ecoinfo/issues/6082
only attempt to generate OREs for objects that we know not to have them already. https://projects.ecoinformatics.org/ecoinfo/issues/6061
change hazelcast group name to match the current context. https://projects.ecoinformatics.org/ecoinfo/issues/5624
Add code to modify the web.xml in metacat-index context if the metacat's name is not knb.
do not create solr-home if there is no template to compy into that directory (need to be able to create it later if/when someone decides to use and deploy metacat-index). https://projects.ecoinformatics.org/ecoinfo/issues/6006
do not attempt to copy solr-home template from metacat-index webapp if it does not exist. This would be in cases where metacat-index is not deployed. https://projects.ecoinformatics.org/ecoinfo/issues/6006
organize imports
initialize the SOLR home directory if it does not already exist.
use maven to manage most jar dependencies in Metacat.Exceptions include: LSID, Datamamager (EML),
select only distinct guids (synch may have failed more than once for any given guid)https://redmine.dataone.org/issues/3539
include xml_revisions.do not allow removal of server_location = 1 documents (these are not replicas).https://redmine.dataone.org/issues/3539
class for removing failed/invalid replicas from target nodes that previously held replicated content (KNB/LTER/PISCO/etc). https://redmine.dataone.org/issues/3539
shorten the systemmetadata* table names for Oracle's 30 character limit. move version to 2.0.5. http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5717
allow SM resynch to be executed any time, not just during start up.https://redmine.dataone.org/issues/3116
check for null archived flag in ORE SMhttps://redmine.dataone.org/issues/3046
utility methods to update/reserialize existing ORE maps that were generated with older foresite (and included bad dateTime strings).https://redmine.dataone.org/issues/3046
include dataone.ore.downloaddata as a configurable property in case MNs (like LTER) want to have the process download externally-stored data files described in an EML data package.
instead of generating SM and ORE maps during dataone configuration/MN registration, moved this all to the replication admin screen where we can target generation for specific nodes. That way it's more controlled as to when and where we generate DataONE required content....
include all EML versions (had been only eml 2.1 for testing)
add "Generate System Metadata" button to the replication server list display. When clicked, we generate SM for records belonging to that source server. This is only enabled when DataONE has been configured.https://redmine.dataone.org/issues/2762
expose serverLocation parameter to run GenerateSystemMetadata for different replication parters as needed.https://redmine.dataone.org/issues/2740
only generate system metadata for original objects.https://redmine.dataone.org/issues/2721
save backup properties before attempting node registration/update so that we don't "forget" the user input
do not download data at this point
only 2.1.0 EML docs for ORE generation right now...
handle last group of ids (oops)
use range of the list for test system metadata
use non-random list for generating system metadata in test mode
use scope prefix (knb-lter-%) for random test doc population
Globally change the property 'dataone.memberNodeId' to 'dataone.nodeId'. This is more useful for both MNs and CNs implemented in Metacat. Also, change D1NodeService.getLogRecords() to return log entries with the actual node id rather than the IP address (looks like a cut/paste error)....
do not subset the list for MS generation testing -- at least not as the default in svn!
only run ORE generation for EML docs -- no need to run this for all documents (yikes!)
use IdMan method to find docids that do not already have system metadata records -- this lets us re-run without re computing system metadata for every entry (in case the process is interrupted). I haven' been using this option because I wanted to continually regenerate all SM for everything in my test DBs, but we are so close to release that I want to get this in there.
for testing: limit and randomize the docs to generate metadata for
FOR TESTING ONLY: limit number of records to 100 so that we can get an estimate
update the memberNodeId in existing system metadata only after the register/update is successful with the CN -- we can avoid unneeded SM updates in cases when the register/update fails because we gave the CN bad info that it rejects. https://redmine.dataone.org/issues/2308
include member node id text field now that the CN is not assigning random Ids.https://redmine.dataone.org/issues/2308
refactor D1-specific upgrade utilities into their own package
generate SystemMetadata during D1 registration (not 2.0.0 upgrade). This process runs in a thread and updates a metacat.properties value when it is complete.
dataone configuration and registration enhancements:-include flag to disable D1 services, currently only the MN side enforces this-do not allow multiple registration attempts if we have just submitted and are awaiting Node verification by the CN.-do not allow configuration "bypass" if D1 settings have been configured previously....
use correct Collections import
Show "Update" button if this MemberNodeId is already registered with DataONE, otherwise use the "Register" label
Modify admin configuration to include default replication policy. Extensively revised the DataONE configuration page, including new wording for intro, improved tooltips throughout, new arrangement of sections, and other cosmetic changes.
Remove ability to edit NodeID from D1 configuration page. Fix update of contactSubject and dataone.ore.generated property name.
do not wait for SM generation to complete during the upgrade -- this way the web UI wont hang for days. the process sets a metacat property when it is complete.
do not shutdown hazelcast -- it needs to be running after the upgrade process so that Metacat actually works.I think the newer version of HZ makes it so the threads are all released as needed.http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5572
Commenting out the parts of the upgrade script that started to refer to EXID. At this point, the registration of EZID identifiers will be done out-of-band with respect to the upgrade.
upgrade to hazelcast 1.9.4.6 so that threadpools are released when not needed (http://code.google.com/p/hazelcast/issues/detail?id=765).include ant target to run a specific main class (mostly for debugging)
multithreaded implementation for processing docids for system metadata generation.need to investigate ant/junit running that deadlocks hazelcast (config?)
only attempt to update date-like nodedata values.
include generate system metadata upgrade in the success flag
more clean up - reuse prepared statement for data update
look up nodedata values first, then update each one - trying to avoid out of memory exception.
rollback processing Error change -- creates a loop on error. ugh
report processing errors after exceptions have been caught and recorded, otherwise the web UI is blank and there is no clue what happened unless you look in the logs.
Update the D1Admin class to set the dataone.contactSubject property. I've added the property to the http request to be added to the JSP form, but for now am setting the property using the dataone.subject field value. Not sure if we want to expose the contact subject in the form yet or not.
generate ORE maps only once -- and persist the flag to the main backup properties so that subsequent Metacat upgrades remember this value.
Added DOI generation to the 2.0.0 upgrade process. To succeed, this script must be run on a fresh 2.0.0 database, or on a 1.9.5 version database, as those are the only ways to get the needed foreign keys to be marked as deferrable. The identifier conversion must be turned on by setting correct properties in metacat.properties. See the comments in GenerateGlobalIdentifiers for details. By default, conversion is set to false in the properties file. If you want to convert an instance to use DOIs, be sure to set metacat.properties up BEFORE running through the Metacat configuration and database upgrade.
Refactoring classes that throw generic Exception class to throw their more specific subclasses so that new exceptions are not hidden behind generic messages. Makes debugging easier.
Use the Collections class from java.util.
-generate system meta for all docids, even those not originating on the server (replicas from the past)-generate ORE docs and download remote data only for those documents that originated on this server being upgraded.http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5522
refactor generate system meta loop to the factory class -- to be reused in sysmeta and ORE generationhttp://bugzilla.ecoinformatics.org/show_bug.cgi?id=5522
do NOT generate ORE maps or download data when we do the initial System Metadata generation -- this is deferred until D1 registration.
make more generic so that a custom list of IDs can be passed in.
insert OR update system metadata -- no need to do an update right after initial insert...
call the System Metadata generator during upgrade to 2.0.0
In IdentifierManager.updateSystemMetadata(), add a check for invalid system metadata (fields that throw a NullPointerException on access) to ensure that system metadata is populated correctly. Updated calling classes to handle the exception.