[merge from branch to keep trunk up to date with upgrade history] prep for Metacat 2.1.1 release
Add a solr-home-dri for the http solr server.
include metacatui "skin" for use in Perl registry rendering and eventually EML/metadata rendering. https://projects.ecoinformatics.org/ecoinfo/issues/5951
Use the hazelcast event log.
merging upgrade scripts from 2.0 branch to trunk. https://redmine.dataone.org/issues/3847
upgrade to Metacat 2.1.0 on the trunk. This includes a new index_event table for storing indexing events that need to be reprocessed. https://projects.ecoinformatics.org/ecoinfo/issues/5944
refactor IndexEventLog a bit to simplify type/action information. prep for serializing IndexEvent objects to Metacat. https://projects.ecoinformatics.org/ecoinfo/issues/5944
correct spelling for index.eventlog.classname property
use an independent ISet<SystemMetadata> structure to communicate objects that should be indexed by metacat-index. https://projects.ecoinformatics.org/ecoinfo/issues/5943
Add a new property for the index event log class name.
Add a new property to specify the interval of a Timer to run the thread generating solr index.
Use the ";" as the seperator to replace "," in the resource name spaces.
Use ";" to separate db.enabledEngines.
change the waiting time to 10 seconds and attempts to 600 for the hazelcast.
Add the new property dbquery.enabledEngines=pathquery,solr.
Add the waiting time and max attempts for regenerating the index.
Add a property containing the list of the resource map namespaces.
Add some new properties for the http solr server.
initialize the SOLR home directory if it does not already exist.
Use 2.0.7 version number in configuration/upgrade/docs (trunk, even though we will not be releasing 2.0.7 from trunk, we want to have the upgrade scripts included here)
change the solr end point to be the localhost.
Add some new properties for the solr.
Add a solr.homeDir property.
include the objectPathMap name in the metacat.properties (used by metacat-index component) https://projects.ecoinformatics.org/ecoinfo/issues/5909
use new, dedicated, LTER test account for LDAP referral test.
use correct default ezid service baseURL
register DOIs with minimal DataCite metadata. still need to determine which details to include and when, but the plumbing is in place as we refine those rules. http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5513
disable EZID/DOI minting by default since we do not yet have a means of tracking minted DOIs and augmenting metadata for them when we actually receive the object in a subsequent create() or update() call. http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5753
ready Metacat for 2.0.6 release (docs, db version, build files etc).
use test doi shoulder as the default for local server, at least during testing phase. http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5755
use ObjectFormatInfo libclient utility to look up mimeType and filename extension during get() calls. Configurable mapping file is deployed by default to /var/metacat/dataone where it can then be augmented as needed. This location is controlled in the metacat.properties file (which is injected into the DataONE Settings values during weapp intitialization)....
integrate ecoinformatics login with the CIlogon identity mapping flow so that a user is directed through the process with no manual navigation needed (at least in the url bar). https://redmine.dataone.org/issues/1480
shorten the systemmetadata* table names for Oracle's 30 character limit. move version to 2.0.5. http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5717
prep for 2.0.4 release
prep for 2.0.3 release
remove semtools skin as a configured skin -- will need to add that if we ever get back to deploying a semtools instance.
configurable replication client timeouthttps://redmine.dataone.org/issues/3078
prep for 2.0.2 release by updating the version numbers.
include dataone.ore.downloaddata as a configurable property in case MNs (like LTER) want to have the process download externally-stored data files described in an EML data package.
update for 2.0.1 upgrade -- scripts, docs, readme
use production CN url as the default (instead of cn-dev)
remove unused replication properties (all certificate file based now)
include test shoulder defaults for KNB, PISCO and LTER
use metacat.properties to specify the default checksum algorithm to use -- this way it will be easy for us to switch to whatever DataONE decrees. https://redmine.dataone.org/issues/2834
Set the default maximum number of database connections back to 200. After discussion, we've decided it will be better to increase the PostgreSQL limit to 300 and keep Metacat's pool size pretty big.
Change the database.maximumConnections property to 100. PostgreSQL's docs says it can handle "a few hundred", and would need to be increased from the default 100 max_connections. For DataONE optimization, we increase max_connections, however there are more processes making connections other than metacat, so I'll reduce metacat's default share.
default replication policy set to 0.
Roll back the nodeId default to blank (used to indicate registration on new installs - thanks Matt.)
Add a default nodeId in metacat.properties of 'urn:node:METACAT1' as a placeholder that needs to be changed on configuration.
Globally change the property 'dataone.memberNodeId' to 'dataone.nodeId'. This is more useful for both MNs and CNs implemented in Metacat. Also, change D1NodeService.getLogRecords() to return log entries with the actual node id rather than the IP address (looks like a cut/paste error)....
dataone configuration and registration enhancements:-include flag to disable D1 services, currently only the MN side enforces this-do not allow multiple registration attempts if we have just submitted and are awaiting Node verification by the CN.-do not allow configuration "bypass" if D1 settings have been configured previously....
Modify admin configuration to include default replication policy. Extensively revised the DataONE configuration page, including new wording for intro, improved tooltips throughout, new arrangement of sections, and other cosmetic changes.
include flag indicating that system metadata generation has completed (useful for independent long-running thread)
Add the dataone.contactSubject property to be used when getCapabilities() returns the MN node instance.
generate ORE maps only once -- and persist the flag to the main backup properties so that subsequent Metacat upgrades remember this value.
Added DOI generation to the 2.0.0 upgrade process. To succeed, this script must be run on a fresh 2.0.0 database, or on a 1.9.5 version database, as those are the only ways to get the needed foreign keys to be marked as deferrable. The identifier conversion must be turned on by setting correct properties in metacat.properties. See the comments in GenerateGlobalIdentifiers for details. By default, conversion is set to false in the properties file. If you want to convert an instance to use DOIs, be sure to set metacat.properties up BEFORE running through the Metacat configuration and database upgrade.
Add the dataone.hazelcast.storageCluster.identifiersSet property that defines the name of the distributed set of unique DataONE identifiers (called 'hzIdentifiers'). This ISet can't be configured in the hazelcast.xml file (only maps and queues can).
simple tests for 2-way data replicationhttp://bugzilla.ecoinformatics.org/show_bug.cgi?id=5520
change the default scheduler and workflow run engine urls.
look-up sych schedule from metacat properties instead of hardcoding themhttps://redmine.dataone.org/issues/1933
use /cn/xslt/ for the standard cn deployment
lookup stylesheet from metacat.properties for CN list objects and list formats. This is used in conjunction with the CN rest service deployment where the xslt is actually kept.
DataONE MN registration/configuration is now its own configuration page in the admin interface.http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5531
started replication unit test
use HttpClient to set up SSL connection when doing replication calls -- this will use the server's configured certificate as the client certificate on the request. The server it is calling can then inspect that certificate and decide whether or not it trusts the caller.
check client-provided certificate when servicing ReplicationServlet requests.
Configure and use CertificateManager in order to act as the MN when performing replicate() and getReplica() mthods.
change upgrade scripts/routines to use 2.0.0 version number instead of 1.10.0
Fix service versioning in metacat.properties so that when Metacat nodes are registered they report the correct service version for use in resolve() -- i.e., use "v1" instead of "0.6.4" in resolve urls.
Fixed the D1 URL to the Coordinating Node that had been missing /cn in the path.
use Settings augmentation to customize the D1Client.CN_URL when used in a Metacat deployment.
Updating CN property to use cn-dev.
Pull the certificate path for dataone certificates from the metacat properties file.
comment out the external hazelcast config location for default building
New default properties for dataone MN properties.
Set the correct cluster name and password for the D1 Hazelcast process cluster. This will likely change for production deployments.
use shared hazelcast properties location on the CN
Add properties for strings used to connect to the DataONE storage and process Hazelcast clusters. The storage cluster is defined in the hazelcast.xml config file, and we list the names of the systemMetadataMap and the tasksIdNamespace in order to connect to these structures. When creating CNReplicationTask objects, use the task id generator to get a unique task id.
Add the dataone.hazelcast.configFilePath property to be used when MetacatServlet becomes a Hazelcast cluster member using HazelcastService. TODO: what are the consequences of restarting the service via doRefresh()? It's set to false for now.
remove unused dataone properties
use 0.6.4 for node capabilities
Use CamelCaps for properties for better readability.
Added node synchronization and replication properties.
Added service availability properties for each of the MN services.
Added service version properties for each of the MN services.
Add a dataone.serviceName property (set to 'd1') to remove hardcoded 'd1' strings in the codebase.
include different (non pisco) referral account (merge from 1.9.5 branch)include upgrade sql scripts
Merged in the D1_0_6_2_BRANCH changes that include the transition from ObjectFormat calls to ObjectFormatCache calls.
add additionalMetadata/metadata/spatialResolution/value and additionalMetadata/metadata/spatialResolution/units for indexing. Also add missing comma behind additionalMetadata/moderatorComment
Removed hardcoded D1 node type in ResourceHandler and added in a new 'dataone.nodeType' property. Also added 'dataone.coordinatingNodeBaseURL' property which points to the CN that stores the authoritative object format list. If this instance of Metacat is a CN, it may point to itself.
add support for temporal element query in pathqueryhttp://bugzilla.ecoinformatics.org/show_bug.cgi?id=2084
add otherEntity physical elements to indexPaths (ndp)
add paths to indexPaths, and kar namespaces to indexNamespaces for Kepler Workflow Run Manager run-kar queries
include temporal coverage dates in the path index - (for opendap metadata queries)
new properties for d1 MN
include default password for geoserver admin - though metacat need not really know what it is.
rework geoserver configuration:-geoserver context is set to 'geoserver' by default, but can be reconfigured-data directory is set in the geoserver web.xml file (we have a template, set the value accordingly, then overwrite the deployed version in the geoserver webapp)...
Add authorizationPath,authenticationPath and queryPath for workflow scheduler.
Add a new property - destination reporsitory name in execution engine part.
include value elements for translations in EML 2.1.1