Metacat

allow SM resynch to be executed any time, not just during start up.
https://redmine.dataone.org/issues/3116

change to debug log level when processing shared/local pids)

only lock the missing pid event if we know we have it locally to contribute.
https://redmine.dataone.org/issues/3117

Add locking to the itemAdded() method so ideally only one CN will respond to the request for a 'wanted' pid from the cluster. The lock is on a string, not the pid, and so won't conflict with system metadata locking. The string is based on the pid, with "missing-" as a prefix.

only publish to the missing pid "wanted list" when resynching system metadata. we were seeing redundant entry added/updated events when looking up the shared systemmetadata first.

print the missing pid count, not the total shared pid count so we know how many will be processed.

change the system metadata resynch approach: nodes will publish PIDs that they are missing after inspecting the shared identifier set. other nodes will be listening for the "wanted" pids and will put their local copy of SystemMetadata on the shared SM map. This should dramatically decrease the hazelcast chatter during a resynch and targets only the pids that are missing from any of the various nodes.

logging for processing identifier set on restart.

remove possibility for infinite loop in case data replication is not configured for the server and a data file is encountered (yikes!)

added logging debug statements to see where the replication timeout might be occurring.