Activity
From 06/03/2013 to 07/02/2013
07/02/2013
- 04:47 PM Revision 7840 (metacat): support a "force replication delete all action" during replication. This is used when we want Metacat to remove the content from the other target replicas because the DataONE delete() action was called (more powerful than just "archive").
- 04:41 PM Revision 7839 (metacat): add simple test for the IndexEventDAO class -- adding, removing, listing events in the DB table. https://projects.ecoinformatics.org/ecoinfo/issues/5944
- 03:21 PM Revision 7838 (metacat): Add the code that only the ids with the correct system metadata modification time will be added to the index queue.
- 12:38 PM Task #6004: Figure out why there are only 422 documents indexed in mn-demo-4.test.dataone.org
- I queried the database and got the following result. There are 4998 in the xml_documents and 0 in the xml_revisions.
... - 12:18 PM Task #5924 (Closed): Add the configuration page for the Solr server in the Metacat configuration utility
- 12:18 PM Task #5924: Add the configuration page for the Solr server in the Metacat configuration utility
- Done. Ben added to main property page.
- 11:25 AM Task #5945: Implement reindex action for individual PID
- Proposal is:
-create Metacat action "reindex" that takes an identifier.
-Metacat places the SystemMetadata for that...
07/01/2013
- 03:09 PM Revision 7837 (metacat): Use the hazelcast event log.
- 01:59 PM Revision 7836 (metacat): Add code to get and set the last process date.
06/26/2013
- 04:43 PM Revision 7834 (metacat): merging upgrade scripts from 2.0 branch to trunk. https://redmine.dataone.org/issues/3847
- 04:42 PM Revision 7833 (metacat): scripts for 2.1.0 upgrade
06/25/2013
- 04:42 PM Revision 7830 (metacat): upgrade to Metacat 2.1.0 on the trunk. This includes a new index_event table for storing indexing events that need to be reprocessed. https://projects.ecoinformatics.org/ecoinfo/issues/5944
- 02:51 PM Task #5915 (Closed): Use MN.query(solr) in NCEAS skin
- 02:50 PM Task #5918 (Closed): Separate unit tests from integration tests
- 02:43 PM Bug #6007 (Resolved): Registry uses filename from upload form to save in temporary directory
- I deployed this in KNB today (June 25th, 2013). This was not part of an official release but is in the code for 2.1.x...
- 01:23 PM Revision 7829 (metacat): stub for storing IndexEvent objects in Metacat (from metacat-index processing). https://projects.ecoinformatics.org/ecoinfo/issues/5944
- 01:23 PM Revision 7828 (metacat): move IndexEvent into metacat-common. Perparation for Metacat responding to events and writing them to a persistent store. https://projects.ecoinformatics.org/ecoinfo/issues/5944
- 01:17 PM Revision 7827 (metacat): do not force a get() during refresh (causing EML-defined data access rules to be lost when inserting EML docs about data files). note that this reverses a change that was meant to trigger indexing, but now we are using a new queue to share index events with metacat-index and so should not be necessary.
- 09:10 AM Bug #6009 (Closed): Metacat uses "metacat.output" temp files for all read() operations
- This is in trunk now.
06/24/2013
- 12:07 PM Bug #6009 (In Progress): Metacat uses "metacat.output" temp files for all read() operations
- now reading directly from the filesystem, or in the case of EML with inline-data, the partial file contents.
- 12:05 PM Revision 7824 (metacat): do not use tmp file to return an inputstream on read() operations - just read from the file we already have. https://projects.ecoinformatics.org/ecoinfo/issues/6009
06/21/2013
- 09:47 PM Bug #6008 (Closed): Metacat servlet API upload() method uses user-supplied filename for temp upload
- Now using File.createTempFile() and deleting the data files when done.
- 05:32 PM Bug #6008 (Closed): Metacat servlet API upload() method uses user-supplied filename for temp upload
- This can be problematic if users simultaneously attempt to upload a data file with the same name.
- 09:44 PM Revision 7823 (metacat): use standard File.createTempFile() method for uploaded data files and delete them when we are done with them. https://projects.ecoinformatics.org/ecoinfo/issues/6008
- 05:35 PM Bug #6009 (Closed): Metacat uses "metacat.output" temp files for all read() operations
- The KNB has 1.3 GB of these that have accumulated since ?.
There should be two scenarios:
-stream the contents fr... - 05:02 PM Bug #6007: Registry uses filename from upload form to save in temporary directory
- I deployed the changes on dev2.nceas for testing. Feel pretty good about them - using File::Temp to manager unique up...
- 12:22 PM Bug #6007 (Resolved): Registry uses filename from upload form to save in temporary directory
- This can cause collisions when a room of 22 people all submit a datapackage with the same data file name....
The r... - 04:15 PM Revision 7822 (metacat): correctly delete data file when we are done with it. https://projects.ecoinformatics.org/ecoinfo/issues/6007
- 04:06 PM Revision 7821 (metacat): include filename in the filepart part. https://projects.ecoinformatics.org/ecoinfo/issues/6007
- 03:18 PM Revision 7820 (metacat): send the original filename in the upload() method since that is supported by the Metacat API. https://projects.ecoinformatics.org/ecoinfo/issues/6007
- 03:10 PM Revision 7819 (metacat): remove the unique string when generating data file metadata. https://projects.ecoinformatics.org/ecoinfo/issues/6007
- 02:48 PM Revision 7818 (metacat): debugging. https://projects.ecoinformatics.org/ecoinfo/issues/6007
- 01:44 PM Revision 7817 (metacat): use File::Temp to write data files in registry. https://projects.ecoinformatics.org/ecoinfo/issues/6007
- 11:11 AM Revision 7816 (metacat): correct regex for whitespace in D1 identifier.
- 09:09 AM Revision 7815 (metacat): refactor IndexEventLog a bit to simplify type/action information. prep for serializing IndexEvent objects to Metacat. https://projects.ecoinformatics.org/ecoinfo/issues/5944
06/20/2013
- 05:28 PM Revision 7814 (metacat): remove serial number from indexeventlog - it is not used elsewhere in the api. https://projects.ecoinformatics.org/ecoinfo/issues/5944
- 05:28 PM Bug #6006 (Closed): Metacat admin screen requires metacat-index directories
- Now skipping the template creation if metacat-index war is not deployed along with metacat.
- 05:21 PM Revision 7813 (metacat): correct spelling for index.eventlog.classname property
- 04:59 PM Task #5943 (Resolved): Eliminate the unnecessary Systemmetadata event between the Metacat and Metacat-index modules.
- Switched over to the new structure for communicating index events. Seems to be working as expected.
- 04:49 PM Revision 7812 (metacat): use an independent ISet<SystemMetadata> structure to communicate objects that should be indexed by metacat-index. https://projects.ecoinformatics.org/ecoinfo/issues/5943
- 12:23 PM Revision 7811 (metacat): consolidate SystemMetadata map retrieval in preparation for using a different structure for objects to index.
- 10:22 AM Task #5944: Metacat-index need an IndexEvent and IndexEventLog mechanism
- Added delete() method to remove entries that we do not need to reprocess again (because they were successful, finally...
- 09:24 AM Revision 7810 (metacat): adding ability to remove event from the [error] queue.
06/18/2013
- 02:59 PM Revision 7809 (metacat): do not create solr-home if there is no template to compy into that directory (need to be able to create it later if/when someone decides to use and deploy metacat-index). https://projects.ecoinformatics.org/ecoinfo/issues/6006
- 02:54 PM Task #5943: Eliminate the unnecessary Systemmetadata event between the Metacat and Metacat-index modules.
- Proposed solution is to use a different/independent structure for SystemMetadata events that should trigger a re-inde...
- 01:54 PM Revision 7808 (metacat): do not attempt to copy solr-home template from metacat-index webapp if it does not exist. This would be in cases where metacat-index is not deployed. https://projects.ecoinformatics.org/ecoinfo/issues/6006
- 12:35 PM Bug #6006 (Closed): Metacat admin screen requires metacat-index directories
- When configuring Metacat there is a dependency on metacat-index. This should be gracefully skipped if that webapp is ...
- 10:19 AM Task #6005 (Closed): Incomplete solr index for the data objects if the resourcemap was indexed first
- Currently the resource map should be indexed after the data objects in the resource map were indexed. If the resource...
06/17/2013
- 05:07 PM Revision 7807 (metacat): Add code to implment set and get the last processed date.
- 04:28 PM Revision 7806 (metacat): It will make the index only for those objects which were modified after the marked time.
- 01:50 PM Revision 7805 (metacat): Add set and get the lastprocessedDate in the IndexEventLog.
- Remove the code to write the successful event.
- 10:31 AM Task #6004: Figure out why there are only 422 documents indexed in mn-demo-4.test.dataone.org
- Attached a log file for the indexing process.
- 10:30 AM Task #6004 (Resolved): Figure out why there are only 422 documents indexed in mn-demo-4.test.dataone.org
- There are more 4000 data objects in the mn-demo-4. But we only gets 422 documents indexed.
06/13/2013
- 05:39 PM Bug #6003 (Closed): The "war" target in the metacat will build the metacat-index.war
- We added the target which build metacat-index.war as the dependency of the "war" target.
- 05:37 PM Bug #6003 (Closed): The "war" target in the metacat will build the metacat-index.war
- The "war" target in the metacat doesn't build the metacat-index.war in the dist directory. It only builds knb.war and...
- 05:35 PM Task #5930: Add a configuration property to turn on/off the indexing processes (SOLR indexing)
- If we removed the "solr" from the engine list, metacat-index will not build index anymore. And the solr query will be...
- 05:20 PM Revision 7804 (metacat): Add the dataone repository.
- 04:41 PM Revision 7803 (metacat): The "war" target will build the metacat-index.war as well.
06/12/2013
- 02:36 PM Revision 7802 (metacat): Log the timed index jobs.
- 02:08 PM Revision 7801 (metacat): Add the code to log the failed events.
06/11/2013
- 06:24 PM Revision 7800 (metacat): Add a temporary file log for debugging.
- 06:23 PM Revision 7799 (metacat): Use commons-io 2.4
- 03:47 PM Revision 7798 (metacat): Add a new property for the index event log class name.
06/07/2013
- 04:38 PM Revision 7797 (metacat): Add a serial number for the event.
- Add method to set events to be archived.
- 11:56 AM Feature #5936: Include certificate delegation inside Metacat
- There's now the package "edu.ucsb.nceas.metacat.portal" that does the authentication delegation. You start a request ...
- 11:22 AM Revision 7796 (metacat): Add a new class variable - isArchived for class IndexEvent.
06/06/2013
- 04:13 PM Revision 7795 (metacat): Update the documentation about those classes.
- 03:48 PM Bug #5997 (Closed): Restrict KNB trusted CAs
- Commented out the line that includes all CAs in /etc/ssl/certs and reloaded Apache. Now we are down to the short list...
- 03:17 PM Revision 7794 (metacat): Add a event and eventlog for the index.
06/05/2013
- 03:03 PM Revision 7793 (metacat): Use the identifier set to get the list of ids in the member node.
- 02:20 PM Revision 7792 (metacat): The returned ISet should be Identifier.
- 01:50 PM Revision 7791 (metacat): Add code to test getIdentifierSet method.
- 01:50 PM Revision 7790 (metacat): Add method to get identifier set.
- 11:02 AM Bug #5997: Restrict KNB trusted CAs
- We should be able to simply use the DataONE chain file that we have installed already:...
- 11:00 AM Bug #5997 (Closed): Restrict KNB trusted CAs
- Instead of trusting all commercial CAs, the KNB Member Node should only trust the DataONE and CILogon certificate aut...
06/04/2013
- 02:49 PM Revision 7789 (metacat): Add a new property to specify the interval of a Timer to run the thread generating solr index.
- 02:47 PM Revision 7788 (metacat): Set up a Timer to run the regenerating solr index task periodically.
06/03/2013
- 05:36 PM Task #5922 (Closed): Modify the solr index configuration for the resource map
- We rewrote the ResourceMapSubprocessor class which now uses the SolrServer.
- 05:22 PM Task #5937 (Closed): The solr indexes of the data file contains obsoleted ids for the resourcemap and documentBy element after updating a data package
- Add code to modify the "documents", "resourceMap" and "documentedBy" element in the solr doc when archive happens.
- 04:35 PM Revision 7787 (metacat): Use the ";" as the seperator to replace "," in the resource name spaces.
Also available in: Atom