Activity
From 05/27/2013 to 06/25/2013
06/25/2013
- 04:42 PM Revision 7830 (metacat): upgrade to Metacat 2.1.0 on the trunk. This includes a new index_event table for storing indexing events that need to be reprocessed. https://projects.ecoinformatics.org/ecoinfo/issues/5944
- 02:51 PM Task #5915 (Closed): Use MN.query(solr) in NCEAS skin
- 02:50 PM Task #5918 (Closed): Separate unit tests from integration tests
- 02:43 PM Bug #6007 (Resolved): Registry uses filename from upload form to save in temporary directory
- I deployed this in KNB today (June 25th, 2013). This was not part of an official release but is in the code for 2.1.x...
- 01:23 PM Revision 7829 (metacat): stub for storing IndexEvent objects in Metacat (from metacat-index processing). https://projects.ecoinformatics.org/ecoinfo/issues/5944
- 01:23 PM Revision 7828 (metacat): move IndexEvent into metacat-common. Perparation for Metacat responding to events and writing them to a persistent store. https://projects.ecoinformatics.org/ecoinfo/issues/5944
- 01:17 PM Revision 7827 (metacat): do not force a get() during refresh (causing EML-defined data access rules to be lost when inserting EML docs about data files). note that this reverses a change that was meant to trigger indexing, but now we are using a new queue to share index events with metacat-index and so should not be necessary.
- 09:10 AM Bug #6009 (Closed): Metacat uses "metacat.output" temp files for all read() operations
- This is in trunk now.
06/24/2013
- 12:07 PM Bug #6009 (In Progress): Metacat uses "metacat.output" temp files for all read() operations
- now reading directly from the filesystem, or in the case of EML with inline-data, the partial file contents.
- 12:05 PM Revision 7824 (metacat): do not use tmp file to return an inputstream on read() operations - just read from the file we already have. https://projects.ecoinformatics.org/ecoinfo/issues/6009
06/21/2013
- 09:47 PM Bug #6008 (Closed): Metacat servlet API upload() method uses user-supplied filename for temp upload
- Now using File.createTempFile() and deleting the data files when done.
- 05:32 PM Bug #6008 (Closed): Metacat servlet API upload() method uses user-supplied filename for temp upload
- This can be problematic if users simultaneously attempt to upload a data file with the same name.
- 09:44 PM Revision 7823 (metacat): use standard File.createTempFile() method for uploaded data files and delete them when we are done with them. https://projects.ecoinformatics.org/ecoinfo/issues/6008
- 05:35 PM Bug #6009 (Closed): Metacat uses "metacat.output" temp files for all read() operations
- The KNB has 1.3 GB of these that have accumulated since ?.
There should be two scenarios:
-stream the contents fr... - 05:02 PM Bug #6007: Registry uses filename from upload form to save in temporary directory
- I deployed the changes on dev2.nceas for testing. Feel pretty good about them - using File::Temp to manager unique up...
- 12:22 PM Bug #6007 (Resolved): Registry uses filename from upload form to save in temporary directory
- This can cause collisions when a room of 22 people all submit a datapackage with the same data file name....
The r... - 04:15 PM Revision 7822 (metacat): correctly delete data file when we are done with it. https://projects.ecoinformatics.org/ecoinfo/issues/6007
- 04:06 PM Revision 7821 (metacat): include filename in the filepart part. https://projects.ecoinformatics.org/ecoinfo/issues/6007
- 03:18 PM Revision 7820 (metacat): send the original filename in the upload() method since that is supported by the Metacat API. https://projects.ecoinformatics.org/ecoinfo/issues/6007
- 03:10 PM Revision 7819 (metacat): remove the unique string when generating data file metadata. https://projects.ecoinformatics.org/ecoinfo/issues/6007
- 02:48 PM Revision 7818 (metacat): debugging. https://projects.ecoinformatics.org/ecoinfo/issues/6007
- 01:44 PM Revision 7817 (metacat): use File::Temp to write data files in registry. https://projects.ecoinformatics.org/ecoinfo/issues/6007
- 11:11 AM Revision 7816 (metacat): correct regex for whitespace in D1 identifier.
- 09:09 AM Revision 7815 (metacat): refactor IndexEventLog a bit to simplify type/action information. prep for serializing IndexEvent objects to Metacat. https://projects.ecoinformatics.org/ecoinfo/issues/5944
06/20/2013
- 05:28 PM Revision 7814 (metacat): remove serial number from indexeventlog - it is not used elsewhere in the api. https://projects.ecoinformatics.org/ecoinfo/issues/5944
- 05:28 PM Bug #6006 (Closed): Metacat admin screen requires metacat-index directories
- Now skipping the template creation if metacat-index war is not deployed along with metacat.
- 05:21 PM Revision 7813 (metacat): correct spelling for index.eventlog.classname property
- 04:59 PM Task #5943 (Resolved): Eliminate the unnecessary Systemmetadata event between the Metacat and Metacat-index modules.
- Switched over to the new structure for communicating index events. Seems to be working as expected.
- 04:49 PM Revision 7812 (metacat): use an independent ISet<SystemMetadata> structure to communicate objects that should be indexed by metacat-index. https://projects.ecoinformatics.org/ecoinfo/issues/5943
- 12:23 PM Revision 7811 (metacat): consolidate SystemMetadata map retrieval in preparation for using a different structure for objects to index.
- 10:22 AM Task #5944: Metacat-index need an IndexEvent and IndexEventLog mechanism
- Added delete() method to remove entries that we do not need to reprocess again (because they were successful, finally...
- 09:24 AM Revision 7810 (metacat): adding ability to remove event from the [error] queue.
06/18/2013
- 02:59 PM Revision 7809 (metacat): do not create solr-home if there is no template to compy into that directory (need to be able to create it later if/when someone decides to use and deploy metacat-index). https://projects.ecoinformatics.org/ecoinfo/issues/6006
- 02:54 PM Task #5943: Eliminate the unnecessary Systemmetadata event between the Metacat and Metacat-index modules.
- Proposed solution is to use a different/independent structure for SystemMetadata events that should trigger a re-inde...
- 01:54 PM Revision 7808 (metacat): do not attempt to copy solr-home template from metacat-index webapp if it does not exist. This would be in cases where metacat-index is not deployed. https://projects.ecoinformatics.org/ecoinfo/issues/6006
- 12:35 PM Bug #6006 (Closed): Metacat admin screen requires metacat-index directories
- When configuring Metacat there is a dependency on metacat-index. This should be gracefully skipped if that webapp is ...
- 10:19 AM Task #6005 (Closed): Incomplete solr index for the data objects if the resourcemap was indexed first
- Currently the resource map should be indexed after the data objects in the resource map were indexed. If the resource...
06/17/2013
- 05:07 PM Revision 7807 (metacat): Add code to implment set and get the last processed date.
- 04:28 PM Revision 7806 (metacat): It will make the index only for those objects which were modified after the marked time.
- 01:50 PM Revision 7805 (metacat): Add set and get the lastprocessedDate in the IndexEventLog.
- Remove the code to write the successful event.
- 10:31 AM Task #6004: Figure out why there are only 422 documents indexed in mn-demo-4.test.dataone.org
- Attached a log file for the indexing process.
- 10:30 AM Task #6004 (Resolved): Figure out why there are only 422 documents indexed in mn-demo-4.test.dataone.org
- There are more 4000 data objects in the mn-demo-4. But we only gets 422 documents indexed.
06/13/2013
- 05:39 PM Bug #6003 (Closed): The "war" target in the metacat will build the metacat-index.war
- We added the target which build metacat-index.war as the dependency of the "war" target.
- 05:37 PM Bug #6003 (Closed): The "war" target in the metacat will build the metacat-index.war
- The "war" target in the metacat doesn't build the metacat-index.war in the dist directory. It only builds knb.war and...
- 05:35 PM Task #5930: Add a configuration property to turn on/off the indexing processes (SOLR indexing)
- If we removed the "solr" from the engine list, metacat-index will not build index anymore. And the solr query will be...
- 05:20 PM Revision 7804 (metacat): Add the dataone repository.
- 04:41 PM Revision 7803 (metacat): The "war" target will build the metacat-index.war as well.
06/12/2013
- 02:36 PM Revision 7802 (metacat): Log the timed index jobs.
- 02:08 PM Revision 7801 (metacat): Add the code to log the failed events.
06/11/2013
- 06:24 PM Revision 7800 (metacat): Add a temporary file log for debugging.
- 06:23 PM Revision 7799 (metacat): Use commons-io 2.4
- 03:47 PM Revision 7798 (metacat): Add a new property for the index event log class name.
06/07/2013
- 04:38 PM Revision 7797 (metacat): Add a serial number for the event.
- Add method to set events to be archived.
- 11:56 AM Feature #5936: Include certificate delegation inside Metacat
- There's now the package "edu.ucsb.nceas.metacat.portal" that does the authentication delegation. You start a request ...
- 11:22 AM Revision 7796 (metacat): Add a new class variable - isArchived for class IndexEvent.
06/06/2013
- 04:13 PM Revision 7795 (metacat): Update the documentation about those classes.
- 03:48 PM Bug #5997 (Closed): Restrict KNB trusted CAs
- Commented out the line that includes all CAs in /etc/ssl/certs and reloaded Apache. Now we are down to the short list...
- 03:17 PM Revision 7794 (metacat): Add a event and eventlog for the index.
06/05/2013
- 03:03 PM Revision 7793 (metacat): Use the identifier set to get the list of ids in the member node.
- 02:20 PM Revision 7792 (metacat): The returned ISet should be Identifier.
- 01:50 PM Revision 7791 (metacat): Add code to test getIdentifierSet method.
- 01:50 PM Revision 7790 (metacat): Add method to get identifier set.
- 11:02 AM Bug #5997: Restrict KNB trusted CAs
- We should be able to simply use the DataONE chain file that we have installed already:...
- 11:00 AM Bug #5997 (Closed): Restrict KNB trusted CAs
- Instead of trusting all commercial CAs, the KNB Member Node should only trust the DataONE and CILogon certificate aut...
06/04/2013
- 02:49 PM Revision 7789 (metacat): Add a new property to specify the interval of a Timer to run the thread generating solr index.
- 02:47 PM Revision 7788 (metacat): Set up a Timer to run the regenerating solr index task periodically.
06/03/2013
- 05:36 PM Task #5922 (Closed): Modify the solr index configuration for the resource map
- We rewrote the ResourceMapSubprocessor class which now uses the SolrServer.
- 05:22 PM Task #5937 (Closed): The solr indexes of the data file contains obsoleted ids for the resourcemap and documentBy element after updating a data package
- Add code to modify the "documents", "resourceMap" and "documentedBy" element in the solr doc when archive happens.
- 04:35 PM Revision 7787 (metacat): Use the ";" as the seperator to replace "," in the resource name spaces.
05/31/2013
- 03:29 PM Revision 7786 (metacat): Add code to handle delete data package information when delete a pid in the solr index.
- 02:05 PM Revision 7785 (metacat): Add two static methods to get the SystemMetadata and data object InputStream for the specified id.
05/28/2013
- 05:19 PM Revision 7784 (metacat): Change the code since the ApplicationController's constructor was changed.
- 05:18 PM Revision 7783 (metacat): Add code to check if the metacat.properties is available.
- 01:21 PM Revision 7782 (metacat): If solr is not enabled, it would not be running.
- 01:20 PM Revision 7781 (metacat): Solr will be enabled if it is in the db.enabledEngines.
- 01:18 PM Revision 7780 (metacat): Use ";" to separate db.enabledEngines.
Also available in: Atom