Activity
From 06/11/2013 to 07/10/2013
07/10/2013
- 04:32 PM Revision 7877 (metacat): Besides the getArchvied() method, the getObsoletedBy method was added to determine if the object is archvied or not.
- 04:14 PM Revision 7876 (metacat): Add code to handle deleted ids.
- 03:47 PM Revision 7875 (metacat): use fluid layout from metacatui/bootstrap css. https://projects.ecoinformatics.org/ecoinfo/issues/5951
- 01:12 PM Bug #6035: The system metadata table in a Metacat was messed up with the value of archived
- The archive flag need not be set to true when there is an obsoletedBy value present. Either the obsoletedBy OR the ar...
- 11:19 AM Bug #6035 (Rejected): The system metadata table in a Metacat was messed up with the value of archived
- In the mn-demo-4.test.dataone.org, I queried the systemmetadata table and got:
metacat=> select count(*) from syst... - 01:09 PM Task #6004: Figure out why there are only 422 documents indexed in mn-demo-4.test.dataone.org
- I did couple queries and found the archived value in the systemmetadata was messed up. Please see the bug:
https://p... - 11:29 AM Revision 7874 (metacat): remove custom classes from login form, simplify template, move js functions to separate file. prep for UI-refresh. https://projects.ecoinformatics.org/ecoinfo/issues/5951
07/09/2013
- 04:46 PM Revision 7873 (metacat): Stubs for 'metacatui' rendering. Initial target is to support Perl registry styling, but also general structured html views on metadata. https://projects.ecoinformatics.org/ecoinfo/issues/5951
- 04:46 PM Revision 7872 (metacat): Stubs for 'metacatui' rendering. Initial target is to support Perl registry styling, but also general structured html views on metadata. https://projects.ecoinformatics.org/ecoinfo/issues/5951
- 04:46 PM Revision 7871 (metacat): Stubs for 'metacatui' rendering. Initial target is to support Perl registry styling, but also general structured html views on metadata. https://projects.ecoinformatics.org/ecoinfo/issues/5951
- 04:46 PM Revision 7870 (metacat): Stubs for 'metacatui' rendering. Initial target is to support Perl registry styling, but also general structured html views on metadata. https://projects.ecoinformatics.org/ecoinfo/issues/5951
- 04:45 PM Revision 7869 (metacat): Stubs for 'metacatui' rendering. Initial target is to support Perl registry styling, but also general structured html views on metadata. https://projects.ecoinformatics.org/ecoinfo/issues/5951
- 04:12 PM Revision 7868 (metacat): Include the 'metacatui' portion of the path to .js file. https://projects.ecoinformatics.org/ecoinfo/issues/5951
- 04:10 PM Revision 7867 (metacat): Initial copy of original Perl templates - before making any changes. https://projects.ecoinformatics.org/ecoinfo/issues/5951
- 03:41 PM Task #6004: Figure out why there are only 422 documents indexed in mn-demo-4.test.dataone.org
- After we modified the SystemMetadataEventListener class to listen to a dedicated map for indexing, I indexed the meta...
- 02:55 PM Task #5921 (Closed): Design and implement Metacat-Index module
- 02:54 PM Task #5921: Design and implement Metacat-Index module
- All sub-tasks were done. Close the feature.
- 02:52 PM Task #5923 (Closed): Discover missed documents and queue them for indexing
- We create a HazelcastEventLog class.
When a timed indexing finishes, it will set a property in a file. The file co... - 12:41 PM Task #5923 (In Progress): Discover missed documents and queue them for indexing
- This is well-underway, mostly just testing (I believe).
- 02:50 PM Bug #5812: Rework default skin[s] to use SOLR query
- TBD: actually do the same thing we did for the NCEAS skin (qformat=nceas) for the standard Metacat default skin (qfor...
- 02:48 PM Story #5811 (Closed): Redesign KNB look and feel
- See the entire MetacatUI project for details. https://projects.ecoinformatics.org/ecoinfo/projects/metacatui
- 02:46 PM Task #6005 (Closed): Incomplete solr index for the data objects if the resourcemap was indexed first
- When Metacat-index indexes a resource map document, it will check if all data objects of the resource map have been i...
- 02:45 PM Story #5940 (Closed): Support session-based Metacat authentication (for D1 services)
- 01:43 PM Story #5940 (In Progress): Support session-based Metacat authentication (for D1 services)
- 02:38 PM Task #5941 (Rejected): Add login/logout methods to DataONE MN service
- These methods don't really fit with our current model. I think it's fine to allow the "back door" (i.e., Metacat serv...
- 02:37 PM Revision 7866 (metacat): Use schedule method to start the index.
- 02:32 PM Feature #6034 (New): Simplify configuration for certificate delegation
- So we don't loose this in the shuffle - we could probably fill out some of the MyProxy configuration values based on ...
- 02:28 PM Feature #5936 (Closed): Include certificate delegation inside Metacat
- 01:41 PM Feature #5936 (In Progress): Include certificate delegation inside Metacat
- Aside from making the myProxy configuration easier, this is done. It does require a bit of administration from regist...
- 02:25 PM Bug #5395: Get unexpected result if a search was done before metacat finishes its indexing during the inserting process.
- Deferring - though we don't really have a solution for this. Only affects a deprecated mechanism (pathquery), however.
- 02:17 PM Feature #5910 (In Progress): Build index from scratch
- 02:16 PM Bug #2792 (Rejected): KNB home link needed on the Map Page
- The entire KNB site is being redone, I don't even know if we will have a full map page!
- 01:59 PM Task #5930 (Closed): Add a configuration property to turn on/off the indexing processes (SOLR indexing)
- 01:58 PM Feature #6033 (Closed): Add configuration to disable 'pathquery'
- As we move toward SOLR-based query support, pathquery will play less of a role in Metacat. We should be able to disab...
- 01:56 PM Feature #5913 (In Progress): Handle indexing errors
- Remains to be seen how quickly this queue gets populated. We dont currently have any qualifiers on the kind of error ...
- 01:53 PM Feature #5914 (Closed): Reconcile SOLR server/client versions
- Both client and server are using v3.x now. Eventually DataONE plans to update the SOLR version, so we can consider t...
- 01:46 PM Feature #5911 (In Progress): Repair index after outage
- This may even be a duplicate of related task #5923
- 01:44 PM Feature #5912 (Closed): Force indexing on individual object[s]
- There is now a Metacat servlet API action for this:...
- 12:43 PM Revision 7865 (metacat): use tagged/released version of d1-portal project. https://projects.ecoinformatics.org/ecoinfo/issues/5936
- 12:38 PM Task #5945 (Closed): Implement reindex action for individual PID
- 12:38 PM Task #5944 (Closed): Metacat-index need an IndexEvent and IndexEventLog mechanism
- Main development on this is complete - enhancements to this can be added in other tasks.
- 10:40 AM Revision 7864 (metacat): include mn.publish() REST endpoint handling. https://projects.ecoinformatics.org/ecoinfo/issues/6024
- 10:16 AM Revision 7863 (metacat): Add the code to write the error message to the log in the itemRemvoed method.
- 09:55 AM Bug #6030 (Closed): Deleting a document causes the method itemAdded(ItemEvent<SystemMetadata> entryEvent) in the SysteMetadataEventListener class being called twice
- I used the morpho to test it and it seems that the issue was fixed.
07/08/2013
- 09:45 PM Feature #6031 (Rejected): Add rendered metadata to Bagit package contents
- 06:20 PM Bug #6030 (In Progress): Deleting a document causes the method itemAdded(ItemEvent<SystemMetadata> entryEvent) in the SysteMetadataEventListener class being called twice
- There were two calls to this - one in the D1 service implementation and another in the Metacat implementation that th...
- 05:23 PM Bug #6030 (Closed): Deleting a document causes the method itemAdded(ItemEvent<SystemMetadata> entryEvent) in the SysteMetadataEventListener class being called twice
- When we delete a document from the Metacat, the method itemAdded in the in the SysteMetadataEventListener class (Meta...
- 06:19 PM Revision 7862 (metacat): comment out the index queue call when archive() is called - I think it is causing the duplicate events for the listener. https://projects.ecoinformatics.org/ecoinfo/issues/6030
- 06:10 PM Revision 7861 (metacat): implement the view service (uses existing skin-based dbtransform) - and include the REST endpoint. https://projects.ecoinformatics.org/ecoinfo/issues/6028
- 05:51 PM Revision 7860 (metacat): use StreamSource instead of StringReader for method signature -- can be used with different sources this way. https://projects.ecoinformatics.org/ecoinfo/issues/6019
- 05:10 PM Revision 7859 (metacat): clean up DBTransform in preparation for "view" service. https://projects.ecoinformatics.org/ecoinfo/issues/6019
- 03:09 PM Revision 7858 (metacat): In determining the time arrange, the equality was removed.
- 03:05 PM Revision 7857 (metacat): Add code to handle failed ids.
- 02:00 PM Revision 7856 (metacat): Remove the EventLog write.
- 12:14 PM Revision 7855 (metacat): include GET /package/{pid} endpoint in MN service. https://projects.ecoinformatics.org/ecoinfo/issues/6027
- 12:09 PM Revision 7854 (metacat): Add the EventLog code.
07/05/2013
- 06:33 PM Revision 7853 (metacat): MN.getPackage() - test with ORE that includes 2 data files and a "metadata" file (still should be using EML for that test). https://projects.ecoinformatics.org/ecoinfo/issues/6026
- 05:14 PM Revision 7852 (metacat): It will throw an exception if the subprocessor can't handle the document.
- 04:59 PM Revision 7851 (metacat): Check if the all components of a resource map have been processed before processing the resource map.
07/04/2013
07/03/2013
- 09:58 PM Revision 7849 (metacat): add method for publishing existing object (usually assumed to be scimeta) with a DOI. https://projects.ecoinformatics.org/ecoinfo/issues/6014
- 04:34 PM Revision 7848 (metacat): Fixed a bug that the event log can't save the real lastest process date.
- 03:56 PM Revision 7847 (metacat): Change the date format.
- Remove the replication part of log4j.
- 10:28 AM Revision 7846 (metacat): Use a new date format.
- 10:25 AM Revision 7845 (metacat): Add a log4j properties file.
- 10:24 AM Revision 7844 (metacat): Add a file to specify the log4j as the logger.
- 10:15 AM Task #5945 (In Progress): Implement reindex action for individual PID
- This is in trunk now. You must be logged in as admin of Metacat in order to invoke it which makes a JUnit test diffic...
- 10:13 AM Revision 7843 (metacat): add Metacat servlet action to force the reindexing of one or more or all pids in the system. https://projects.ecoinformatics.org/ecoinfo/issues/5945
- 09:02 AM Task #5944 (In Progress): Metacat-index need an IndexEvent and IndexEventLog mechanism
- I have the hzIndexEventMap set up and persisting IndexEvent objects to postgres db now (using Metacat's connection). ...
- 12:28 AM Revision 7842 (metacat): only use MapStore/MapLoader for saving/loading IndexEvent objects. No need to use a listener since there is only the single node -- all entries are persisted to DB using the hazelcast.xml config we have for the map. https://projects.ecoinformatics.org/ecoinfo/issues/5944
- 12:03 AM Revision 7841 (metacat): add MapStore/Loader test for the IndexEvents -- adding and removing events in the DB table through hazelcast. https://projects.ecoinformatics.org/ecoinfo/issues/5944
07/02/2013
- 04:47 PM Revision 7840 (metacat): support a "force replication delete all action" during replication. This is used when we want Metacat to remove the content from the other target replicas because the DataONE delete() action was called (more powerful than just "archive").
- 04:41 PM Revision 7839 (metacat): add simple test for the IndexEventDAO class -- adding, removing, listing events in the DB table. https://projects.ecoinformatics.org/ecoinfo/issues/5944
- 03:21 PM Revision 7838 (metacat): Add the code that only the ids with the correct system metadata modification time will be added to the index queue.
- 12:38 PM Task #6004: Figure out why there are only 422 documents indexed in mn-demo-4.test.dataone.org
- I queried the database and got the following result. There are 4998 in the xml_documents and 0 in the xml_revisions.
... - 12:18 PM Task #5924 (Closed): Add the configuration page for the Solr server in the Metacat configuration utility
- 12:18 PM Task #5924: Add the configuration page for the Solr server in the Metacat configuration utility
- Done. Ben added to main property page.
- 11:25 AM Task #5945: Implement reindex action for individual PID
- Proposal is:
-create Metacat action "reindex" that takes an identifier.
-Metacat places the SystemMetadata for that...
07/01/2013
- 03:09 PM Revision 7837 (metacat): Use the hazelcast event log.
- 01:59 PM Revision 7836 (metacat): Add code to get and set the last process date.
06/26/2013
- 04:43 PM Revision 7834 (metacat): merging upgrade scripts from 2.0 branch to trunk. https://redmine.dataone.org/issues/3847
- 04:42 PM Revision 7833 (metacat): scripts for 2.1.0 upgrade
06/25/2013
- 04:42 PM Revision 7830 (metacat): upgrade to Metacat 2.1.0 on the trunk. This includes a new index_event table for storing indexing events that need to be reprocessed. https://projects.ecoinformatics.org/ecoinfo/issues/5944
- 02:51 PM Task #5915 (Closed): Use MN.query(solr) in NCEAS skin
- 02:50 PM Task #5918 (Closed): Separate unit tests from integration tests
- 02:43 PM Bug #6007 (Resolved): Registry uses filename from upload form to save in temporary directory
- I deployed this in KNB today (June 25th, 2013). This was not part of an official release but is in the code for 2.1.x...
- 01:23 PM Revision 7829 (metacat): stub for storing IndexEvent objects in Metacat (from metacat-index processing). https://projects.ecoinformatics.org/ecoinfo/issues/5944
- 01:23 PM Revision 7828 (metacat): move IndexEvent into metacat-common. Perparation for Metacat responding to events and writing them to a persistent store. https://projects.ecoinformatics.org/ecoinfo/issues/5944
- 01:17 PM Revision 7827 (metacat): do not force a get() during refresh (causing EML-defined data access rules to be lost when inserting EML docs about data files). note that this reverses a change that was meant to trigger indexing, but now we are using a new queue to share index events with metacat-index and so should not be necessary.
- 09:10 AM Bug #6009 (Closed): Metacat uses "metacat.output" temp files for all read() operations
- This is in trunk now.
06/24/2013
- 12:07 PM Bug #6009 (In Progress): Metacat uses "metacat.output" temp files for all read() operations
- now reading directly from the filesystem, or in the case of EML with inline-data, the partial file contents.
- 12:05 PM Revision 7824 (metacat): do not use tmp file to return an inputstream on read() operations - just read from the file we already have. https://projects.ecoinformatics.org/ecoinfo/issues/6009
06/21/2013
- 09:47 PM Bug #6008 (Closed): Metacat servlet API upload() method uses user-supplied filename for temp upload
- Now using File.createTempFile() and deleting the data files when done.
- 05:32 PM Bug #6008 (Closed): Metacat servlet API upload() method uses user-supplied filename for temp upload
- This can be problematic if users simultaneously attempt to upload a data file with the same name.
- 09:44 PM Revision 7823 (metacat): use standard File.createTempFile() method for uploaded data files and delete them when we are done with them. https://projects.ecoinformatics.org/ecoinfo/issues/6008
- 05:35 PM Bug #6009 (Closed): Metacat uses "metacat.output" temp files for all read() operations
- The KNB has 1.3 GB of these that have accumulated since ?.
There should be two scenarios:
-stream the contents fr... - 05:02 PM Bug #6007: Registry uses filename from upload form to save in temporary directory
- I deployed the changes on dev2.nceas for testing. Feel pretty good about them - using File::Temp to manager unique up...
- 12:22 PM Bug #6007 (Resolved): Registry uses filename from upload form to save in temporary directory
- This can cause collisions when a room of 22 people all submit a datapackage with the same data file name....
The r... - 04:15 PM Revision 7822 (metacat): correctly delete data file when we are done with it. https://projects.ecoinformatics.org/ecoinfo/issues/6007
- 04:06 PM Revision 7821 (metacat): include filename in the filepart part. https://projects.ecoinformatics.org/ecoinfo/issues/6007
- 03:18 PM Revision 7820 (metacat): send the original filename in the upload() method since that is supported by the Metacat API. https://projects.ecoinformatics.org/ecoinfo/issues/6007
- 03:10 PM Revision 7819 (metacat): remove the unique string when generating data file metadata. https://projects.ecoinformatics.org/ecoinfo/issues/6007
- 02:48 PM Revision 7818 (metacat): debugging. https://projects.ecoinformatics.org/ecoinfo/issues/6007
- 01:44 PM Revision 7817 (metacat): use File::Temp to write data files in registry. https://projects.ecoinformatics.org/ecoinfo/issues/6007
- 11:11 AM Revision 7816 (metacat): correct regex for whitespace in D1 identifier.
- 09:09 AM Revision 7815 (metacat): refactor IndexEventLog a bit to simplify type/action information. prep for serializing IndexEvent objects to Metacat. https://projects.ecoinformatics.org/ecoinfo/issues/5944
06/20/2013
- 05:28 PM Revision 7814 (metacat): remove serial number from indexeventlog - it is not used elsewhere in the api. https://projects.ecoinformatics.org/ecoinfo/issues/5944
- 05:28 PM Bug #6006 (Closed): Metacat admin screen requires metacat-index directories
- Now skipping the template creation if metacat-index war is not deployed along with metacat.
- 05:21 PM Revision 7813 (metacat): correct spelling for index.eventlog.classname property
- 04:59 PM Task #5943 (Resolved): Eliminate the unnecessary Systemmetadata event between the Metacat and Metacat-index modules.
- Switched over to the new structure for communicating index events. Seems to be working as expected.
- 04:49 PM Revision 7812 (metacat): use an independent ISet<SystemMetadata> structure to communicate objects that should be indexed by metacat-index. https://projects.ecoinformatics.org/ecoinfo/issues/5943
- 12:23 PM Revision 7811 (metacat): consolidate SystemMetadata map retrieval in preparation for using a different structure for objects to index.
- 10:22 AM Task #5944: Metacat-index need an IndexEvent and IndexEventLog mechanism
- Added delete() method to remove entries that we do not need to reprocess again (because they were successful, finally...
- 09:24 AM Revision 7810 (metacat): adding ability to remove event from the [error] queue.
06/18/2013
- 02:59 PM Revision 7809 (metacat): do not create solr-home if there is no template to compy into that directory (need to be able to create it later if/when someone decides to use and deploy metacat-index). https://projects.ecoinformatics.org/ecoinfo/issues/6006
- 02:54 PM Task #5943: Eliminate the unnecessary Systemmetadata event between the Metacat and Metacat-index modules.
- Proposed solution is to use a different/independent structure for SystemMetadata events that should trigger a re-inde...
- 01:54 PM Revision 7808 (metacat): do not attempt to copy solr-home template from metacat-index webapp if it does not exist. This would be in cases where metacat-index is not deployed. https://projects.ecoinformatics.org/ecoinfo/issues/6006
- 12:35 PM Bug #6006 (Closed): Metacat admin screen requires metacat-index directories
- When configuring Metacat there is a dependency on metacat-index. This should be gracefully skipped if that webapp is ...
- 10:19 AM Task #6005 (Closed): Incomplete solr index for the data objects if the resourcemap was indexed first
- Currently the resource map should be indexed after the data objects in the resource map were indexed. If the resource...
06/17/2013
- 05:07 PM Revision 7807 (metacat): Add code to implment set and get the last processed date.
- 04:28 PM Revision 7806 (metacat): It will make the index only for those objects which were modified after the marked time.
- 01:50 PM Revision 7805 (metacat): Add set and get the lastprocessedDate in the IndexEventLog.
- Remove the code to write the successful event.
- 10:31 AM Task #6004: Figure out why there are only 422 documents indexed in mn-demo-4.test.dataone.org
- Attached a log file for the indexing process.
- 10:30 AM Task #6004 (Resolved): Figure out why there are only 422 documents indexed in mn-demo-4.test.dataone.org
- There are more 4000 data objects in the mn-demo-4. But we only gets 422 documents indexed.
06/13/2013
- 05:39 PM Bug #6003 (Closed): The "war" target in the metacat will build the metacat-index.war
- We added the target which build metacat-index.war as the dependency of the "war" target.
- 05:37 PM Bug #6003 (Closed): The "war" target in the metacat will build the metacat-index.war
- The "war" target in the metacat doesn't build the metacat-index.war in the dist directory. It only builds knb.war and...
- 05:35 PM Task #5930: Add a configuration property to turn on/off the indexing processes (SOLR indexing)
- If we removed the "solr" from the engine list, metacat-index will not build index anymore. And the solr query will be...
- 05:20 PM Revision 7804 (metacat): Add the dataone repository.
- 04:41 PM Revision 7803 (metacat): The "war" target will build the metacat-index.war as well.
06/12/2013
- 02:36 PM Revision 7802 (metacat): Log the timed index jobs.
- 02:08 PM Revision 7801 (metacat): Add the code to log the failed events.
06/11/2013
Also available in: Atom