Project

General

Profile

Activity

From 05/09/2013 to 06/07/2013

06/07/2013

04:38 PM Revision 7797 (metacat): Add a serial number for the event.
Add method to set events to be archived. Jing Tao
11:56 AM Feature #5936: Include certificate delegation inside Metacat
There's now the package "edu.ucsb.nceas.metacat.portal" that does the authentication delegation. You start a request ... ben leinfelder
11:22 AM Revision 7796 (metacat): Add a new class variable - isArchived for class IndexEvent.
Jing Tao

06/06/2013

04:13 PM Revision 7795 (metacat): Update the documentation about those classes.
Jing Tao
03:48 PM Bug #5997 (Closed): Restrict KNB trusted CAs
Commented out the line that includes all CAs in /etc/ssl/certs and reloaded Apache. Now we are down to the short list... ben leinfelder
03:17 PM Revision 7794 (metacat): Add a event and eventlog for the index.
Jing Tao

06/05/2013

03:03 PM Revision 7793 (metacat): Use the identifier set to get the list of ids in the member node.
Jing Tao
02:20 PM Revision 7792 (metacat): The returned ISet should be Identifier.
Jing Tao
01:50 PM Revision 7791 (metacat): Add code to test getIdentifierSet method.
Jing Tao
01:50 PM Revision 7790 (metacat): Add method to get identifier set.
Jing Tao
11:02 AM Bug #5997: Restrict KNB trusted CAs
We should be able to simply use the DataONE chain file that we have installed already:... ben leinfelder
11:00 AM Bug #5997 (Closed): Restrict KNB trusted CAs
Instead of trusting all commercial CAs, the KNB Member Node should only trust the DataONE and CILogon certificate aut... ben leinfelder

06/04/2013

02:49 PM Revision 7789 (metacat): Add a new property to specify the interval of a Timer to run the thread generating solr index.
Jing Tao
02:47 PM Revision 7788 (metacat): Set up a Timer to run the regenerating solr index task periodically.
Jing Tao

06/03/2013

05:36 PM Task #5922 (Closed): Modify the solr index configuration for the resource map
We rewrote the ResourceMapSubprocessor class which now uses the SolrServer. Jing Tao
05:22 PM Task #5937 (Closed): The solr indexes of the data file contains obsoleted ids for the resourcemap and documentBy element after updating a data package
Add code to modify the "documents", "resourceMap" and "documentedBy" element in the solr doc when archive happens. Jing Tao
04:35 PM Revision 7787 (metacat): Use the ";" as the seperator to replace "," in the resource name spaces.
Jing Tao

05/31/2013

03:29 PM Revision 7786 (metacat): Add code to handle delete data package information when delete a pid in the solr index.
Jing Tao
02:05 PM Revision 7785 (metacat): Add two static methods to get the SystemMetadata and data object InputStream for the specified id.
Jing Tao

05/28/2013

05:19 PM Revision 7784 (metacat): Change the code since the ApplicationController's constructor was changed.
Jing Tao
05:18 PM Revision 7783 (metacat): Add code to check if the metacat.properties is available.
Jing Tao
01:21 PM Revision 7782 (metacat): If solr is not enabled, it would not be running.
Jing Tao
01:20 PM Revision 7781 (metacat): Solr will be enabled if it is in the db.enabledEngines.
Jing Tao
01:18 PM Revision 7780 (metacat): Use ";" to separate db.enabledEngines.
Jing Tao

05/24/2013

06:16 PM Task #5994 (New): Create REST API for accessing statistics
For objects, users, packages, nodes, etc. Matt Jones
06:15 PM Task #5993 (New): Summarize and index statistics for fast access
Matt Jones
06:15 PM Task #5992 (New): Track citations
Or interface with Impact Story Matt Jones
06:14 PM Task #5991 (New): Track views
Matt Jones
06:14 PM Task #5990 (New): Track downloads
Matt Jones
06:14 PM Feature #5989 (In Progress): Track data download, view and citation statistics
Currently the only usage stats we have in Metacat are the raw logs. This new service would provide several statistic... Matt Jones
04:47 PM Revision 7779 (metacat): Use the ";" as the seperator for properties.
Jing Tao
04:38 PM Task #5945 (Closed): Implement reindex action for individual PID
We may need issue a command to metacat-index to regenerate index for all missed ids or just regenerate one for a spec... Jing Tao
04:31 PM Task #5944 (Closed): Metacat-index need an IndexEvent and IndexEventLog mechanism
We need the IndexEvent and IndexEventLog to handle the events in building the solr index. For example, a failure even... Jing Tao
04:24 PM Task #5943 (Resolved): Eliminate the unnecessary Systemmetadata event between the Metacat and Metacat-index modules.
Currently we split metacat and metacat-index web apps. The communication between them is the Systemmetadata entry eve... Jing Tao
04:12 PM Task #5926 (Resolved): Metacat-index doesn't build solr index during a Metadata inserting document
It turned out that the solr doc was generated. However, the building index and querying index use different EmbeddedS... Jing Tao
04:05 PM Task #5927 (Resolved): Implements listing the query engines and fields for the solr query engine
We add a new class - MetacatSolrEngineDescriptionHandler to handle the request. It works on both http solr server and... Jing Tao
04:03 PM Task #5822 (Resolved): Enforce access control for SOLR-based search implementation
Jing Tao
04:02 PM Task #5904 (Resolved): Design mechanism to enforce access policy
We append a filter query to enforce the access rules. Jing Tao
04:01 PM Task #5905 (Resolved): Implement access control filter
Add junit test to test the public, specified user and group users readable documents. Also it tests a use with untru... Jing Tao
03:58 PM Task #5820 (Resolved): Incorporate DataONE indexing
Create metacat-index web apps to avoid the spring in the metacat. We also create the metacat-common to share code bet... Jing Tao
03:57 PM Task #5819 (Resolved): MN query for SOLR
The query and query description is done. Jing Tao
03:56 PM Task #5906 (Resolved): Implement MetacatSolrIndex class
We added two classes MetacatSolrIndex and MetacatSolrEngineDescriptionHandler. This two classes handle the query and ... Jing Tao
03:07 PM Task #5942 (Closed): Handle sessionid (cookie from login method) in D1 REST service
This is now in D1ResourceHandler in Metacat trunk ben leinfelder
10:20 AM Revision 7778 (metacat): do not require PortalCertificateManager be configured. Fix NPE because session was not created when using old sessionid-based authentication. https://projects.ecoinformatics.org/ecoinfo/issues/5942
ben leinfelder
12:09 AM Revision 7777 (metacat): change the waiting time to 10 seconds and attempts to 600 for the hazelcast.
Jing Tao

05/23/2013

11:57 PM Revision 7776 (metacat): Use another thread in the Servlet init method to wait hazelcast.
Jing Tao
10:42 PM Revision 7775 (metacat): Make the target init depend on build-metacat-common.
Jing Tao
09:59 PM Revision 7774 (metacat): Put the waiting mechanism for the hazelcast at the first place.
Jing Tao
09:45 PM Revision 7773 (metacat): handle client certificates, portal certificates and jsessionid as three ways to prove you are an uthenticated user. https://projects.ecoinformatics.org/ecoinfo/issues/5942
ben leinfelder
06:39 PM Revision 7772 (metacat): Use some contants from the EnabledQueryEngines.
Jing Tao
06:28 PM Revision 7771 (metacat): Temporarily remove the code to disable solr engine if it isn't listed in the property file.
Jing Tao
05:22 PM Task #5942 (Closed): Handle sessionid (cookie from login method) in D1 REST service
If there is no client certificate on the request or in the portal delegation store, then we should look for a valid s... ben leinfelder
05:19 PM Task #5941 (Rejected): Add login/logout methods to DataONE MN service
These will use the existing functionality of Metacat servlet's action=login/logout. ben leinfelder
05:17 PM Story #5940 (Closed): Support session-based Metacat authentication (for D1 services)
We want to be able to deploy Metacat without relying on DataONE, CILogon, ECP, or even client certificates.
Idea i...
ben leinfelder
04:44 PM Feature #5939 (Closed): Create view service that can render metadata documents at REST URL
We need a 'landing page' for metadata views that can be referenced as REST URLs and that show an HTML'ised version of... Matt Jones
04:41 PM Feature #5936: Include certificate delegation inside Metacat
Here's the documentation for MyProxy that I was using to set this all up: http://grid.ncsa.illinois.edu/myproxy/oauth... ben leinfelder
04:29 PM Revision 7770 (metacat): exlude /lib/maven from the war file
ben leinfelder
04:27 PM Revision 7769 (metacat): If the solr engine is disabled, the metacat index will do nothing.
Jing Tao
01:43 PM Revision 7768 (metacat): Use the new name of a method.
Jing Tao
01:42 PM Revision 7767 (metacat): Rename a method to isEnabled.
Jing Tao
01:26 PM Revision 7766 (metacat): Updated documentation, and added modification date to the sitemap index file entries.
Matt Jones
12:55 PM Revision 7765 (metacat): Remove unused import.
Matt Jones
12:52 PM Revision 7764 (metacat): Mofdified Sitemap class to also generate the sitemap index file that is needed when more than one sitemap file is provided.
Matt Jones
12:42 PM Revision 7763 (metacat): Remove the junit test for an obsoleted class.
Jing Tao
12:41 PM Revision 7762 (metacat): Remove the obsoleted class.
Jing Tao
12:40 PM Revision 7761 (metacat): Add a junit test class for EnabledQueryEngines.
Jing Tao
12:40 PM Revision 7760 (metacat): Add a test base class.
Jing Tao
12:39 PM Revision 7759 (metacat): Add a class to repsent the enabled engine list.
Jing Tao
11:55 AM Revision 7758 (metacat): Change the junit version to 4.8
Jing Tao
10:20 AM Revision 7757 (metacat): use ContentTypeInputStream interface (and ByteArray implementation) to specify the desired content-type of the InputStream returned by MN.query().
ben leinfelder
09:15 AM Revision 7756 (metacat): Add the new property dbquery.enabledEngines=pathquery,solr.
Jing Tao

05/22/2013

04:44 PM Revision 7755 (metacat): load the evicted SM back into the map on a "Refresh" so that listeners hear the update. (metacat-index, for example)
ben leinfelder
04:18 PM Revision 7754 (metacat): set the dataone.hazelcast.location.clientconfig property to be the local config file. It will be used in the d1_cn_index_processor
Jing Tao
03:50 PM Revision 7753 (metacat): use DataONE cilogon skin so that we are not prompted for certificate duration (and to make it more cohesive as a prototype using the Metacat DataONE skin
ben leinfelder
03:49 PM Revision 7752 (metacat): switch back to log4j statements now that I am sure certificate delegation is working.
ben leinfelder
03:35 PM Revision 7751 (metacat): add link to login or switch user
ben leinfelder
03:29 PM Revision 7750 (metacat): add simple page to show logged-in user subject DN
ben leinfelder
03:02 PM Revision 7749 (metacat): use System.out.println until the oa4mp logging issue is resolved.
ben leinfelder
11:23 AM Task #5928 (Closed): Make sure to start metacat-index web apps when the metacat web apps is start-up during the tomcat startup
Added the mechanism to wait some time if the MNode or the Hazelcast is not ready. We also add a maximum attempt contr... Jing Tao
11:19 AM Revision 7748 (metacat): Read the waiting time and maximum attempts from the metacat.properties file.
Jing Tao
10:56 AM Revision 7747 (metacat): Add the waiting time and max attempts for regenerating the index.
Jing Tao
10:36 AM Revision 7746 (metacat): use filesystem store. in-memory store seems to get into deadlock with multiple environments accessing it.
ben leinfelder
10:34 AM Revision 7745 (metacat): add logging for portal certificate look up process.
ben leinfelder
10:29 AM Revision 7744 (metacat): Add some new properties for the test.
Jing Tao
09:34 AM Revision 7743 (metacat): use relative path for oa4mp_client.xml (within servlet context). https://projects.ecoinformatics.org/ecoinfo/issues/5936
ben leinfelder
08:05 AM Revision 7742 (metacat): use in-memory storage for myproxy. at least for now.
ben leinfelder

05/21/2013

11:35 PM Revision 7741 (metacat): Add a property containing the list of the resource map namespaces.
Jing Tao
11:33 PM Revision 7740 (metacat): Index the none-resourcemap data objects first, then the resourcemap objects.
Jing Tao
11:32 PM Revision 7739 (metacat): used pk8 format private key file name
ben leinfelder
11:12 PM Revision 7738 (metacat): use java 1.6 compliance for compilation
ben leinfelder
11:05 PM Revision 7737 (metacat): first pass at integrating CILogon/MyProxy certificates in Metacat. Configuration is specific to mn-demo-4.test.dataone.org for the time being (this will cause localhost deployments to fail webapp deployment). https://projects.ecoinformatics.org/ecoinfo/issues/5936
ben leinfelder
11:05 PM Revision 7736 (metacat): first pass at integrating CILogon/MyProxy certificates in Metacat. Configuration is specific to mn-demo-4.test.dataone.org for the time being (this will cause localhost deployments to fail webapp deployment). https://projects.ecoinformatics.org/ecoinfo/issues/5936
ben leinfelder
07:35 PM Bug #5938 (Closed): sitemap format is deprecated
Updated the namespace to use the 0.9 format. No other changes needed for validity. Committed in r7735. Matt Jones
07:34 PM Bug #5938 (Closed): sitemap format is deprecated
The sitemap format used by Metacat has been deprecated, and should be updated to the current release (0.9) as publish... Matt Jones
07:32 PM Revision 7735 (metacat): Updated Sitemap generation to use latest version of the sitemap protocol schemas.
Matt Jones
02:23 PM Revision 7734 (metacat): Try to get the metacat ids first.
Jing Tao
11:29 AM Task #5937 (Closed): The solr indexes of the data file contains obsoleted ids for the resourcemap and documentBy element after updating a data package
I used a morpho to insert a data package which includes the data file, eml file and resource map file to the metacat.... Jing Tao
11:11 AM Revision 7733 (metacat): Use SolrQueryServiceController to get the ValidFieldList.
Jing Tao

05/20/2013

11:39 PM Revision 7732 (metacat): Remove the code to throw an exception if the subjects is null in the query method.
Jing Tao
10:42 PM Revision 7731 (metacat): changed the xpath to get solr_spec_version.
Jing Tao
10:20 PM Revision 7730 (metacat): Use the SolrQueryServiceController to get the spec version and index schema information.
Jing Tao
10:18 PM Revision 7729 (metacat): Add some new properties for the http solr server.
Jing Tao
09:33 PM Revision 7728 (metacat): Add the getIndexFields and getValidIndexFields methods.
Jing Tao
04:27 PM Revision 7727 (metacat): Add getSolrSpecVersion method for the HttpServer and the controller.
Jing Tao
02:12 PM Revision 7726 (metacat): Imeplement the getSolrVersion, getSchemaField and getValidSchemaFields for the EmbeddedSolrServer.
Jing Tao

05/18/2013

11:46 PM Revision 7725 (metacat): Change the package of SolrQueryReponseWriterFactory and SolrQueryResponseTransformer.
Jing Tao
11:32 PM Revision 7724 (metacat): Change the package name of SolrQueryResponseTransformer and SolrQueryWriterFactory.
Jing Tao
11:29 PM Revision 7723 (metacat): move SolrQueryResponseTransformer and SolrQueryReponseWriterFactory to the query package.
Jing Tao
11:28 PM Revision 7722 (metacat): Move it to the query package.
Jing Tao
11:28 PM Revision 7721 (metacat): move it to the query package.
Jing Tao
11:24 PM Revision 7720 (metacat): Use the new query(SolrParams param) method of the SolrQueryServiceController.
Jing Tao
11:22 PM Revision 7719 (metacat): Remove the query(String query) methods from SolrQueryService.java.
Jing Tao
05:19 PM Revision 7718 (metacat): Use the SolrQueryServiceController class to handle the query.
Jing Tao
04:49 PM Revision 7717 (metacat): Add the depency of d1_cn_index_processor here.
Jing Tao
04:48 PM Revision 7716 (metacat): Remove the depency on the d1_cn_index_processor.
Jing Tao
03:57 PM Revision 7715 (metacat): move the depency of d1_cn_index_processor to the metacat-common module.
Jing Tao
03:56 PM Revision 7714 (metacat): Use one query method in the SolrQueryServiceController.
Jing Tao
03:38 PM Revision 7713 (metacat): Move the solr query part here since metacat-index needs to query the solr server too.
Jing Tao
03:37 PM Revision 7712 (metacat): Use the sepcified exceptions to replace the Exception.
Jing Tao

05/17/2013

05:01 PM Revision 7711 (metacat): Add the code to add a resource map information.
Jing Tao
04:07 PM Revision 7710 (metacat): Add code the transform query response to SolrDoc.
Jing Tao
02:22 PM Revision 7709 (metacat): Move the cod which transformed the query response to the inputstream to the metacat-common module.
Remove some obsoleted imports. Jing Tao
02:05 PM Revision 7708 (metacat): Add a transformer to process the query response object.
Jing Tao
12:57 PM Revision 7707 (metacat): Move the code to generate the QueryResponseWriter to the metacat-common module. So it can be shared with the metacat-index module.
Jing Tao
12:40 PM Revision 7706 (metacat): Add a new class to create the solr QueryResponseWriter.
Jing Tao
10:58 AM Feature #5936 (Closed): Include certificate delegation inside Metacat
Like the d1_portal, we want to be able to have users authenticate with their preferred Identity Provider via the brow... ben leinfelder
10:52 AM Task #5928: Make sure to start metacat-index web apps when the metacat web apps is start-up during the tomcat startup
Is there a way to tell Tomcat which order to load the webapps? I have seen some errors on mn-demo-4 when metacat-inde... ben leinfelder
10:44 AM Task #5935 (Rejected): Display the PID not the docid in the rendered metadata
I think we decided that rendering the PID on the EML view page is not a requirement at this time, though I still thin... ben leinfelder
10:41 AM Revision 7705 (metacat): use pid, not docid as the parameter to the registry when editing/deleting objects since the vlaue is really the pid. https://projects.ecoinformatics.org/ecoinfo/issues/5932
ben leinfelder
10:29 AM Revision 7704 (metacat): exclude jibx-tools, maven cruft and org.eclipse "dependencies" to cut down on the number of jar files in the lib folder.
ben leinfelder
10:27 AM Revision 7703 (metacat): organize imports
ben leinfelder
09:57 AM Revision 7702 (metacat): format before editing
ben leinfelder

05/16/2013

04:02 PM Revision 7701 (metacat): use java 1.6 for maven compiler
ben leinfelder
03:42 PM Task #5932 (Closed): Allow use of PID (or docid) using cgi-registry modification form
The registry now calls a new Metacat servlet method: getDocid(pid). Then it continues on as though the docid was prov... ben leinfelder
03:22 PM Revision 7700 (metacat): use correct test for PID param.
ben leinfelder
03:21 PM Revision 7699 (metacat): remove extra lines from returned <docid/> block. https://projects.ecoinformatics.org/ecoinfo/issues/5932
ben leinfelder
01:53 PM Revision 7698 (metacat): move to libclient/common snapshot to match metacat-common project.
ben leinfelder
12:04 PM Revision 7697 (metacat): Use the new class to handle the resource map.
Jing Tao

05/15/2013

05:48 PM Revision 7696 (metacat): Modify the subprocessor to use SolrServer rather than solr service url.
Jing Tao
05:16 PM Revision 7695 (metacat): Allow use of PID instead of docid in the Perl registry. At least for reading/editing and deleting existing content. Does not create content using a pid. https://projects.ecoinformatics.org/ecoinfo/issues/5932
ben leinfelder
03:53 PM Revision 7694 (metacat): initialize the SOLR home directory if it does not already exist.
ben leinfelder

05/14/2013

06:10 PM Revision 7693 (metacat): Add code to test title for the query result.
Jing Tao
05:51 PM Revision 7692 (metacat): Rewrite some methods, so the query result can be processed many times.
Jing Tao

05/13/2013

09:23 PM Revision 7691 (metacat): Add code to wait the readiness of the hazelcast service.
Jing Tao
04:34 PM Revision 7690 (metacat): Use the DistributedMapsFactory to get maps.
Jing Tao
03:51 PM Revision 7689 (metacat): Add a class to get distributed maps such as the system metadata map.
Jing Tao
11:58 AM Revision 7688 (metacat): Removed the comment on the process of building index.
Jing Tao
11:55 AM Revision 7687 (metacat): Add code to wait the startup of the MNode.
Jing Tao

05/10/2013

04:09 PM Task #5905: Implement access control filter
I also tested that if there is a fq in the user specified query, it wouldn't cause any issue:
The user specified q...
Jing Tao
04:04 PM Task #5905: Implement access control filter
Append filter query (fq) to the main query. The fq looks like:
fq=(readPermission:"public")OR(readPermission:"CN=ben...
Jing Tao
03:02 PM Revision 7686 (metacat): Removed the commented out text cases and add a test for archvied document.
Jing Tao
02:59 PM Revision 7685 (metacat): Only after reloading the core, the query result can reflect the change made in metacat-index module.
Jing Tao
02:37 PM Revision 7684 (metacat): Use the log info to replace the print.
Jing Tao
02:36 PM Revision 7683 (metacat): Add log information for archiving or inserting indexes.
Jing Tao

05/09/2013

01:26 PM Bug #3017 (Closed): The Ampersand returns a blank page on searches
This is in trunk now, though it is not applicable for solr queries. (I'm sure there are other character escaping issu... ben leinfelder
12:11 PM Revision 7682 (metacat): Consider mutithread issue there.
Jing Tao
12:10 PM Revision 7681 (metacat): Fixed a bug to put "OR" correctly in the query.
And remove the user "authorized_user" from the rightsholder clause in the query. Jing Tao
10:04 AM Revision 7680 (metacat): Use the set of subjects to replace the user and groups for the solr query.
Jing Tao
12:19 AM Revision 7679 (metacat): Call setting certificate location to be the test one after getting the MN baseurl.
The method to getting MN baseurl somehow calls CN and it set the certificate location to be /var/metacat/certs/METACA... Jing Tao
 

Also available in: Atom