Activity
From 05/02/2013 to 05/31/2013
05/31/2013
- 03:29 PM Revision 7786 (metacat): Add code to handle delete data package information when delete a pid in the solr index.
- 02:05 PM Revision 7785 (metacat): Add two static methods to get the SystemMetadata and data object InputStream for the specified id.
05/28/2013
- 05:19 PM Revision 7784 (metacat): Change the code since the ApplicationController's constructor was changed.
- 05:18 PM Revision 7783 (metacat): Add code to check if the metacat.properties is available.
- 01:21 PM Revision 7782 (metacat): If solr is not enabled, it would not be running.
- 01:20 PM Revision 7781 (metacat): Solr will be enabled if it is in the db.enabledEngines.
- 01:18 PM Revision 7780 (metacat): Use ";" to separate db.enabledEngines.
05/24/2013
- 06:16 PM Task #5994 (New): Create REST API for accessing statistics
- For objects, users, packages, nodes, etc.
- 06:15 PM Task #5993 (New): Summarize and index statistics for fast access
- 06:15 PM Task #5992 (New): Track citations
- Or interface with Impact Story
- 06:14 PM Task #5991 (New): Track views
- 06:14 PM Task #5990 (New): Track downloads
- 06:14 PM Feature #5989 (In Progress): Track data download, view and citation statistics
- Currently the only usage stats we have in Metacat are the raw logs. This new service would provide several statistic...
- 04:47 PM Revision 7779 (metacat): Use the ";" as the seperator for properties.
- 04:38 PM Task #5945 (Closed): Implement reindex action for individual PID
- We may need issue a command to metacat-index to regenerate index for all missed ids or just regenerate one for a spec...
- 04:31 PM Task #5944 (Closed): Metacat-index need an IndexEvent and IndexEventLog mechanism
- We need the IndexEvent and IndexEventLog to handle the events in building the solr index. For example, a failure even...
- 04:24 PM Task #5943 (Resolved): Eliminate the unnecessary Systemmetadata event between the Metacat and Metacat-index modules.
- Currently we split metacat and metacat-index web apps. The communication between them is the Systemmetadata entry eve...
- 04:12 PM Task #5926 (Resolved): Metacat-index doesn't build solr index during a Metadata inserting document
- It turned out that the solr doc was generated. However, the building index and querying index use different EmbeddedS...
- 04:05 PM Task #5927 (Resolved): Implements listing the query engines and fields for the solr query engine
- We add a new class - MetacatSolrEngineDescriptionHandler to handle the request. It works on both http solr server and...
- 04:03 PM Task #5822 (Resolved): Enforce access control for SOLR-based search implementation
- 04:02 PM Task #5904 (Resolved): Design mechanism to enforce access policy
- We append a filter query to enforce the access rules.
- 04:01 PM Task #5905 (Resolved): Implement access control filter
- Add junit test to test the public, specified user and group users readable documents. Also it tests a use with untru...
- 03:58 PM Task #5820 (Resolved): Incorporate DataONE indexing
- Create metacat-index web apps to avoid the spring in the metacat. We also create the metacat-common to share code bet...
- 03:57 PM Task #5819 (Resolved): MN query for SOLR
- The query and query description is done.
- 03:56 PM Task #5906 (Resolved): Implement MetacatSolrIndex class
- We added two classes MetacatSolrIndex and MetacatSolrEngineDescriptionHandler. This two classes handle the query and ...
- 03:07 PM Task #5942 (Closed): Handle sessionid (cookie from login method) in D1 REST service
- This is now in D1ResourceHandler in Metacat trunk
- 10:20 AM Revision 7778 (metacat): do not require PortalCertificateManager be configured. Fix NPE because session was not created when using old sessionid-based authentication. https://projects.ecoinformatics.org/ecoinfo/issues/5942
- 12:09 AM Revision 7777 (metacat): change the waiting time to 10 seconds and attempts to 600 for the hazelcast.
05/23/2013
- 11:57 PM Revision 7776 (metacat): Use another thread in the Servlet init method to wait hazelcast.
- 10:42 PM Revision 7775 (metacat): Make the target init depend on build-metacat-common.
- 09:59 PM Revision 7774 (metacat): Put the waiting mechanism for the hazelcast at the first place.
- 09:45 PM Revision 7773 (metacat): handle client certificates, portal certificates and jsessionid as three ways to prove you are an uthenticated user. https://projects.ecoinformatics.org/ecoinfo/issues/5942
- 06:39 PM Revision 7772 (metacat): Use some contants from the EnabledQueryEngines.
- 06:28 PM Revision 7771 (metacat): Temporarily remove the code to disable solr engine if it isn't listed in the property file.
- 05:22 PM Task #5942 (Closed): Handle sessionid (cookie from login method) in D1 REST service
- If there is no client certificate on the request or in the portal delegation store, then we should look for a valid s...
- 05:19 PM Task #5941 (Rejected): Add login/logout methods to DataONE MN service
- These will use the existing functionality of Metacat servlet's action=login/logout.
- 05:17 PM Story #5940 (Closed): Support session-based Metacat authentication (for D1 services)
- We want to be able to deploy Metacat without relying on DataONE, CILogon, ECP, or even client certificates.
Idea i... - 04:44 PM Feature #5939 (Closed): Create view service that can render metadata documents at REST URL
- We need a 'landing page' for metadata views that can be referenced as REST URLs and that show an HTML'ised version of...
- 04:41 PM Feature #5936: Include certificate delegation inside Metacat
- Here's the documentation for MyProxy that I was using to set this all up: http://grid.ncsa.illinois.edu/myproxy/oauth...
- 04:29 PM Revision 7770 (metacat): exlude /lib/maven from the war file
- 04:27 PM Revision 7769 (metacat): If the solr engine is disabled, the metacat index will do nothing.
- 01:43 PM Revision 7768 (metacat): Use the new name of a method.
- 01:42 PM Revision 7767 (metacat): Rename a method to isEnabled.
- 01:26 PM Revision 7766 (metacat): Updated documentation, and added modification date to the sitemap index file entries.
- 12:55 PM Revision 7765 (metacat): Remove unused import.
- 12:52 PM Revision 7764 (metacat): Mofdified Sitemap class to also generate the sitemap index file that is needed when more than one sitemap file is provided.
- 12:42 PM Revision 7763 (metacat): Remove the junit test for an obsoleted class.
- 12:41 PM Revision 7762 (metacat): Remove the obsoleted class.
- 12:40 PM Revision 7761 (metacat): Add a junit test class for EnabledQueryEngines.
- 12:40 PM Revision 7760 (metacat): Add a test base class.
- 12:39 PM Revision 7759 (metacat): Add a class to repsent the enabled engine list.
- 11:55 AM Revision 7758 (metacat): Change the junit version to 4.8
- 10:20 AM Revision 7757 (metacat): use ContentTypeInputStream interface (and ByteArray implementation) to specify the desired content-type of the InputStream returned by MN.query().
- 09:15 AM Revision 7756 (metacat): Add the new property dbquery.enabledEngines=pathquery,solr.
05/22/2013
- 04:44 PM Revision 7755 (metacat): load the evicted SM back into the map on a "Refresh" so that listeners hear the update. (metacat-index, for example)
- 04:18 PM Revision 7754 (metacat): set the dataone.hazelcast.location.clientconfig property to be the local config file. It will be used in the d1_cn_index_processor
- 03:50 PM Revision 7753 (metacat): use DataONE cilogon skin so that we are not prompted for certificate duration (and to make it more cohesive as a prototype using the Metacat DataONE skin
- 03:49 PM Revision 7752 (metacat): switch back to log4j statements now that I am sure certificate delegation is working.
- 03:35 PM Revision 7751 (metacat): add link to login or switch user
- 03:29 PM Revision 7750 (metacat): add simple page to show logged-in user subject DN
- 03:02 PM Revision 7749 (metacat): use System.out.println until the oa4mp logging issue is resolved.
- 11:23 AM Task #5928 (Closed): Make sure to start metacat-index web apps when the metacat web apps is start-up during the tomcat startup
- Added the mechanism to wait some time if the MNode or the Hazelcast is not ready. We also add a maximum attempt contr...
- 11:19 AM Revision 7748 (metacat): Read the waiting time and maximum attempts from the metacat.properties file.
- 10:56 AM Revision 7747 (metacat): Add the waiting time and max attempts for regenerating the index.
- 10:36 AM Revision 7746 (metacat): use filesystem store. in-memory store seems to get into deadlock with multiple environments accessing it.
- 10:34 AM Revision 7745 (metacat): add logging for portal certificate look up process.
- 10:29 AM Revision 7744 (metacat): Add some new properties for the test.
- 09:34 AM Revision 7743 (metacat): use relative path for oa4mp_client.xml (within servlet context). https://projects.ecoinformatics.org/ecoinfo/issues/5936
- 08:05 AM Revision 7742 (metacat): use in-memory storage for myproxy. at least for now.
05/21/2013
- 11:35 PM Revision 7741 (metacat): Add a property containing the list of the resource map namespaces.
- 11:33 PM Revision 7740 (metacat): Index the none-resourcemap data objects first, then the resourcemap objects.
- 11:32 PM Revision 7739 (metacat): used pk8 format private key file name
- 11:12 PM Revision 7738 (metacat): use java 1.6 compliance for compilation
- 11:05 PM Revision 7737 (metacat): first pass at integrating CILogon/MyProxy certificates in Metacat. Configuration is specific to mn-demo-4.test.dataone.org for the time being (this will cause localhost deployments to fail webapp deployment). https://projects.ecoinformatics.org/ecoinfo/issues/5936
- 11:05 PM Revision 7736 (metacat): first pass at integrating CILogon/MyProxy certificates in Metacat. Configuration is specific to mn-demo-4.test.dataone.org for the time being (this will cause localhost deployments to fail webapp deployment). https://projects.ecoinformatics.org/ecoinfo/issues/5936
- 07:35 PM Bug #5938 (Closed): sitemap format is deprecated
- Updated the namespace to use the 0.9 format. No other changes needed for validity. Committed in r7735.
- 07:34 PM Bug #5938 (Closed): sitemap format is deprecated
- The sitemap format used by Metacat has been deprecated, and should be updated to the current release (0.9) as publish...
- 07:32 PM Revision 7735 (metacat): Updated Sitemap generation to use latest version of the sitemap protocol schemas.
- 02:23 PM Revision 7734 (metacat): Try to get the metacat ids first.
- 11:29 AM Task #5937 (Closed): The solr indexes of the data file contains obsoleted ids for the resourcemap and documentBy element after updating a data package
- I used a morpho to insert a data package which includes the data file, eml file and resource map file to the metacat....
- 11:11 AM Revision 7733 (metacat): Use SolrQueryServiceController to get the ValidFieldList.
05/20/2013
- 11:39 PM Revision 7732 (metacat): Remove the code to throw an exception if the subjects is null in the query method.
- 10:42 PM Revision 7731 (metacat): changed the xpath to get solr_spec_version.
- 10:20 PM Revision 7730 (metacat): Use the SolrQueryServiceController to get the spec version and index schema information.
- 10:18 PM Revision 7729 (metacat): Add some new properties for the http solr server.
- 09:33 PM Revision 7728 (metacat): Add the getIndexFields and getValidIndexFields methods.
- 04:27 PM Revision 7727 (metacat): Add getSolrSpecVersion method for the HttpServer and the controller.
- 02:12 PM Revision 7726 (metacat): Imeplement the getSolrVersion, getSchemaField and getValidSchemaFields for the EmbeddedSolrServer.
05/18/2013
- 11:46 PM Revision 7725 (metacat): Change the package of SolrQueryReponseWriterFactory and SolrQueryResponseTransformer.
- 11:32 PM Revision 7724 (metacat): Change the package name of SolrQueryResponseTransformer and SolrQueryWriterFactory.
- 11:29 PM Revision 7723 (metacat): move SolrQueryResponseTransformer and SolrQueryReponseWriterFactory to the query package.
- 11:28 PM Revision 7722 (metacat): Move it to the query package.
- 11:28 PM Revision 7721 (metacat): move it to the query package.
- 11:24 PM Revision 7720 (metacat): Use the new query(SolrParams param) method of the SolrQueryServiceController.
- 11:22 PM Revision 7719 (metacat): Remove the query(String query) methods from SolrQueryService.java.
- 05:19 PM Revision 7718 (metacat): Use the SolrQueryServiceController class to handle the query.
- 04:49 PM Revision 7717 (metacat): Add the depency of d1_cn_index_processor here.
- 04:48 PM Revision 7716 (metacat): Remove the depency on the d1_cn_index_processor.
- 03:57 PM Revision 7715 (metacat): move the depency of d1_cn_index_processor to the metacat-common module.
- 03:56 PM Revision 7714 (metacat): Use one query method in the SolrQueryServiceController.
- 03:38 PM Revision 7713 (metacat): Move the solr query part here since metacat-index needs to query the solr server too.
- 03:37 PM Revision 7712 (metacat): Use the sepcified exceptions to replace the Exception.
05/17/2013
- 05:01 PM Revision 7711 (metacat): Add the code to add a resource map information.
- 04:07 PM Revision 7710 (metacat): Add code the transform query response to SolrDoc.
- 02:22 PM Revision 7709 (metacat): Move the cod which transformed the query response to the inputstream to the metacat-common module.
- Remove some obsoleted imports.
- 02:05 PM Revision 7708 (metacat): Add a transformer to process the query response object.
- 12:57 PM Revision 7707 (metacat): Move the code to generate the QueryResponseWriter to the metacat-common module. So it can be shared with the metacat-index module.
- 12:40 PM Revision 7706 (metacat): Add a new class to create the solr QueryResponseWriter.
- 10:58 AM Feature #5936 (Closed): Include certificate delegation inside Metacat
- Like the d1_portal, we want to be able to have users authenticate with their preferred Identity Provider via the brow...
- 10:52 AM Task #5928: Make sure to start metacat-index web apps when the metacat web apps is start-up during the tomcat startup
- Is there a way to tell Tomcat which order to load the webapps? I have seen some errors on mn-demo-4 when metacat-inde...
- 10:44 AM Task #5935 (Rejected): Display the PID not the docid in the rendered metadata
- I think we decided that rendering the PID on the EML view page is not a requirement at this time, though I still thin...
- 10:41 AM Revision 7705 (metacat): use pid, not docid as the parameter to the registry when editing/deleting objects since the vlaue is really the pid. https://projects.ecoinformatics.org/ecoinfo/issues/5932
- 10:29 AM Revision 7704 (metacat): exclude jibx-tools, maven cruft and org.eclipse "dependencies" to cut down on the number of jar files in the lib folder.
- 10:27 AM Revision 7703 (metacat): organize imports
- 09:57 AM Revision 7702 (metacat): format before editing
05/16/2013
- 04:02 PM Revision 7701 (metacat): use java 1.6 for maven compiler
- 03:42 PM Task #5932 (Closed): Allow use of PID (or docid) using cgi-registry modification form
- The registry now calls a new Metacat servlet method: getDocid(pid). Then it continues on as though the docid was prov...
- 03:22 PM Revision 7700 (metacat): use correct test for PID param.
- 03:21 PM Revision 7699 (metacat): remove extra lines from returned <docid/> block. https://projects.ecoinformatics.org/ecoinfo/issues/5932
- 01:53 PM Revision 7698 (metacat): move to libclient/common snapshot to match metacat-common project.
- 12:04 PM Revision 7697 (metacat): Use the new class to handle the resource map.
05/15/2013
- 05:48 PM Revision 7696 (metacat): Modify the subprocessor to use SolrServer rather than solr service url.
- 05:16 PM Revision 7695 (metacat): Allow use of PID instead of docid in the Perl registry. At least for reading/editing and deleting existing content. Does not create content using a pid. https://projects.ecoinformatics.org/ecoinfo/issues/5932
- 03:53 PM Revision 7694 (metacat): initialize the SOLR home directory if it does not already exist.
05/14/2013
- 06:10 PM Revision 7693 (metacat): Add code to test title for the query result.
- 05:51 PM Revision 7692 (metacat): Rewrite some methods, so the query result can be processed many times.
05/13/2013
- 09:23 PM Revision 7691 (metacat): Add code to wait the readiness of the hazelcast service.
- 04:34 PM Revision 7690 (metacat): Use the DistributedMapsFactory to get maps.
- 03:51 PM Revision 7689 (metacat): Add a class to get distributed maps such as the system metadata map.
- 11:58 AM Revision 7688 (metacat): Removed the comment on the process of building index.
- 11:55 AM Revision 7687 (metacat): Add code to wait the startup of the MNode.
05/10/2013
- 04:09 PM Task #5905: Implement access control filter
- I also tested that if there is a fq in the user specified query, it wouldn't cause any issue:
The user specified q... - 04:04 PM Task #5905: Implement access control filter
- Append filter query (fq) to the main query. The fq looks like:
fq=(readPermission:"public")OR(readPermission:"CN=ben... - 03:02 PM Revision 7686 (metacat): Removed the commented out text cases and add a test for archvied document.
- 02:59 PM Revision 7685 (metacat): Only after reloading the core, the query result can reflect the change made in metacat-index module.
- 02:37 PM Revision 7684 (metacat): Use the log info to replace the print.
- 02:36 PM Revision 7683 (metacat): Add log information for archiving or inserting indexes.
05/09/2013
- 01:26 PM Bug #3017 (Closed): The Ampersand returns a blank page on searches
- This is in trunk now, though it is not applicable for solr queries. (I'm sure there are other character escaping issu...
- 12:11 PM Revision 7682 (metacat): Consider mutithread issue there.
- 12:10 PM Revision 7681 (metacat): Fixed a bug to put "OR" correctly in the query.
- And remove the user "authorized_user" from the rightsholder clause in the query.
- 10:04 AM Revision 7680 (metacat): Use the set of subjects to replace the user and groups for the solr query.
- 12:19 AM Revision 7679 (metacat): Call setting certificate location to be the test one after getting the MN baseurl.
- The method to getting MN baseurl somehow calls CN and it set the certificate location to be /var/metacat/certs/METACA...
05/08/2013
- 10:14 PM Revision 7678 (metacat): escape reserved XML characters when constructing a pathquery from user input (&). https://projects.ecoinformatics.org/ecoinfo/issues/3017
- 05:21 PM Revision 7677 (metacat): escape special XML characters when constructing a pathquery from user input (&). https://projects.ecoinformatics.org/ecoinfo/issues/3017
- 04:53 PM Revision 7676 (metacat): Add a distrusted certificate for the testing.
- 04:53 PM Revision 7675 (metacat): Use the MNode to query the server when we use certificates to set up the session.
- 03:35 PM Bug #3085 (Resolved): document the impact of re-deploying on metacat.properties
- Added clarification and a note to documentation about what is backed up and when a full metacat.properties file shoul...
- 03:34 PM Revision 7674 (metacat): Add the code to test the user with a distrusted certificate.
- 03:34 PM Revision 7673 (metacat): add note about metacat.properties backup. https://projects.ecoinformatics.org/ecoinfo/issues/3085
- 03:14 PM Task #5931 (Closed): Allow use of PID (or docid) using Metacat API "read" action
- This was committed along with a different change. But might as well stick with it as I think it will be important as ...
- 03:10 PM Revision 7672 (metacat): remove xslt sort for solr search results in case someone actually specifies a sort field in the query. https://projects.ecoinformatics.org/ecoinfo/issues/3397
- 03:05 PM Bug #3397 (Resolved): metacat needs server side sort mechanism
- SOLR queries can specify a sort field 9or even a function!) - we will not implement this in pathquery.
- 02:58 PM Bug #3816 (Closed): Export action doesn't create complete package
- this has been corrected in trunk and will be included in the next release built from trunk.
- 02:58 PM Revision 7671 (metacat): adjust action=zip behavior to use full docids and entity names (data files) for the zip entry. Also uses the given qformat to render the metadata. https://projects.ecoinformatics.org/ecoinfo/issues/3816
- 10:42 AM Revision 7670 (metacat): Add tests to test group and rightsholder.
- 10:41 AM Revision 7669 (metacat): Add the rightsHolder in the access filter.
05/07/2013
- 08:34 PM Revision 7668 (metacat): Change the delete to archive.
- 06:52 PM Revision 7667 (metacat): Add a test for testing access control for the solr query.
- 05:29 PM Revision 7666 (metacat): force the use of newer xercesImpl 2.7.1
- 04:54 PM Revision 7665 (metacat): adjust action=zip behavior to use full docids when checking for permissions/existence. https://projects.ecoinformatics.org/ecoinfo/issues/3816
05/06/2013
- 11:44 AM Bug #5929 (Closed): replication update action times out
- This is included in the 2.0.7 release targeted for CN environment in DataONE.
- 11:43 AM Bug #5925 (Resolved): Clean up the jar files in the Metacat deploy directory
- Now using Maven to handle dependencies, especially as relates to geotools for generating the spatial cache and all th...
05/03/2013
- 03:52 PM Task #5935 (Rejected): Display the PID not the docid in the rendered metadata
- This would apply to the FGDC and EML stylesheets. It means adding in an additional parameter (default to docid if pid...
- 09:53 AM Task #5932: Allow use of PID (or docid) using cgi-registry modification form
- Another option would be to "look-up" the docid from Metacat using the pid and a yet-to-be-defined new Metacat API ser...
- 09:44 AM Task #5932 (Closed): Allow use of PID (or docid) using cgi-registry modification form
- The SOLR search results XSLT includes a section for editing packages via the Perl registry. But the registry assumes ...
- 09:40 AM Task #5931 (Closed): Allow use of PID (or docid) using Metacat API "read" action
- The SOLR results use PID as the identifier but the Metacat API for rendering metadata uses docid. We should allow opt...
05/02/2013
- 05:30 PM Revision 7664 (metacat): Add code to handle query for the http solr server.
- 04:58 PM Revision 7663 (metacat): Add some java doc.
- 01:54 PM Task #5916 (Closed): Create resultset-solr.xsl
- This is available in the style/common directory for all skins to utilize. Skins can either include the root style she...
- 01:52 PM Task #5917 (Closed): Create NCEAS search form that uses SOLR instead of pathquery
- It's very basic, but works. And searches should be so much quicker!
- 12:47 PM Revision 7662 (metacat): Use a new class to handle the solr query engine description request.
- 12:46 PM Revision 7661 (metacat): Copy the index fields description file to the web-inf dir.
- 12:44 PM Revision 7660 (metacat): A property file to describe the index fields.
- 11:30 AM Revision 7659 (metacat): Fixed a bug the collection name was not set.
- 11:21 AM Revision 7658 (metacat): Catch a new exception base on the change of the calling method.
- 10:56 AM Revision 7657 (metacat): Add getCollectionName method.
Also available in: Atom