Activity
From 04/23/2013 to 05/22/2013
05/22/2013
- 04:44 PM Revision 7755 (metacat): load the evicted SM back into the map on a "Refresh" so that listeners hear the update. (metacat-index, for example)
- 04:18 PM Revision 7754 (metacat): set the dataone.hazelcast.location.clientconfig property to be the local config file. It will be used in the d1_cn_index_processor
- 03:50 PM Revision 7753 (metacat): use DataONE cilogon skin so that we are not prompted for certificate duration (and to make it more cohesive as a prototype using the Metacat DataONE skin
- 03:49 PM Revision 7752 (metacat): switch back to log4j statements now that I am sure certificate delegation is working.
- 03:35 PM Revision 7751 (metacat): add link to login or switch user
- 03:29 PM Revision 7750 (metacat): add simple page to show logged-in user subject DN
- 03:02 PM Revision 7749 (metacat): use System.out.println until the oa4mp logging issue is resolved.
- 11:23 AM Task #5928 (Closed): Make sure to start metacat-index web apps when the metacat web apps is start-up during the tomcat startup
- Added the mechanism to wait some time if the MNode or the Hazelcast is not ready. We also add a maximum attempt contr...
- 11:19 AM Revision 7748 (metacat): Read the waiting time and maximum attempts from the metacat.properties file.
- 10:56 AM Revision 7747 (metacat): Add the waiting time and max attempts for regenerating the index.
- 10:36 AM Revision 7746 (metacat): use filesystem store. in-memory store seems to get into deadlock with multiple environments accessing it.
- 10:34 AM Revision 7745 (metacat): add logging for portal certificate look up process.
- 10:29 AM Revision 7744 (metacat): Add some new properties for the test.
- 09:34 AM Revision 7743 (metacat): use relative path for oa4mp_client.xml (within servlet context). https://projects.ecoinformatics.org/ecoinfo/issues/5936
- 08:05 AM Revision 7742 (metacat): use in-memory storage for myproxy. at least for now.
05/21/2013
- 11:35 PM Revision 7741 (metacat): Add a property containing the list of the resource map namespaces.
- 11:33 PM Revision 7740 (metacat): Index the none-resourcemap data objects first, then the resourcemap objects.
- 11:32 PM Revision 7739 (metacat): used pk8 format private key file name
- 11:12 PM Revision 7738 (metacat): use java 1.6 compliance for compilation
- 11:05 PM Revision 7737 (metacat): first pass at integrating CILogon/MyProxy certificates in Metacat. Configuration is specific to mn-demo-4.test.dataone.org for the time being (this will cause localhost deployments to fail webapp deployment). https://projects.ecoinformatics.org/ecoinfo/issues/5936
- 11:05 PM Revision 7736 (metacat): first pass at integrating CILogon/MyProxy certificates in Metacat. Configuration is specific to mn-demo-4.test.dataone.org for the time being (this will cause localhost deployments to fail webapp deployment). https://projects.ecoinformatics.org/ecoinfo/issues/5936
- 07:35 PM Bug #5938 (Closed): sitemap format is deprecated
- Updated the namespace to use the 0.9 format. No other changes needed for validity. Committed in r7735.
- 07:34 PM Bug #5938 (Closed): sitemap format is deprecated
- The sitemap format used by Metacat has been deprecated, and should be updated to the current release (0.9) as publish...
- 07:32 PM Revision 7735 (metacat): Updated Sitemap generation to use latest version of the sitemap protocol schemas.
- 02:23 PM Revision 7734 (metacat): Try to get the metacat ids first.
- 11:29 AM Task #5937 (Closed): The solr indexes of the data file contains obsoleted ids for the resourcemap and documentBy element after updating a data package
- I used a morpho to insert a data package which includes the data file, eml file and resource map file to the metacat....
- 11:11 AM Revision 7733 (metacat): Use SolrQueryServiceController to get the ValidFieldList.
05/20/2013
- 11:39 PM Revision 7732 (metacat): Remove the code to throw an exception if the subjects is null in the query method.
- 10:42 PM Revision 7731 (metacat): changed the xpath to get solr_spec_version.
- 10:20 PM Revision 7730 (metacat): Use the SolrQueryServiceController to get the spec version and index schema information.
- 10:18 PM Revision 7729 (metacat): Add some new properties for the http solr server.
- 09:33 PM Revision 7728 (metacat): Add the getIndexFields and getValidIndexFields methods.
- 04:27 PM Revision 7727 (metacat): Add getSolrSpecVersion method for the HttpServer and the controller.
- 02:12 PM Revision 7726 (metacat): Imeplement the getSolrVersion, getSchemaField and getValidSchemaFields for the EmbeddedSolrServer.
05/18/2013
- 11:46 PM Revision 7725 (metacat): Change the package of SolrQueryReponseWriterFactory and SolrQueryResponseTransformer.
- 11:32 PM Revision 7724 (metacat): Change the package name of SolrQueryResponseTransformer and SolrQueryWriterFactory.
- 11:29 PM Revision 7723 (metacat): move SolrQueryResponseTransformer and SolrQueryReponseWriterFactory to the query package.
- 11:28 PM Revision 7722 (metacat): Move it to the query package.
- 11:28 PM Revision 7721 (metacat): move it to the query package.
- 11:24 PM Revision 7720 (metacat): Use the new query(SolrParams param) method of the SolrQueryServiceController.
- 11:22 PM Revision 7719 (metacat): Remove the query(String query) methods from SolrQueryService.java.
- 05:19 PM Revision 7718 (metacat): Use the SolrQueryServiceController class to handle the query.
- 04:49 PM Revision 7717 (metacat): Add the depency of d1_cn_index_processor here.
- 04:48 PM Revision 7716 (metacat): Remove the depency on the d1_cn_index_processor.
- 03:57 PM Revision 7715 (metacat): move the depency of d1_cn_index_processor to the metacat-common module.
- 03:56 PM Revision 7714 (metacat): Use one query method in the SolrQueryServiceController.
- 03:38 PM Revision 7713 (metacat): Move the solr query part here since metacat-index needs to query the solr server too.
- 03:37 PM Revision 7712 (metacat): Use the sepcified exceptions to replace the Exception.
05/17/2013
- 05:01 PM Revision 7711 (metacat): Add the code to add a resource map information.
- 04:07 PM Revision 7710 (metacat): Add code the transform query response to SolrDoc.
- 02:22 PM Revision 7709 (metacat): Move the cod which transformed the query response to the inputstream to the metacat-common module.
- Remove some obsoleted imports.
- 02:05 PM Revision 7708 (metacat): Add a transformer to process the query response object.
- 12:57 PM Revision 7707 (metacat): Move the code to generate the QueryResponseWriter to the metacat-common module. So it can be shared with the metacat-index module.
- 12:40 PM Revision 7706 (metacat): Add a new class to create the solr QueryResponseWriter.
- 10:58 AM Feature #5936 (Closed): Include certificate delegation inside Metacat
- Like the d1_portal, we want to be able to have users authenticate with their preferred Identity Provider via the brow...
- 10:52 AM Task #5928: Make sure to start metacat-index web apps when the metacat web apps is start-up during the tomcat startup
- Is there a way to tell Tomcat which order to load the webapps? I have seen some errors on mn-demo-4 when metacat-inde...
- 10:44 AM Task #5935 (Rejected): Display the PID not the docid in the rendered metadata
- I think we decided that rendering the PID on the EML view page is not a requirement at this time, though I still thin...
- 10:41 AM Revision 7705 (metacat): use pid, not docid as the parameter to the registry when editing/deleting objects since the vlaue is really the pid. https://projects.ecoinformatics.org/ecoinfo/issues/5932
- 10:29 AM Revision 7704 (metacat): exclude jibx-tools, maven cruft and org.eclipse "dependencies" to cut down on the number of jar files in the lib folder.
- 10:27 AM Revision 7703 (metacat): organize imports
- 09:57 AM Revision 7702 (metacat): format before editing
05/16/2013
- 04:02 PM Revision 7701 (metacat): use java 1.6 for maven compiler
- 03:42 PM Task #5932 (Closed): Allow use of PID (or docid) using cgi-registry modification form
- The registry now calls a new Metacat servlet method: getDocid(pid). Then it continues on as though the docid was prov...
- 03:22 PM Revision 7700 (metacat): use correct test for PID param.
- 03:21 PM Revision 7699 (metacat): remove extra lines from returned <docid/> block. https://projects.ecoinformatics.org/ecoinfo/issues/5932
- 01:53 PM Revision 7698 (metacat): move to libclient/common snapshot to match metacat-common project.
- 12:04 PM Revision 7697 (metacat): Use the new class to handle the resource map.
05/15/2013
- 05:48 PM Revision 7696 (metacat): Modify the subprocessor to use SolrServer rather than solr service url.
- 05:16 PM Revision 7695 (metacat): Allow use of PID instead of docid in the Perl registry. At least for reading/editing and deleting existing content. Does not create content using a pid. https://projects.ecoinformatics.org/ecoinfo/issues/5932
- 03:53 PM Revision 7694 (metacat): initialize the SOLR home directory if it does not already exist.
05/14/2013
- 06:10 PM Revision 7693 (metacat): Add code to test title for the query result.
- 05:51 PM Revision 7692 (metacat): Rewrite some methods, so the query result can be processed many times.
05/13/2013
- 09:23 PM Revision 7691 (metacat): Add code to wait the readiness of the hazelcast service.
- 04:34 PM Revision 7690 (metacat): Use the DistributedMapsFactory to get maps.
- 03:51 PM Revision 7689 (metacat): Add a class to get distributed maps such as the system metadata map.
- 11:58 AM Revision 7688 (metacat): Removed the comment on the process of building index.
- 11:55 AM Revision 7687 (metacat): Add code to wait the startup of the MNode.
05/10/2013
- 04:09 PM Task #5905: Implement access control filter
- I also tested that if there is a fq in the user specified query, it wouldn't cause any issue:
The user specified q... - 04:04 PM Task #5905: Implement access control filter
- Append filter query (fq) to the main query. The fq looks like:
fq=(readPermission:"public")OR(readPermission:"CN=ben... - 03:02 PM Revision 7686 (metacat): Removed the commented out text cases and add a test for archvied document.
- 02:59 PM Revision 7685 (metacat): Only after reloading the core, the query result can reflect the change made in metacat-index module.
- 02:37 PM Revision 7684 (metacat): Use the log info to replace the print.
- 02:36 PM Revision 7683 (metacat): Add log information for archiving or inserting indexes.
05/09/2013
- 01:26 PM Bug #3017 (Closed): The Ampersand returns a blank page on searches
- This is in trunk now, though it is not applicable for solr queries. (I'm sure there are other character escaping issu...
- 12:11 PM Revision 7682 (metacat): Consider mutithread issue there.
- 12:10 PM Revision 7681 (metacat): Fixed a bug to put "OR" correctly in the query.
- And remove the user "authorized_user" from the rightsholder clause in the query.
- 10:04 AM Revision 7680 (metacat): Use the set of subjects to replace the user and groups for the solr query.
- 12:19 AM Revision 7679 (metacat): Call setting certificate location to be the test one after getting the MN baseurl.
- The method to getting MN baseurl somehow calls CN and it set the certificate location to be /var/metacat/certs/METACA...
05/08/2013
- 10:14 PM Revision 7678 (metacat): escape reserved XML characters when constructing a pathquery from user input (&). https://projects.ecoinformatics.org/ecoinfo/issues/3017
- 05:21 PM Revision 7677 (metacat): escape special XML characters when constructing a pathquery from user input (&). https://projects.ecoinformatics.org/ecoinfo/issues/3017
- 04:53 PM Revision 7676 (metacat): Add a distrusted certificate for the testing.
- 04:53 PM Revision 7675 (metacat): Use the MNode to query the server when we use certificates to set up the session.
- 03:35 PM Bug #3085 (Resolved): document the impact of re-deploying on metacat.properties
- Added clarification and a note to documentation about what is backed up and when a full metacat.properties file shoul...
- 03:34 PM Revision 7674 (metacat): Add the code to test the user with a distrusted certificate.
- 03:34 PM Revision 7673 (metacat): add note about metacat.properties backup. https://projects.ecoinformatics.org/ecoinfo/issues/3085
- 03:14 PM Task #5931 (Closed): Allow use of PID (or docid) using Metacat API "read" action
- This was committed along with a different change. But might as well stick with it as I think it will be important as ...
- 03:10 PM Revision 7672 (metacat): remove xslt sort for solr search results in case someone actually specifies a sort field in the query. https://projects.ecoinformatics.org/ecoinfo/issues/3397
- 03:05 PM Bug #3397 (Resolved): metacat needs server side sort mechanism
- SOLR queries can specify a sort field 9or even a function!) - we will not implement this in pathquery.
- 02:58 PM Bug #3816 (Closed): Export action doesn't create complete package
- this has been corrected in trunk and will be included in the next release built from trunk.
- 02:58 PM Revision 7671 (metacat): adjust action=zip behavior to use full docids and entity names (data files) for the zip entry. Also uses the given qformat to render the metadata. https://projects.ecoinformatics.org/ecoinfo/issues/3816
- 10:42 AM Revision 7670 (metacat): Add tests to test group and rightsholder.
- 10:41 AM Revision 7669 (metacat): Add the rightsHolder in the access filter.
05/07/2013
- 08:34 PM Revision 7668 (metacat): Change the delete to archive.
- 06:52 PM Revision 7667 (metacat): Add a test for testing access control for the solr query.
- 05:29 PM Revision 7666 (metacat): force the use of newer xercesImpl 2.7.1
- 04:54 PM Revision 7665 (metacat): adjust action=zip behavior to use full docids when checking for permissions/existence. https://projects.ecoinformatics.org/ecoinfo/issues/3816
05/06/2013
- 11:44 AM Bug #5929 (Closed): replication update action times out
- This is included in the 2.0.7 release targeted for CN environment in DataONE.
- 11:43 AM Bug #5925 (Resolved): Clean up the jar files in the Metacat deploy directory
- Now using Maven to handle dependencies, especially as relates to geotools for generating the spatial cache and all th...
05/03/2013
- 03:52 PM Task #5935 (Rejected): Display the PID not the docid in the rendered metadata
- This would apply to the FGDC and EML stylesheets. It means adding in an additional parameter (default to docid if pid...
- 09:53 AM Task #5932: Allow use of PID (or docid) using cgi-registry modification form
- Another option would be to "look-up" the docid from Metacat using the pid and a yet-to-be-defined new Metacat API ser...
- 09:44 AM Task #5932 (Closed): Allow use of PID (or docid) using cgi-registry modification form
- The SOLR search results XSLT includes a section for editing packages via the Perl registry. But the registry assumes ...
- 09:40 AM Task #5931 (Closed): Allow use of PID (or docid) using Metacat API "read" action
- The SOLR results use PID as the identifier but the Metacat API for rendering metadata uses docid. We should allow opt...
05/02/2013
- 05:30 PM Revision 7664 (metacat): Add code to handle query for the http solr server.
- 04:58 PM Revision 7663 (metacat): Add some java doc.
- 01:54 PM Task #5916 (Closed): Create resultset-solr.xsl
- This is available in the style/common directory for all skins to utilize. Skins can either include the root style she...
- 01:52 PM Task #5917 (Closed): Create NCEAS search form that uses SOLR instead of pathquery
- It's very basic, but works. And searches should be so much quicker!
- 12:47 PM Revision 7662 (metacat): Use a new class to handle the solr query engine description request.
- 12:46 PM Revision 7661 (metacat): Copy the index fields description file to the web-inf dir.
- 12:44 PM Revision 7660 (metacat): A property file to describe the index fields.
- 11:30 AM Revision 7659 (metacat): Fixed a bug the collection name was not set.
- 11:21 AM Revision 7658 (metacat): Catch a new exception base on the change of the calling method.
- 10:56 AM Revision 7657 (metacat): Add getCollectionName method.
05/01/2013
- 10:57 PM Revision 7656 (metacat): Use solr query instead of path query in the NCEAS skin. https://projects.ecoinformatics.org/ecoinfo/issues/5917
- 10:55 PM Revision 7655 (metacat): only build metacat-common before compile, no need to do it every time the ant build is called.
- 05:17 PM Revision 7654 (metacat): enable solr results rendering on the server side for the NCEAS skin (does not include UI for generating the query yet). https://projects.ecoinformatics.org/ecoinfo/issues/5917
- 05:16 PM Revision 7653 (metacat): clean-up the solr results XSLT to display more pertinent fields (many paths have been collapsed into single fields, etc). Added an abstract snippet in the search results (140 chars). https://projects.ecoinformatics.org/ecoinfo/issues/5916
- 04:57 PM Revision 7652 (metacat): Add double quotes to surrend the user or group names in the access fq. This will fix the issue if the names have white spaces.
- 03:33 PM Revision 7650 (metacat): rename resultset template to include "solr" so that it can co-exist with the original pathquery-based resultset template (NCEAS skin approach that has one style sheet for multiple document types)
- 03:20 PM Revision 7649 (metacat): use charset=UTF-8 for the meta tag (especially important if other international sites want to use default skin with translations)
- 02:38 PM Revision 7648 (metacat): Add the access query filter.
- 01:53 PM Revision 7647 (metacat): Allow use of server-side XSLT for SOLR queries that include "wt=<qformat>". https://projects.ecoinformatics.org/ecoinfo/issues/5812
- 01:53 PM Revision 7646 (metacat): Allow use of server-side XSLT for SOLR queries that include "wt=<qformat>". https://projects.ecoinformatics.org/ecoinfo/issues/5812
- 01:52 PM Revision 7645 (metacat): Allow use of server-side XSLT for SOLR queries that include "wt=<qformat>". https://projects.ecoinformatics.org/ecoinfo/issues/5812
- 12:51 PM Task #5917: Create NCEAS search form that uses SOLR instead of pathquery
- This is under way. Decided to use the server-side XSLT processing we already have for pathquery and other XML->HTML t...
- 07:37 AM Revision 7644 (metacat): Use 2.0.7 version number in configuration/upgrade/docs (trunk, even though we will not be releasing 2.0.7 from trunk, we want to have the upgrade scripts included here)
04/30/2013
- 09:35 PM Revision 7639 (metacat): Allow null SM.submitter (per schema). There were null values in cn-dev (and probably elsewhere since it is technically allowed in the schema. But with a null value, we need to have a null Subject for the SM.submitter field, not a Subject with a null getValue() return. Encountered this when testing for: https://projects.ecoinformatics.org/ecoinfo/issues/5929.
- 04:47 PM Revision 7638 (metacat): add space to prevent syntax error when additional clause is appended. https://projects.ecoinformatics.org/ecoinfo/issues/5929.
- 03:24 PM Task #5930 (Closed): Add a configuration property to turn on/off the indexing processes (SOLR indexing)
- So that Metacat installations can decide which (or both) they want to use.
The proposed the property is:
dbquery.... - 03:09 PM Revision 7637 (metacat): move maven.home into build properties (hoping that hudson will be able to correctly override it)
- 03:02 PM Bug #5929: replication update action times out
- By replacing the subselect with a LEFT JOIN, we can massively improve performance of this query. Here's the same quer...
- 02:35 PM Bug #5929 (Closed): replication update action times out
- With large database sizes, the replications "update" action times out with normal settings for HTTP timeouts. On Dat...
- 02:50 PM Revision 7636 (metacat): CHange replication 'update' query to use a LEFT JOIN so that the performance of the replication update action is improved, which had been causing an HTTP timeout for large metacat installations. See https://projects.ecoinformatics.org/ecoinfo/issues/5929.
- 02:46 PM Revision 7635 (metacat): Add the code to read the index field information from the schema.xml.
- 02:35 PM Task #5928 (Closed): Make sure to start metacat-index web apps when the metacat web apps is start-up during the tomcat startup
- Metacat-index will use the distribute system metadata map from the metacat during the startup. So we need to make sur...
04/29/2013
- 04:59 PM Revision 7634 (metacat): Add code to handle the solr index information. we still need to figure out how to get the information.
- 09:16 AM Revision 7633 (metacat): Add the code to build the metacat-common.
- 09:15 AM Revision 7632 (metacat): Add the dependency to the metacat-common.
- 09:14 AM Revision 7631 (metacat): use the sharing code from the metacat-common.
- 09:13 AM Revision 7630 (metacat): Move this class to the metacat-common.
- 09:11 AM Revision 7629 (metacat): Remove the bin directory.
- 09:10 AM Revision 7628 (metacat): Add a new module for sharing the between the metacat and metacat-index.
04/26/2013
- 05:12 PM Revision 7627 (metacat): Use an update method to replace the insert and update methods in the SolrIndex class.
- 05:10 PM Revision 7626 (metacat): Add a system metadata for archiving.
04/25/2013
- 05:09 PM Revision 7625 (metacat): Add the solr engine to the engine list.
- 02:40 PM Task #5927 (Resolved): Implements listing the query engines and fields for the solr query engine
- We need to add the solr query engine the mn api:
GET /query/{queryType}
GET /query
- 02:18 PM Revision 7624 (metacat): create the lib/maven directory in order to resolve the deps and carry on with the build.
- 02:04 PM Revision 7623 (metacat): require utilities.jar before compile
- 01:13 PM Task #5926 (Resolved): Metacat-index doesn't build solr index during a Metadata inserting document
- I inserted an eml document through the metacat devtools.html, but the metacat-index didn't build the solr index.
- 12:50 PM Bug #5919 (Resolved): Upgrade KNB to 2.0.6
- 11:13 AM Bug #5925 (Resolved): Clean up the jar files in the Metacat deploy directory
- In the knb/WEB-INF/lib, i saw some jar files like:
spring-aop-2.5.5.jar spring-context-2.5.5.jar spring-core-2.5.... - 11:09 AM Task #5924 (Closed): Add the configuration page for the Solr server in the Metacat configuration utility
- We need to a new page for the Solr server in the Metacat configuration utility.
This page will include the server ... - 11:05 AM Task #5923 (Closed): Discover missed documents and queue them for indexing
- Metacat-index needs a mechanism to regenerate the solr index for those missed during the insert or update. This mecha...
- 11:01 AM Task #5922 (Closed): Modify the solr index configuration for the resource map
- The index configuration of the resource map in the d1_cn_index_processor has parameter - service url which we don't u...
- 10:58 AM Task #5921 (Closed): Design and implement Metacat-Index module
- We decided we would use as much the code of the d1_cn_index_processor as possible. The d1_cn_index_processor uses S...
04/24/2013
- 07:34 PM Revision 7622 (metacat): use maven to manage most jar dependencies in Metacat.
- Exceptions include: LSID, Datamamager (EML),
- 05:03 PM Revision 7621 (metacat): Include solr and lucene jar files into the war. We maybe change it when we use pom.xml to manage the jar dependency.
- 05:00 PM Revision 7620 (metacat): Add code to handle solr query.
- 04:29 PM Bug #5920 (Resolved): Test PG dump on dev server
- I restored the production dump to dev2.nceas.ucsb.edu (1-2 hours to load).
Installed Metacat v2.0.3.
Configured.
U... - 12:41 PM Bug #5920 (Resolved): Test PG dump on dev server
- dev2.nceas.ucsb.edu is our testing area and I will load the production dump there for testing the 2.0.3->2.0.6 upgrade
- 04:12 PM Revision 7619 (metacat): Remove the jar file.
- 04:10 PM Revision 7618 (metacat): Add the jar file which will be used by solr.
- 04:09 PM Revision 7617 (metacat): Add 3.4 jar files.
- 04:08 PM Revision 7616 (metacat): Remove the 4.1 jar files.
- 04:06 PM Revision 7615 (metacat): Add the 3.4 jar.
- 04:05 PM Revision 7614 (metacat): Remove the 4.1 jar and use the 3.4 jar.
- 02:23 PM Revision 7613 (metacat): Add code to generate index during the servlet initialization.
- 12:40 PM Bug #5919 (Resolved): Upgrade KNB to 2.0.6
- Steps for upgrading:
- stop Tomcat... - 12:12 PM Revision 7612 (metacat): Add the names of the system metadata and the data object imaps.
- 11:57 AM Revision 7611 (metacat): Use the Identifier object rather than String as the key to get the system metadata and object path.
- 11:30 AM Revision 7610 (metacat): Add code to start hazelcast.
04/23/2013
- 05:56 PM Revision 7609 (metacat): Add a class to handle solr query.
- 04:26 PM Revision 7608 (metacat): Remove those obsolete index classes.
- 04:17 PM Revision 7607 (metacat): Add a junit test for the IndexGenerator class.
- 12:00 PM Revision 7606 (metacat): Add a new class which will generate index for the ids.
Also available in: Atom