Project

General

Profile

Activity

From 04/14/2013 to 05/13/2013

05/13/2013

09:23 PM Revision 7691 (metacat): Add code to wait the readiness of the hazelcast service.
Jing Tao
04:34 PM Revision 7690 (metacat): Use the DistributedMapsFactory to get maps.
Jing Tao
03:51 PM Revision 7689 (metacat): Add a class to get distributed maps such as the system metadata map.
Jing Tao
11:58 AM Revision 7688 (metacat): Removed the comment on the process of building index.
Jing Tao
11:55 AM Revision 7687 (metacat): Add code to wait the startup of the MNode.
Jing Tao

05/10/2013

04:09 PM Task #5905: Implement access control filter
I also tested that if there is a fq in the user specified query, it wouldn't cause any issue:
The user specified q...
Jing Tao
04:04 PM Task #5905: Implement access control filter
Append filter query (fq) to the main query. The fq looks like:
fq=(readPermission:"public")OR(readPermission:"CN=ben...
Jing Tao
03:02 PM Revision 7686 (metacat): Removed the commented out text cases and add a test for archvied document.
Jing Tao
02:59 PM Revision 7685 (metacat): Only after reloading the core, the query result can reflect the change made in metacat-index module.
Jing Tao
02:37 PM Revision 7684 (metacat): Use the log info to replace the print.
Jing Tao
02:36 PM Revision 7683 (metacat): Add log information for archiving or inserting indexes.
Jing Tao

05/09/2013

01:26 PM Bug #3017 (Closed): The Ampersand returns a blank page on searches
This is in trunk now, though it is not applicable for solr queries. (I'm sure there are other character escaping issu... ben leinfelder
12:11 PM Revision 7682 (metacat): Consider mutithread issue there.
Jing Tao
12:10 PM Revision 7681 (metacat): Fixed a bug to put "OR" correctly in the query.
And remove the user "authorized_user" from the rightsholder clause in the query. Jing Tao
10:04 AM Revision 7680 (metacat): Use the set of subjects to replace the user and groups for the solr query.
Jing Tao
12:19 AM Revision 7679 (metacat): Call setting certificate location to be the test one after getting the MN baseurl.
The method to getting MN baseurl somehow calls CN and it set the certificate location to be /var/metacat/certs/METACA... Jing Tao

05/08/2013

10:14 PM Revision 7678 (metacat): escape reserved XML characters when constructing a pathquery from user input (&). https://projects.ecoinformatics.org/ecoinfo/issues/3017
ben leinfelder
05:21 PM Revision 7677 (metacat): escape special XML characters when constructing a pathquery from user input (&). https://projects.ecoinformatics.org/ecoinfo/issues/3017
ben leinfelder
04:53 PM Revision 7676 (metacat): Add a distrusted certificate for the testing.
Jing Tao
04:53 PM Revision 7675 (metacat): Use the MNode to query the server when we use certificates to set up the session.
Jing Tao
03:35 PM Bug #3085 (Resolved): document the impact of re-deploying on metacat.properties
Added clarification and a note to documentation about what is backed up and when a full metacat.properties file shoul... ben leinfelder
03:34 PM Revision 7674 (metacat): Add the code to test the user with a distrusted certificate.
Jing Tao
03:34 PM Revision 7673 (metacat): add note about metacat.properties backup. https://projects.ecoinformatics.org/ecoinfo/issues/3085
ben leinfelder
03:14 PM Task #5931 (Closed): Allow use of PID (or docid) using Metacat API "read" action
This was committed along with a different change. But might as well stick with it as I think it will be important as ... ben leinfelder
03:10 PM Revision 7672 (metacat): remove xslt sort for solr search results in case someone actually specifies a sort field in the query. https://projects.ecoinformatics.org/ecoinfo/issues/3397
ben leinfelder
03:05 PM Bug #3397 (Resolved): metacat needs server side sort mechanism
SOLR queries can specify a sort field 9or even a function!) - we will not implement this in pathquery. ben leinfelder
02:58 PM Bug #3816 (Closed): Export action doesn't create complete package
this has been corrected in trunk and will be included in the next release built from trunk. ben leinfelder
02:58 PM Revision 7671 (metacat): adjust action=zip behavior to use full docids and entity names (data files) for the zip entry. Also uses the given qformat to render the metadata. https://projects.ecoinformatics.org/ecoinfo/issues/3816
ben leinfelder
10:42 AM Revision 7670 (metacat): Add tests to test group and rightsholder.
Jing Tao
10:41 AM Revision 7669 (metacat): Add the rightsHolder in the access filter.
Jing Tao

05/07/2013

08:34 PM Revision 7668 (metacat): Change the delete to archive.
Jing Tao
06:52 PM Revision 7667 (metacat): Add a test for testing access control for the solr query.
Jing Tao
05:29 PM Revision 7666 (metacat): force the use of newer xercesImpl 2.7.1
ben leinfelder
04:54 PM Revision 7665 (metacat): adjust action=zip behavior to use full docids when checking for permissions/existence. https://projects.ecoinformatics.org/ecoinfo/issues/3816
ben leinfelder

05/06/2013

11:44 AM Bug #5929 (Closed): replication update action times out
This is included in the 2.0.7 release targeted for CN environment in DataONE. ben leinfelder
11:43 AM Bug #5925 (Resolved): Clean up the jar files in the Metacat deploy directory
Now using Maven to handle dependencies, especially as relates to geotools for generating the spatial cache and all th... ben leinfelder

05/03/2013

03:52 PM Task #5935 (Rejected): Display the PID not the docid in the rendered metadata
This would apply to the FGDC and EML stylesheets. It means adding in an additional parameter (default to docid if pid... ben leinfelder
09:53 AM Task #5932: Allow use of PID (or docid) using cgi-registry modification form
Another option would be to "look-up" the docid from Metacat using the pid and a yet-to-be-defined new Metacat API ser... ben leinfelder
09:44 AM Task #5932 (Closed): Allow use of PID (or docid) using cgi-registry modification form
The SOLR search results XSLT includes a section for editing packages via the Perl registry. But the registry assumes ... ben leinfelder
09:40 AM Task #5931 (Closed): Allow use of PID (or docid) using Metacat API "read" action
The SOLR results use PID as the identifier but the Metacat API for rendering metadata uses docid. We should allow opt... ben leinfelder

05/02/2013

05:30 PM Revision 7664 (metacat): Add code to handle query for the http solr server.
Jing Tao
04:58 PM Revision 7663 (metacat): Add some java doc.
Jing Tao
01:54 PM Task #5916 (Closed): Create resultset-solr.xsl
This is available in the style/common directory for all skins to utilize. Skins can either include the root style she... ben leinfelder
01:52 PM Task #5917 (Closed): Create NCEAS search form that uses SOLR instead of pathquery
It's very basic, but works. And searches should be so much quicker! ben leinfelder
12:47 PM Revision 7662 (metacat): Use a new class to handle the solr query engine description request.
Jing Tao
12:46 PM Revision 7661 (metacat): Copy the index fields description file to the web-inf dir.
Jing Tao
12:44 PM Revision 7660 (metacat): A property file to describe the index fields.
Jing Tao
11:30 AM Revision 7659 (metacat): Fixed a bug the collection name was not set.
Jing Tao
11:21 AM Revision 7658 (metacat): Catch a new exception base on the change of the calling method.
Jing Tao
10:56 AM Revision 7657 (metacat): Add getCollectionName method.
Jing Tao

05/01/2013

10:57 PM Revision 7656 (metacat): Use solr query instead of path query in the NCEAS skin. https://projects.ecoinformatics.org/ecoinfo/issues/5917
ben leinfelder
10:55 PM Revision 7655 (metacat): only build metacat-common before compile, no need to do it every time the ant build is called.
ben leinfelder
05:17 PM Revision 7654 (metacat): enable solr results rendering on the server side for the NCEAS skin (does not include UI for generating the query yet). https://projects.ecoinformatics.org/ecoinfo/issues/5917
ben leinfelder
05:16 PM Revision 7653 (metacat): clean-up the solr results XSLT to display more pertinent fields (many paths have been collapsed into single fields, etc). Added an abstract snippet in the search results (140 chars). https://projects.ecoinformatics.org/ecoinfo/issues/5916
ben leinfelder
04:57 PM Revision 7652 (metacat): Add double quotes to surrend the user or group names in the access fq. This will fix the issue if the names have white spaces.
Jing Tao
03:33 PM Revision 7650 (metacat): rename resultset template to include "solr" so that it can co-exist with the original pathquery-based resultset template (NCEAS skin approach that has one style sheet for multiple document types)
ben leinfelder
03:20 PM Revision 7649 (metacat): use charset=UTF-8 for the meta tag (especially important if other international sites want to use default skin with translations)
ben leinfelder
02:38 PM Revision 7648 (metacat): Add the access query filter.
Jing Tao
01:53 PM Revision 7647 (metacat): Allow use of server-side XSLT for SOLR queries that include "wt=<qformat>". https://projects.ecoinformatics.org/ecoinfo/issues/5812
ben leinfelder
01:53 PM Revision 7646 (metacat): Allow use of server-side XSLT for SOLR queries that include "wt=<qformat>". https://projects.ecoinformatics.org/ecoinfo/issues/5812
ben leinfelder
01:52 PM Revision 7645 (metacat): Allow use of server-side XSLT for SOLR queries that include "wt=<qformat>". https://projects.ecoinformatics.org/ecoinfo/issues/5812
ben leinfelder
12:51 PM Task #5917: Create NCEAS search form that uses SOLR instead of pathquery
This is under way. Decided to use the server-side XSLT processing we already have for pathquery and other XML->HTML t... ben leinfelder
07:37 AM Revision 7644 (metacat): Use 2.0.7 version number in configuration/upgrade/docs (trunk, even though we will not be releasing 2.0.7 from trunk, we want to have the upgrade scripts included here)
ben leinfelder

04/30/2013

09:35 PM Revision 7639 (metacat): Allow null SM.submitter (per schema). There were null values in cn-dev (and probably elsewhere since it is technically allowed in the schema. But with a null value, we need to have a null Subject for the SM.submitter field, not a Subject with a null getValue() return. Encountered this when testing for: https://projects.ecoinformatics.org/ecoinfo/issues/5929.
ben leinfelder
04:47 PM Revision 7638 (metacat): add space to prevent syntax error when additional clause is appended. https://projects.ecoinformatics.org/ecoinfo/issues/5929.
ben leinfelder
03:24 PM Task #5930 (Closed): Add a configuration property to turn on/off the indexing processes (SOLR indexing)
So that Metacat installations can decide which (or both) they want to use.
The proposed the property is:
dbquery....
Jing Tao
03:09 PM Revision 7637 (metacat): move maven.home into build properties (hoping that hudson will be able to correctly override it)
ben leinfelder
03:02 PM Bug #5929: replication update action times out
By replacing the subselect with a LEFT JOIN, we can massively improve performance of this query. Here's the same quer... Matt Jones
02:35 PM Bug #5929 (Closed): replication update action times out
With large database sizes, the replications "update" action times out with normal settings for HTTP timeouts. On Dat... Matt Jones
02:50 PM Revision 7636 (metacat): CHange replication 'update' query to use a LEFT JOIN so that the performance of the replication update action is improved, which had been causing an HTTP timeout for large metacat installations. See https://projects.ecoinformatics.org/ecoinfo/issues/5929.
Matt Jones
02:46 PM Revision 7635 (metacat): Add the code to read the index field information from the schema.xml.
Jing Tao
02:35 PM Task #5928 (Closed): Make sure to start metacat-index web apps when the metacat web apps is start-up during the tomcat startup
Metacat-index will use the distribute system metadata map from the metacat during the startup. So we need to make sur... Jing Tao

04/29/2013

04:59 PM Revision 7634 (metacat): Add code to handle the solr index information. we still need to figure out how to get the information.
Jing Tao
09:16 AM Revision 7633 (metacat): Add the code to build the metacat-common.
Jing Tao
09:15 AM Revision 7632 (metacat): Add the dependency to the metacat-common.
Jing Tao
09:14 AM Revision 7631 (metacat): use the sharing code from the metacat-common.
Jing Tao
09:13 AM Revision 7630 (metacat): Move this class to the metacat-common.
Jing Tao
09:11 AM Revision 7629 (metacat): Remove the bin directory.
Jing Tao
09:10 AM Revision 7628 (metacat): Add a new module for sharing the between the metacat and metacat-index.
Jing Tao

04/26/2013

05:12 PM Revision 7627 (metacat): Use an update method to replace the insert and update methods in the SolrIndex class.
Jing Tao
05:10 PM Revision 7626 (metacat): Add a system metadata for archiving.
Jing Tao

04/25/2013

05:09 PM Revision 7625 (metacat): Add the solr engine to the engine list.
Jing Tao
02:40 PM Task #5927 (Resolved): Implements listing the query engines and fields for the solr query engine
We need to add the solr query engine the mn api:
GET /query/{queryType}
GET /query
Jing Tao
02:18 PM Revision 7624 (metacat): create the lib/maven directory in order to resolve the deps and carry on with the build.
ben leinfelder
02:04 PM Revision 7623 (metacat): require utilities.jar before compile
ben leinfelder
01:13 PM Task #5926 (Resolved): Metacat-index doesn't build solr index during a Metadata inserting document
I inserted an eml document through the metacat devtools.html, but the metacat-index didn't build the solr index. Jing Tao
12:50 PM Bug #5919 (Resolved): Upgrade KNB to 2.0.6
ben leinfelder
11:13 AM Bug #5925 (Resolved): Clean up the jar files in the Metacat deploy directory
In the knb/WEB-INF/lib, i saw some jar files like:
spring-aop-2.5.5.jar spring-context-2.5.5.jar spring-core-2.5....
Jing Tao
11:09 AM Task #5924 (Closed): Add the configuration page for the Solr server in the Metacat configuration utility
We need to a new page for the Solr server in the Metacat configuration utility.
This page will include the server ...
Jing Tao
11:05 AM Task #5923 (Closed): Discover missed documents and queue them for indexing
Metacat-index needs a mechanism to regenerate the solr index for those missed during the insert or update. This mecha... Jing Tao
11:01 AM Task #5922 (Closed): Modify the solr index configuration for the resource map
The index configuration of the resource map in the d1_cn_index_processor has parameter - service url which we don't u... Jing Tao
10:58 AM Task #5921 (Closed): Design and implement Metacat-Index module
We decided we would use as much the code of the d1_cn_index_processor as possible. The d1_cn_index_processor uses S... Jing Tao

04/24/2013

07:34 PM Revision 7622 (metacat): use maven to manage most jar dependencies in Metacat.
Exceptions include: LSID, Datamamager (EML), ben leinfelder
05:03 PM Revision 7621 (metacat): Include solr and lucene jar files into the war. We maybe change it when we use pom.xml to manage the jar dependency.
Jing Tao
05:00 PM Revision 7620 (metacat): Add code to handle solr query.
Jing Tao
04:29 PM Bug #5920 (Resolved): Test PG dump on dev server
I restored the production dump to dev2.nceas.ucsb.edu (1-2 hours to load).
Installed Metacat v2.0.3.
Configured.
U...
ben leinfelder
12:41 PM Bug #5920 (Resolved): Test PG dump on dev server
dev2.nceas.ucsb.edu is our testing area and I will load the production dump there for testing the 2.0.3->2.0.6 upgrade ben leinfelder
04:12 PM Revision 7619 (metacat): Remove the jar file.
Jing Tao
04:10 PM Revision 7618 (metacat): Add the jar file which will be used by solr.
Jing Tao
04:09 PM Revision 7617 (metacat): Add 3.4 jar files.
Jing Tao
04:08 PM Revision 7616 (metacat): Remove the 4.1 jar files.
Jing Tao
04:06 PM Revision 7615 (metacat): Add the 3.4 jar.
Jing Tao
04:05 PM Revision 7614 (metacat): Remove the 4.1 jar and use the 3.4 jar.
Jing Tao
02:23 PM Revision 7613 (metacat): Add code to generate index during the servlet initialization.
Jing Tao
12:40 PM Bug #5919 (Resolved): Upgrade KNB to 2.0.6
Steps for upgrading:
- stop Tomcat...
ben leinfelder
12:12 PM Revision 7612 (metacat): Add the names of the system metadata and the data object imaps.
Jing Tao
11:57 AM Revision 7611 (metacat): Use the Identifier object rather than String as the key to get the system metadata and object path.
Jing Tao
11:30 AM Revision 7610 (metacat): Add code to start hazelcast.
Jing Tao

04/23/2013

05:56 PM Revision 7609 (metacat): Add a class to handle solr query.
Jing Tao
04:26 PM Revision 7608 (metacat): Remove those obsolete index classes.
Jing Tao
04:17 PM Revision 7607 (metacat): Add a junit test for the IndexGenerator class.
Jing Tao
12:00 PM Revision 7606 (metacat): Add a new class which will generate index for the ids.
Jing Tao

04/22/2013

06:08 PM Revision 7604 (metacat): Add code to get all doc ids in the solr server.
Jing Tao
02:46 PM Revision 7603 (metacat): Add a remove(List) method in the class and an obsoletes list parameter in the update method.
Jing Tao
02:23 PM Revision 7602 (metacat): Add the getSystemMetadata and getObsoletes methods.
Jing Tao

04/20/2013

06:19 PM Revision 7600 (metacat): Merging the METACAT_2_0_6_BRANCH changes for [M|C]NodeService into the trunk.
Chris Jones

04/19/2013

05:52 PM Revision 7597 (metacat): change the solr end point to be the localhost.
Jing Tao
05:51 PM Revision 7596 (metacat): Change the default solr endpoint to be the localhost.
Jing Tao
05:47 PM Revision 7595 (metacat): Base the configuration to create either EmbeddedSolrServer or HttpSolrServer.
Jing Tao
05:20 PM Revision 7594 (metacat): Add some new properties for the solr.
Jing Tao
05:19 PM Revision 7593 (metacat): Read the solr configuration file name and collection name from the property file.
Jing Tao
04:37 PM Revision 7592 (metacat): Print out the solr query result in the xml format.
Jing Tao
01:03 PM Revision 7591 (metacat): naive version of SolrServerFactory - no reflection. It returns an EmbeddedSolrServer implementation based on the solr.homeDir set in Settings class. https://projects.ecoinformatics.org/ecoinfo/issues/5883
ben leinfelder
12:43 PM Revision 7590 (metacat): simplify testing with embedded solr server locations -- no need to copy solr-home or set system properties. Still more to do (factory method) but this is getting cleaner.
ben leinfelder
11:38 AM Revision 7589 (metacat): add place-holder for using SolrServerFactory to get the correct SOlrServer impl during application init
ben leinfelder
11:24 AM Revision 7588 (metacat): clean up configuration initialization
ben leinfelder
11:15 AM Revision 7587 (metacat): clean up logging for hz config file error
ben leinfelder
10:25 AM Revision 7586 (metacat): rely on Settings loaded from test resources config rather than hardcoding the paths in code. https://projects.ecoinformatics.org/ecoinfo/issues/5918
ben leinfelder
09:48 AM Revision 7585 (metacat): include test resource bundles to be loaded into Settings configuration during testing.
ben leinfelder
09:48 AM Revision 7584 (metacat): move hazelcast config file location code into the MetacatIndexServlet init method
ben leinfelder
09:08 AM Revision 7583 (metacat): do not skip tests during metacat-index package phase. https://projects.ecoinformatics.org/ecoinfo/issues/5918
ben leinfelder
09:01 AM Revision 7582 (metacat): use integration-test ("mvn verify") to test things that require Metacat to be running. These classes should end be named "*IT.java" and unit tests that do not require metacat to be running can be named "*Test.java" - https://projects.ecoinformatics.org/ecoinfo/issues/5918
ben leinfelder

04/18/2013

05:35 PM Revision 7581 (metacat): format before editing.
ben leinfelder
05:18 PM Task #5918 (Closed): Separate unit tests from integration tests
Since some of the metacat-index tests currently require Metacat be running in order to succeed, we need to skip them ... ben leinfelder
05:14 PM Task #5909 (Resolved): Design configuration for metacat-index
Now sharing metacat.properties using the Settings class from DataONE libclient.
Also sharing the hazelcast.xml fi...
ben leinfelder
05:10 PM Task #5917 (Closed): Create NCEAS search form that uses SOLR instead of pathquery
The UI can stay the same, but the underlying query that is generated (via JS?) would be SOLR syntax and would be sent... ben leinfelder
05:08 PM Task #5916 (Closed): Create resultset-solr.xsl
For rendering SOLR query results from MN.query() on the client side. ben leinfelder
05:07 PM Task #5915 (Closed): Use MN.query(solr) in NCEAS skin
As a first pass, we should refactor the NCEAS skin as it currently exists in the Metacat project and make it call MN.... ben leinfelder
01:14 PM Revision 7580 (metacat): Comment out the testRemove method temporarily.
Jing Tao
01:10 PM Revision 7579 (metacat): Temporarily to comment out the remove test.
Jing Tao
11:28 AM Revision 7578 (metacat): Add a test method to test the update method.
Jing Tao
11:27 AM Revision 7577 (metacat): Add the update method.
Jing Tao
11:26 AM Revision 7576 (metacat): Add documents for updating solr index.
Jing Tao

04/17/2013

05:07 PM Revision 7575 (metacat): Add a solr.homeDir property.
Jing Tao
05:00 PM Revision 7574 (metacat): Get the solr home dir from the metacat.properties.
Jing Tao
04:01 PM Revision 7573 (metacat): Set the test solr home in the target directory.
Jing Tao
03:53 PM Revision 7572 (metacat): Add the necessary files for the solr home.
Jing Tao
01:36 PM Revision 7571 (metacat): Set the hazelcast.xml path for the test.
Jing Tao

04/16/2013

05:14 PM Revision 7569 (metacat): Add a query method in the test.
Jing Tao
04:26 PM Revision 7568 (metacat): split the deployment of metacat-index.war copy into two different tasks so that the "install" target puts it in both the /dist and "deployment directory" whereas the "install-metacat-index" bypasses metacat building and kust rebuilds the metacat-index.war and puts it in the deply directory.
ben leinfelder
04:08 PM Revision 7567 (metacat): copy the metacat-index.war file to the app.deploy.dir rather than dist.dir.
Jing Tao
04:08 PM Revision 7566 (metacat): correctly look up the context init parameter for the metacat properties file
ben leinfelder
03:44 PM Revision 7565 (metacat): Move this file to the solr home and rename it to the schema.xml.
Jing Tao
03:42 PM Revision 7564 (metacat): Add the solr home directory.
Jing Tao
03:34 PM Revision 7563 (metacat): include the objectPathMap name in the metacat.properties (used by metacat-index component) https://projects.ecoinformatics.org/ecoinfo/issues/5909
ben leinfelder
03:34 PM Revision 7562 (metacat): lookup the hz client connection information from the hazelcast.xml file being used by metacat. https://projects.ecoinformatics.org/ecoinfo/issues/5909
ben leinfelder
02:13 PM Task #5908 (Rejected): Investigate EAR deployment of all Metacat webapps
Tomcat is not a full-fledged application server - just a container - so it does not support EARs. I think we should c... ben leinfelder
01:05 PM Task #5907 (Closed): Integrate metacat-index build in Metacat build
Now building into dist folder. ben leinfelder
01:04 PM Revision 7561 (metacat): build metacat-index.war using maven and copy to the metacat /dist folder. https://projects.ecoinformatics.org/ecoinfo/issues/5907
ben leinfelder
11:28 AM Revision 7560 (metacat): start the SystemMetadataListener for each solr index configured (expecting only one).
ben leinfelder
11:22 AM Revision 7559 (metacat): Remove this obsoleted file.
Jing Tao
11:19 AM Revision 7558 (metacat): draft for sharing the metacat.properties with the metacat-index webapp. https://projects.ecoinformatics.org/ecoinfo/issues/5909
ben leinfelder
10:18 AM Revision 7557 (metacat): add methods to set the SolrIndex instance and initialize the listener (includes overloaded constructor)
ben leinfelder
10:13 AM Revision 7556 (metacat): use SystemMetadata parameter for inserting into SolrIndex
ben leinfelder
10:04 AM Revision 7555 (metacat): Change one parameter type of the insert method from the InputStream to the SystemMetadata.
Jing Tao

04/15/2013

05:51 PM Revision 7554 (metacat): Add a file prefix to tell it is absolute path.
Jing Tao
05:50 PM Revision 7553 (metacat): Excludes the slf4j-log4j12.jar for removing duplicated class.
Jing Tao
05:04 PM Revision 7552 (metacat): The module will create a servlet class in a war file.
Jing Tao
05:03 PM Revision 7551 (metacat): Add a web.xml file for the war.
Jing Tao
04:50 PM Revision 7550 (metacat): add first pass at listening for SM events on HZ. Note that the configuration and even the call to solrIndex.insert() are not actually working.
ben leinfelder
 

Also available in: Atom