Activity
From 04/14/2013 to 05/13/2013
05/13/2013
- 09:23 PM Revision 7691 (metacat): Add code to wait the readiness of the hazelcast service.
- 04:34 PM Revision 7690 (metacat): Use the DistributedMapsFactory to get maps.
- 03:51 PM Revision 7689 (metacat): Add a class to get distributed maps such as the system metadata map.
- 11:58 AM Revision 7688 (metacat): Removed the comment on the process of building index.
- 11:55 AM Revision 7687 (metacat): Add code to wait the startup of the MNode.
05/10/2013
- 04:09 PM Task #5905: Implement access control filter
- I also tested that if there is a fq in the user specified query, it wouldn't cause any issue:
The user specified q... - 04:04 PM Task #5905: Implement access control filter
- Append filter query (fq) to the main query. The fq looks like:
fq=(readPermission:"public")OR(readPermission:"CN=ben... - 03:02 PM Revision 7686 (metacat): Removed the commented out text cases and add a test for archvied document.
- 02:59 PM Revision 7685 (metacat): Only after reloading the core, the query result can reflect the change made in metacat-index module.
- 02:37 PM Revision 7684 (metacat): Use the log info to replace the print.
- 02:36 PM Revision 7683 (metacat): Add log information for archiving or inserting indexes.
05/09/2013
- 01:26 PM Bug #3017 (Closed): The Ampersand returns a blank page on searches
- This is in trunk now, though it is not applicable for solr queries. (I'm sure there are other character escaping issu...
- 12:11 PM Revision 7682 (metacat): Consider mutithread issue there.
- 12:10 PM Revision 7681 (metacat): Fixed a bug to put "OR" correctly in the query.
- And remove the user "authorized_user" from the rightsholder clause in the query.
- 10:04 AM Revision 7680 (metacat): Use the set of subjects to replace the user and groups for the solr query.
- 12:19 AM Revision 7679 (metacat): Call setting certificate location to be the test one after getting the MN baseurl.
- The method to getting MN baseurl somehow calls CN and it set the certificate location to be /var/metacat/certs/METACA...
05/08/2013
- 10:14 PM Revision 7678 (metacat): escape reserved XML characters when constructing a pathquery from user input (&). https://projects.ecoinformatics.org/ecoinfo/issues/3017
- 05:21 PM Revision 7677 (metacat): escape special XML characters when constructing a pathquery from user input (&). https://projects.ecoinformatics.org/ecoinfo/issues/3017
- 04:53 PM Revision 7676 (metacat): Add a distrusted certificate for the testing.
- 04:53 PM Revision 7675 (metacat): Use the MNode to query the server when we use certificates to set up the session.
- 03:35 PM Bug #3085 (Resolved): document the impact of re-deploying on metacat.properties
- Added clarification and a note to documentation about what is backed up and when a full metacat.properties file shoul...
- 03:34 PM Revision 7674 (metacat): Add the code to test the user with a distrusted certificate.
- 03:34 PM Revision 7673 (metacat): add note about metacat.properties backup. https://projects.ecoinformatics.org/ecoinfo/issues/3085
- 03:14 PM Task #5931 (Closed): Allow use of PID (or docid) using Metacat API "read" action
- This was committed along with a different change. But might as well stick with it as I think it will be important as ...
- 03:10 PM Revision 7672 (metacat): remove xslt sort for solr search results in case someone actually specifies a sort field in the query. https://projects.ecoinformatics.org/ecoinfo/issues/3397
- 03:05 PM Bug #3397 (Resolved): metacat needs server side sort mechanism
- SOLR queries can specify a sort field 9or even a function!) - we will not implement this in pathquery.
- 02:58 PM Bug #3816 (Closed): Export action doesn't create complete package
- this has been corrected in trunk and will be included in the next release built from trunk.
- 02:58 PM Revision 7671 (metacat): adjust action=zip behavior to use full docids and entity names (data files) for the zip entry. Also uses the given qformat to render the metadata. https://projects.ecoinformatics.org/ecoinfo/issues/3816
- 10:42 AM Revision 7670 (metacat): Add tests to test group and rightsholder.
- 10:41 AM Revision 7669 (metacat): Add the rightsHolder in the access filter.
05/07/2013
- 08:34 PM Revision 7668 (metacat): Change the delete to archive.
- 06:52 PM Revision 7667 (metacat): Add a test for testing access control for the solr query.
- 05:29 PM Revision 7666 (metacat): force the use of newer xercesImpl 2.7.1
- 04:54 PM Revision 7665 (metacat): adjust action=zip behavior to use full docids when checking for permissions/existence. https://projects.ecoinformatics.org/ecoinfo/issues/3816
05/06/2013
- 11:44 AM Bug #5929 (Closed): replication update action times out
- This is included in the 2.0.7 release targeted for CN environment in DataONE.
- 11:43 AM Bug #5925 (Resolved): Clean up the jar files in the Metacat deploy directory
- Now using Maven to handle dependencies, especially as relates to geotools for generating the spatial cache and all th...
05/03/2013
- 03:52 PM Task #5935 (Rejected): Display the PID not the docid in the rendered metadata
- This would apply to the FGDC and EML stylesheets. It means adding in an additional parameter (default to docid if pid...
- 09:53 AM Task #5932: Allow use of PID (or docid) using cgi-registry modification form
- Another option would be to "look-up" the docid from Metacat using the pid and a yet-to-be-defined new Metacat API ser...
- 09:44 AM Task #5932 (Closed): Allow use of PID (or docid) using cgi-registry modification form
- The SOLR search results XSLT includes a section for editing packages via the Perl registry. But the registry assumes ...
- 09:40 AM Task #5931 (Closed): Allow use of PID (or docid) using Metacat API "read" action
- The SOLR results use PID as the identifier but the Metacat API for rendering metadata uses docid. We should allow opt...
05/02/2013
- 05:30 PM Revision 7664 (metacat): Add code to handle query for the http solr server.
- 04:58 PM Revision 7663 (metacat): Add some java doc.
- 01:54 PM Task #5916 (Closed): Create resultset-solr.xsl
- This is available in the style/common directory for all skins to utilize. Skins can either include the root style she...
- 01:52 PM Task #5917 (Closed): Create NCEAS search form that uses SOLR instead of pathquery
- It's very basic, but works. And searches should be so much quicker!
- 12:47 PM Revision 7662 (metacat): Use a new class to handle the solr query engine description request.
- 12:46 PM Revision 7661 (metacat): Copy the index fields description file to the web-inf dir.
- 12:44 PM Revision 7660 (metacat): A property file to describe the index fields.
- 11:30 AM Revision 7659 (metacat): Fixed a bug the collection name was not set.
- 11:21 AM Revision 7658 (metacat): Catch a new exception base on the change of the calling method.
- 10:56 AM Revision 7657 (metacat): Add getCollectionName method.
05/01/2013
- 10:57 PM Revision 7656 (metacat): Use solr query instead of path query in the NCEAS skin. https://projects.ecoinformatics.org/ecoinfo/issues/5917
- 10:55 PM Revision 7655 (metacat): only build metacat-common before compile, no need to do it every time the ant build is called.
- 05:17 PM Revision 7654 (metacat): enable solr results rendering on the server side for the NCEAS skin (does not include UI for generating the query yet). https://projects.ecoinformatics.org/ecoinfo/issues/5917
- 05:16 PM Revision 7653 (metacat): clean-up the solr results XSLT to display more pertinent fields (many paths have been collapsed into single fields, etc). Added an abstract snippet in the search results (140 chars). https://projects.ecoinformatics.org/ecoinfo/issues/5916
- 04:57 PM Revision 7652 (metacat): Add double quotes to surrend the user or group names in the access fq. This will fix the issue if the names have white spaces.
- 03:33 PM Revision 7650 (metacat): rename resultset template to include "solr" so that it can co-exist with the original pathquery-based resultset template (NCEAS skin approach that has one style sheet for multiple document types)
- 03:20 PM Revision 7649 (metacat): use charset=UTF-8 for the meta tag (especially important if other international sites want to use default skin with translations)
- 02:38 PM Revision 7648 (metacat): Add the access query filter.
- 01:53 PM Revision 7647 (metacat): Allow use of server-side XSLT for SOLR queries that include "wt=<qformat>". https://projects.ecoinformatics.org/ecoinfo/issues/5812
- 01:53 PM Revision 7646 (metacat): Allow use of server-side XSLT for SOLR queries that include "wt=<qformat>". https://projects.ecoinformatics.org/ecoinfo/issues/5812
- 01:52 PM Revision 7645 (metacat): Allow use of server-side XSLT for SOLR queries that include "wt=<qformat>". https://projects.ecoinformatics.org/ecoinfo/issues/5812
- 12:51 PM Task #5917: Create NCEAS search form that uses SOLR instead of pathquery
- This is under way. Decided to use the server-side XSLT processing we already have for pathquery and other XML->HTML t...
- 07:37 AM Revision 7644 (metacat): Use 2.0.7 version number in configuration/upgrade/docs (trunk, even though we will not be releasing 2.0.7 from trunk, we want to have the upgrade scripts included here)
04/30/2013
- 09:35 PM Revision 7639 (metacat): Allow null SM.submitter (per schema). There were null values in cn-dev (and probably elsewhere since it is technically allowed in the schema. But with a null value, we need to have a null Subject for the SM.submitter field, not a Subject with a null getValue() return. Encountered this when testing for: https://projects.ecoinformatics.org/ecoinfo/issues/5929.
- 04:47 PM Revision 7638 (metacat): add space to prevent syntax error when additional clause is appended. https://projects.ecoinformatics.org/ecoinfo/issues/5929.
- 03:24 PM Task #5930 (Closed): Add a configuration property to turn on/off the indexing processes (SOLR indexing)
- So that Metacat installations can decide which (or both) they want to use.
The proposed the property is:
dbquery.... - 03:09 PM Revision 7637 (metacat): move maven.home into build properties (hoping that hudson will be able to correctly override it)
- 03:02 PM Bug #5929: replication update action times out
- By replacing the subselect with a LEFT JOIN, we can massively improve performance of this query. Here's the same quer...
- 02:35 PM Bug #5929 (Closed): replication update action times out
- With large database sizes, the replications "update" action times out with normal settings for HTTP timeouts. On Dat...
- 02:50 PM Revision 7636 (metacat): CHange replication 'update' query to use a LEFT JOIN so that the performance of the replication update action is improved, which had been causing an HTTP timeout for large metacat installations. See https://projects.ecoinformatics.org/ecoinfo/issues/5929.
- 02:46 PM Revision 7635 (metacat): Add the code to read the index field information from the schema.xml.
- 02:35 PM Task #5928 (Closed): Make sure to start metacat-index web apps when the metacat web apps is start-up during the tomcat startup
- Metacat-index will use the distribute system metadata map from the metacat during the startup. So we need to make sur...
04/29/2013
- 04:59 PM Revision 7634 (metacat): Add code to handle the solr index information. we still need to figure out how to get the information.
- 09:16 AM Revision 7633 (metacat): Add the code to build the metacat-common.
- 09:15 AM Revision 7632 (metacat): Add the dependency to the metacat-common.
- 09:14 AM Revision 7631 (metacat): use the sharing code from the metacat-common.
- 09:13 AM Revision 7630 (metacat): Move this class to the metacat-common.
- 09:11 AM Revision 7629 (metacat): Remove the bin directory.
- 09:10 AM Revision 7628 (metacat): Add a new module for sharing the between the metacat and metacat-index.
04/26/2013
- 05:12 PM Revision 7627 (metacat): Use an update method to replace the insert and update methods in the SolrIndex class.
- 05:10 PM Revision 7626 (metacat): Add a system metadata for archiving.
04/25/2013
- 05:09 PM Revision 7625 (metacat): Add the solr engine to the engine list.
- 02:40 PM Task #5927 (Resolved): Implements listing the query engines and fields for the solr query engine
- We need to add the solr query engine the mn api:
GET /query/{queryType}
GET /query
- 02:18 PM Revision 7624 (metacat): create the lib/maven directory in order to resolve the deps and carry on with the build.
- 02:04 PM Revision 7623 (metacat): require utilities.jar before compile
- 01:13 PM Task #5926 (Resolved): Metacat-index doesn't build solr index during a Metadata inserting document
- I inserted an eml document through the metacat devtools.html, but the metacat-index didn't build the solr index.
- 12:50 PM Bug #5919 (Resolved): Upgrade KNB to 2.0.6
- 11:13 AM Bug #5925 (Resolved): Clean up the jar files in the Metacat deploy directory
- In the knb/WEB-INF/lib, i saw some jar files like:
spring-aop-2.5.5.jar spring-context-2.5.5.jar spring-core-2.5.... - 11:09 AM Task #5924 (Closed): Add the configuration page for the Solr server in the Metacat configuration utility
- We need to a new page for the Solr server in the Metacat configuration utility.
This page will include the server ... - 11:05 AM Task #5923 (Closed): Discover missed documents and queue them for indexing
- Metacat-index needs a mechanism to regenerate the solr index for those missed during the insert or update. This mecha...
- 11:01 AM Task #5922 (Closed): Modify the solr index configuration for the resource map
- The index configuration of the resource map in the d1_cn_index_processor has parameter - service url which we don't u...
- 10:58 AM Task #5921 (Closed): Design and implement Metacat-Index module
- We decided we would use as much the code of the d1_cn_index_processor as possible. The d1_cn_index_processor uses S...
04/24/2013
- 07:34 PM Revision 7622 (metacat): use maven to manage most jar dependencies in Metacat.
- Exceptions include: LSID, Datamamager (EML),
- 05:03 PM Revision 7621 (metacat): Include solr and lucene jar files into the war. We maybe change it when we use pom.xml to manage the jar dependency.
- 05:00 PM Revision 7620 (metacat): Add code to handle solr query.
- 04:29 PM Bug #5920 (Resolved): Test PG dump on dev server
- I restored the production dump to dev2.nceas.ucsb.edu (1-2 hours to load).
Installed Metacat v2.0.3.
Configured.
U... - 12:41 PM Bug #5920 (Resolved): Test PG dump on dev server
- dev2.nceas.ucsb.edu is our testing area and I will load the production dump there for testing the 2.0.3->2.0.6 upgrade
- 04:12 PM Revision 7619 (metacat): Remove the jar file.
- 04:10 PM Revision 7618 (metacat): Add the jar file which will be used by solr.
- 04:09 PM Revision 7617 (metacat): Add 3.4 jar files.
- 04:08 PM Revision 7616 (metacat): Remove the 4.1 jar files.
- 04:06 PM Revision 7615 (metacat): Add the 3.4 jar.
- 04:05 PM Revision 7614 (metacat): Remove the 4.1 jar and use the 3.4 jar.
- 02:23 PM Revision 7613 (metacat): Add code to generate index during the servlet initialization.
- 12:40 PM Bug #5919 (Resolved): Upgrade KNB to 2.0.6
- Steps for upgrading:
- stop Tomcat... - 12:12 PM Revision 7612 (metacat): Add the names of the system metadata and the data object imaps.
- 11:57 AM Revision 7611 (metacat): Use the Identifier object rather than String as the key to get the system metadata and object path.
- 11:30 AM Revision 7610 (metacat): Add code to start hazelcast.
04/23/2013
- 05:56 PM Revision 7609 (metacat): Add a class to handle solr query.
- 04:26 PM Revision 7608 (metacat): Remove those obsolete index classes.
- 04:17 PM Revision 7607 (metacat): Add a junit test for the IndexGenerator class.
- 12:00 PM Revision 7606 (metacat): Add a new class which will generate index for the ids.
04/22/2013
- 06:08 PM Revision 7604 (metacat): Add code to get all doc ids in the solr server.
- 02:46 PM Revision 7603 (metacat): Add a remove(List) method in the class and an obsoletes list parameter in the update method.
- 02:23 PM Revision 7602 (metacat): Add the getSystemMetadata and getObsoletes methods.
04/20/2013
04/19/2013
- 05:52 PM Revision 7597 (metacat): change the solr end point to be the localhost.
- 05:51 PM Revision 7596 (metacat): Change the default solr endpoint to be the localhost.
- 05:47 PM Revision 7595 (metacat): Base the configuration to create either EmbeddedSolrServer or HttpSolrServer.
- 05:20 PM Revision 7594 (metacat): Add some new properties for the solr.
- 05:19 PM Revision 7593 (metacat): Read the solr configuration file name and collection name from the property file.
- 04:37 PM Revision 7592 (metacat): Print out the solr query result in the xml format.
- 01:03 PM Revision 7591 (metacat): naive version of SolrServerFactory - no reflection. It returns an EmbeddedSolrServer implementation based on the solr.homeDir set in Settings class. https://projects.ecoinformatics.org/ecoinfo/issues/5883
- 12:43 PM Revision 7590 (metacat): simplify testing with embedded solr server locations -- no need to copy solr-home or set system properties. Still more to do (factory method) but this is getting cleaner.
- 11:38 AM Revision 7589 (metacat): add place-holder for using SolrServerFactory to get the correct SOlrServer impl during application init
- 11:24 AM Revision 7588 (metacat): clean up configuration initialization
- 11:15 AM Revision 7587 (metacat): clean up logging for hz config file error
- 10:25 AM Revision 7586 (metacat): rely on Settings loaded from test resources config rather than hardcoding the paths in code. https://projects.ecoinformatics.org/ecoinfo/issues/5918
- 09:48 AM Revision 7585 (metacat): include test resource bundles to be loaded into Settings configuration during testing.
- 09:48 AM Revision 7584 (metacat): move hazelcast config file location code into the MetacatIndexServlet init method
- 09:08 AM Revision 7583 (metacat): do not skip tests during metacat-index package phase. https://projects.ecoinformatics.org/ecoinfo/issues/5918
- 09:01 AM Revision 7582 (metacat): use integration-test ("mvn verify") to test things that require Metacat to be running. These classes should end be named "*IT.java" and unit tests that do not require metacat to be running can be named "*Test.java" - https://projects.ecoinformatics.org/ecoinfo/issues/5918
04/18/2013
- 05:35 PM Revision 7581 (metacat): format before editing.
- 05:18 PM Task #5918 (Closed): Separate unit tests from integration tests
- Since some of the metacat-index tests currently require Metacat be running in order to succeed, we need to skip them ...
- 05:14 PM Task #5909 (Resolved): Design configuration for metacat-index
- Now sharing metacat.properties using the Settings class from DataONE libclient.
Also sharing the hazelcast.xml fi... - 05:10 PM Task #5917 (Closed): Create NCEAS search form that uses SOLR instead of pathquery
- The UI can stay the same, but the underlying query that is generated (via JS?) would be SOLR syntax and would be sent...
- 05:08 PM Task #5916 (Closed): Create resultset-solr.xsl
- For rendering SOLR query results from MN.query() on the client side.
- 05:07 PM Task #5915 (Closed): Use MN.query(solr) in NCEAS skin
- As a first pass, we should refactor the NCEAS skin as it currently exists in the Metacat project and make it call MN....
- 01:14 PM Revision 7580 (metacat): Comment out the testRemove method temporarily.
- 01:10 PM Revision 7579 (metacat): Temporarily to comment out the remove test.
- 11:28 AM Revision 7578 (metacat): Add a test method to test the update method.
- 11:27 AM Revision 7577 (metacat): Add the update method.
- 11:26 AM Revision 7576 (metacat): Add documents for updating solr index.
04/17/2013
- 05:07 PM Revision 7575 (metacat): Add a solr.homeDir property.
- 05:00 PM Revision 7574 (metacat): Get the solr home dir from the metacat.properties.
- 04:01 PM Revision 7573 (metacat): Set the test solr home in the target directory.
- 03:53 PM Revision 7572 (metacat): Add the necessary files for the solr home.
- 01:36 PM Revision 7571 (metacat): Set the hazelcast.xml path for the test.
04/16/2013
- 05:14 PM Revision 7569 (metacat): Add a query method in the test.
- 04:26 PM Revision 7568 (metacat): split the deployment of metacat-index.war copy into two different tasks so that the "install" target puts it in both the /dist and "deployment directory" whereas the "install-metacat-index" bypasses metacat building and kust rebuilds the metacat-index.war and puts it in the deply directory.
- 04:08 PM Revision 7567 (metacat): copy the metacat-index.war file to the app.deploy.dir rather than dist.dir.
- 04:08 PM Revision 7566 (metacat): correctly look up the context init parameter for the metacat properties file
- 03:44 PM Revision 7565 (metacat): Move this file to the solr home and rename it to the schema.xml.
- 03:42 PM Revision 7564 (metacat): Add the solr home directory.
- 03:34 PM Revision 7563 (metacat): include the objectPathMap name in the metacat.properties (used by metacat-index component) https://projects.ecoinformatics.org/ecoinfo/issues/5909
- 03:34 PM Revision 7562 (metacat): lookup the hz client connection information from the hazelcast.xml file being used by metacat. https://projects.ecoinformatics.org/ecoinfo/issues/5909
- 02:13 PM Task #5908 (Rejected): Investigate EAR deployment of all Metacat webapps
- Tomcat is not a full-fledged application server - just a container - so it does not support EARs. I think we should c...
- 01:05 PM Task #5907 (Closed): Integrate metacat-index build in Metacat build
- Now building into dist folder.
- 01:04 PM Revision 7561 (metacat): build metacat-index.war using maven and copy to the metacat /dist folder. https://projects.ecoinformatics.org/ecoinfo/issues/5907
- 11:28 AM Revision 7560 (metacat): start the SystemMetadataListener for each solr index configured (expecting only one).
- 11:22 AM Revision 7559 (metacat): Remove this obsoleted file.
- 11:19 AM Revision 7558 (metacat): draft for sharing the metacat.properties with the metacat-index webapp. https://projects.ecoinformatics.org/ecoinfo/issues/5909
- 10:18 AM Revision 7557 (metacat): add methods to set the SolrIndex instance and initialize the listener (includes overloaded constructor)
- 10:13 AM Revision 7556 (metacat): use SystemMetadata parameter for inserting into SolrIndex
- 10:04 AM Revision 7555 (metacat): Change one parameter type of the insert method from the InputStream to the SystemMetadata.
04/15/2013
- 05:51 PM Revision 7554 (metacat): Add a file prefix to tell it is absolute path.
- 05:50 PM Revision 7553 (metacat): Excludes the slf4j-log4j12.jar for removing duplicated class.
- 05:04 PM Revision 7552 (metacat): The module will create a servlet class in a war file.
- 05:03 PM Revision 7551 (metacat): Add a web.xml file for the war.
- 04:50 PM Revision 7550 (metacat): add first pass at listening for SM events on HZ. Note that the configuration and even the call to solrIndex.insert() are not actually working.
Also available in: Atom