SEEK: Issueshttps://projects.ecoinformatics.org/ecoinfo/https://projects.ecoinformatics.org/ecoinfo/ecoinfo/favicon.ico?14691340362010-11-17T23:19:14ZEcoinformatics Redmine
Redmine Bug #5246 (Resolved): HTTP response code: 500 when trying to upload large KAR through keplerhttps://projects.ecoinformatics.org/ecoinfo/issues/52462010-11-17T23:19:14ZDerik Barseghianbarseghian@nceas.ucsb.edu
<p>Recently I tried (a few times) to upload a large "run-kar" from the Workflow Run Manager in Kepler to the KeplerDev metacat, but get the following stacktrace from Kepler. The kar tpc03-veld-condition-woody-run1.kar, is ~1M, and the karXml (the metadata) is ~1.9M. The run kar is created by running the following workflow once, and then clicking on the run row in the WRM and selecting upload:<br /><a class="external" href="https://code.ecoinformatics.org/code/kruger/trunk/workflows/tpc03-veld-condition/tpc03-veld-condition-woody.kar">https://code.ecoinformatics.org/code/kruger/trunk/workflows/tpc03-veld-condition/tpc03-veld-condition-woody.kar</a></p>
<p>--------<br />org.kepler.objectmanager.repository.RepositoryException: java.rmi.RemoteException: Server returned HTTP response code: 500 for URL: <a class="external" href="http://kepler-dev.nceas.ucsb.edu:80/kepler/metacat">http://kepler-dev.nceas.ucsb.edu:80/kepler/metacat</a><br /> at org.kepler.objectmanager.repository.EcogridRepository.put(EcogridRepository.java:334)<br /> at org.kepler.kar.UploadToRepository.uploadMetadata(UploadToRepository.java:154)<br /> at org.kepler.gui.kar.ComponentUploader.upload(ComponentUploader.java:119)<br /> at org.kepler.gui.kar.ExportArchiveAction.actionPerformed(ExportArchiveAction.java:478)<br /> at org.kepler.workflowrunmanager.gui.kar.ExportRunsArchiveAction.actionPerformed(ExportRunsArchiveAction.java:109)<br /> at org.kepler.workflowrunmanager.gui.WorkflowRunManagerPanel$2.actionPerformed(WorkflowRunManagerPanel.java:1220)<br />--------</p>
<p>Uploading smaller run-kars works.<br />Jing and I took a look at catalina.out on dev and see the following associated errors to do with 'Post too large':</p>
<p>--------<br />knb 20101117-15:01:57: [ERROR]: Servlet.service() for servlet metacat threw exception [org.apache.catalina.core.ContainerBase.[Catalina].[kepler-dev.nceas.ucsb.edu].[/kepler].[metacat]]<br />java.lang.IllegalStateException: Post too large<br /> at org.apache.catalina.connector.Request.parseParameters(Request.java:2388)<br /> at org.apache.catalina.connector.Request.getParameterNames(Request.java:1047)<br /> at org.apache.catalina.connector.RequestFacade.getParameterNames(RequestFacade.java:369)<br /> at edu.ucsb.nceas.metacat.MetaCatServlet.handleGetOrPost(MetaCatServlet.java:766)<br /> at edu.ucsb.nceas.metacat.MetaCatServlet.doPost(MetaCatServlet.java:534)<br /> at javax.servlet.http.HttpServlet.service(HttpServlet.java:710)<br /> at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)<br /> at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)<br /> at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)<br /> at org.vfny.geoserver.filters.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:122)<br /> at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)<br /> at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)<br /> at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)<br /> at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:174)<br /> at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:433)<br /> at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)<br /> at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)<br /> at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)<br /> at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:174)<br /> at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:200)<br /> at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:283)<br /> at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:773)<br /> at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:703)<br /> at org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:895)<br /> at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689)<br /> at java.lang.Thread.run(Thread.java:619)<br />knb 20101117-15:01:57: [ERROR]: Servlet.service() for servlet metacat threw exception [org.apache.catalina.core.ContainerBase.[Catalina].[kepler-dev.nceas.ucsb.edu].[/kepler].[metacat]]<br />java.lang.IllegalStateException: Post too large<br /> at org.apache.catalina.connector.Request.parseParameters(Request.java:2388)<br /> at org.apache.catalina.connector.Request.getParameterNames(Request.java:1047)<br /> at org.apache.catalina.connector.RequestFacade.getParameterNames(RequestFacade.java:369)<br /> at edu.ucsb.nceas.metacat.MetaCatServlet.handleGetOrPost(MetaCatServlet.java:766)<br /> at edu.ucsb.nceas.metacat.MetaCatServlet.doPost(MetaCatServlet.java:534)<br /> at javax.servlet.http.HttpServlet.service(HttpServlet.java:710)<br /> at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)<br /> at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)<br /> at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)<br /> at org.vfny.geoserver.filters.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:122)<br /> at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)<br /> at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)<br /> at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)<br /> at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:174)<br /> at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:433)<br /> at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)<br /> at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)<br /> at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)<br /> at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:174)<br /> at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:200)<br /> at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:283)<br /> at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:773)<br /> at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:703)<br /> at org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:895)<br /> at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689)<br /> at java.lang.Thread.run(Thread.java:619)<br />knb 20101117-15:01:57: [ERROR]: Servlet.service() for servlet metacat threw exception [org.apache.catalina.core.ContainerBase.[Catalina].[kepler-dev.nceas.ucsb.edu].[/kepler].[metacat]]<br />java.lang.IllegalStateException: Post too large<br /> at org.apache.catalina.connector.Request.parseParameters(Request.java:2388)<br /> at org.apache.catalina.connector.Request.getParameterNames(Request.java:1047)<br /> at org.apache.catalina.connector.RequestFacade.getParameterNames(RequestFacade.java:369)<br /> at edu.ucsb.nceas.metacat.MetaCatServlet.handleGetOrPost(MetaCatServlet.java:766)<br /> at edu.ucsb.nceas.metacat.MetaCatServlet.doPost(MetaCatServlet.java:534)<br /> at javax.servlet.http.HttpServlet.service(HttpServlet.java:710)<br /> at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)<br /> at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)<br /> at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)<br /> at org.vfny.geoserver.filters.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:122)<br /> at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)<br /> at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)<br /> at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)<br /> at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:174)<br /> at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:433)<br /> at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)<br /> at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)<br /> at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)<br /> at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:174)<br /> at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:200)<br /> at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:283)<br /> at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:773)<br /> at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:703)<br /> at org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:895)<br /> at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689)<br /> at java.lang.Thread.run(Thread.java:619)<br />Failed to send data to metacat 3 times: Server returned HTTP response code: 500 for URL: <a class="external" href="http://kepler-dev.nceas.ucsb.edu:80/kepler/metacat">http://kepler-dev.nceas.ucsb.edu:80/kepler/metacat</a><br />metacaturl: <a class="external" href="http://kepler-dev.nceas.ucsb.edu:80/kepler/metacat">http://kepler-dev.nceas.ucsb.edu:80/kepler/metacat</a><br />knb 20101117-15:01:57: [WARN]: UploadMetacatMetadata() exception: Server returned HTTP response code: 500 for URL: <a class="external" href="http://kepler-dev.nceas.ucsb.edu:80/kepler/metacat">http://kepler-dev.nceas.ucsb.edu:80/kepler/metacat</a> [org.ecoinformatics.ecogrid.metacat.put.MetacatPutImpl]</p> Bug #3145 (New): Fix [or remove] web interface for ecogrid queryhttps://projects.ecoinformatics.org/ecoinfo/issues/31452008-02-11T22:05:25Zben leinfelderleinfelder@nceas.ucsb.edu
<p>The Kepler user manual docs (chapter 6) have link to:<br /><a class="external" href="http://seek.ecoinformatics.org/Wiki.jsp?page=EarthGridPortal">http://seek.ecoinformatics.org/Wiki.jsp?page=EarthGridPortal</a><br />which in turn has a link to:<br /><a class="external" href="http://ecogrid.ecoinformatics.org/ecogrid/advancedQuery.jspx">http://ecogrid.ecoinformatics.org/ecogrid/advancedQuery.jspx</a></p>
<p>This search returns no data for KNB. Jing and I suspect it is due to the change in ecogrid query service names. The search should probably be updated to reflect those new endpoints. Also note that it has an option for searching GEON, too. The GEON service has not been operational for sometime, and could also be commented out, at the very least.</p> Bug #2731 (New): Make TCS document the direct data sourcehttps://projects.ecoinformatics.org/ecoinfo/issues/27312007-01-17T01:35:00Zxianhua liurobertliu325@yahoo.com
<p>Develop new functions to directly access TCS document.</p> Bug #2730 (New): Publication problemhttps://projects.ecoinformatics.org/ecoinfo/issues/27302007-01-17T01:30:39Zxianhua liurobertliu325@yahoo.com
<p>For publication let's use be use to implement both short and long full text and guid. The rest is unnecessary for now.</p> Bug #2729 (New): Support third party parent-child relationshiphttps://projects.ecoinformatics.org/ecoinfo/issues/27292007-01-17T01:28:45Zxianhua liurobertliu325@yahoo.com
<p>Third party parent-child, and should we allow parents to be specified for genus in definition.</p> Bug #2727 (New): lost data during importhttps://projects.ecoinformatics.org/ecoinfo/issues/27272007-01-17T01:20:04Zxianhua liurobertliu325@yahoo.com
<p>One problem is that superfluous stuff is lost. For example:<br /><teeth>Y-5 molar pattern</teeth><br />We should capture all this in some form of buffer so it can be <br />exported. For example, all 18 relationship types shoudl come in and go out.</p> Bug #2681 (Resolved): testhttps://projects.ecoinformatics.org/ecoinfo/issues/26812006-11-27T17:02:25ZRobert Peetpeet@unc.edu
<p>test</p> Bug #2174 (Resolved): Problems with multiple eml2datasource actors in a Kepler workflowhttps://projects.ecoinformatics.org/ecoinfo/issues/21742005-09-01T15:51:04ZDan Higginshiggins@nceas.ucsb.edu
<p>I have recently created several Kepler workflow that contain multiple<br />eml200datasources. In particular, see 'workflows/eco/IPCC_Base_Layers.xml' and<br />'workflows/eco/GDAL_h1K_NS_Composite.xml'. The first was created by seaching for<br />'IPCC' and dragging the 10 baseline historical (1961-1990) climate icons to the<br />graph pane and then attaching to ClimateFileProcessor actors to create *.asc<br />files for use in GARP. The second contains 12 Hydro1K datasources (6 for North<br />America and 6 for South America). These data sources are projected, rescaled,<br />and merged into 6 *.asc files with the same resolution as the the IPCC output files.</p>
<p>I have run into several problems with these workflows that are apparently due to<br />the multiple eml200 datasources. These problems include:</p>
<p>1) First of all, the workflows are somewhat awkward to create. One needs to<br />seperately drag each datasource from the search results to the graph frame and<br />then configure the actor. Configuration is somewhat confusing with a very large<br />collection of options. (At a minimum, we need documentation for all the choices<br />somewhere.) Maybe we need some sort of datasource array (or sequence) actor<br />which could represent multiple data sources?</p>
<p>2) When one drags a datasource to the graph pane, the actor turns red while<br />loading data/metadata from the ecogrid and then turns yellow when no longer<br />busy. The delay isn't too bad while building the workflow (and it is in its own<br />thread, so it doesn't stop all other activities). However, when you reopen a<br />workflow with multiple data sources, you have to wait for all the sources to<br />turn from red to yellow. This can take a ridiculously long time on the machine<br />that was not used to create the workflow. [The time may be very, very long the<br />first time because the data has not been cached locally. It took 4-6 HOURS to<br />open the Hydro1K workflow (which has very large files) on a new machine!] And<br />even though each of the sources has its own thread, when there are a number of<br />such actor, they grap so much CPU time that it is not practical to do anything<br />else on a PC.</p>
<p>If I remember correctly, the main reason for the activity when an eml2datasource<br />is dragged to the graph pane is to create the dymanic ports for all the columns<br />in a data table. But the datasources here are not tables and the output ports<br />are determined by the configuration settings. Maybe whatever is going on in the<br />threads when the workflow is opened should be deferred until the workflow is<br />executed? This would let a user open a workflow and examine and edit it without<br />waiting a very long time.</p>
<p>3) There seems to be some errors that occur when trying to open a workflow with<br />multiple data sources. The IPCC workflow will sometimes open OK, but more often<br />all the sources except one will eventually turn from red to yellow. One,<br />however, will remain red and the CPU usage will stay pegged at 100%. The<br />datasource that will not load is not always the same if you shut down java and<br />start again!</p>
<p>Dan Higgins</p> Bug #1745 (Resolved): Develop methods for creating/controlling parallel Kepler workflow calculationshttps://projects.ecoinformatics.org/ecoinfo/issues/17452004-10-22T21:59:00ZDan Higginshiggins@nceas.ucsb.edu
<p>The ENM/GARP BEAM group started considering ways to carry out parallel<br />calculations for different species. A more general method for carrying out long<br />calculatons using multiple computers needs to be developed.</p> Bug #1743 (Resolved): ENM IIIF - ROC Threshold Generatorhttps://projects.ecoinformatics.org/ecoinfo/issues/17432004-10-22T18:58:19ZDan Higginshiggins@nceas.ucsb.edu
<p>Description and Useage uncertain</p>
<p>Postponed</p> Bug #1740 (Resolved): ENM IIIC - Example of ENM workflow iteration over several specieshttps://projects.ecoinformatics.org/ecoinfo/issues/17402004-10-22T18:25:28ZDan Higginshiggins@nceas.ucsb.edu
<p>Eventually, we want to be able to iterate the climate change GARP calculation<br />over many species (e.g. all mammals in western hemisphere). That will probably<br />require a Kepler parallelization capability. In the near term, a demonstration<br />of iterating over a few (~10) species is needed.</p> Bug #1739 (Resolved): ENM IIIB - Revise GARP workflow to show Climate Variation iterationshttps://projects.ecoinformatics.org/ecoinfo/issues/17392004-10-22T18:19:12ZDan Higginshiggins@nceas.ucsb.edu
<p>For each species, we want to calculate the GARP species occurence prediction for<br />the current climate and then apply the best ruleset to alternate climate<br />predictions. An explicit workflow showing how the calculations are iterated over<br />climate change predictions needs to be created.</p> Bug #1738 (Resolved): ENM IIIA - Update GARP Workflowhttps://projects.ecoinformatics.org/ecoinfo/issues/17382004-10-22T18:14:36ZDan Higginshiggins@nceas.ucsb.edu
<p>It was decided to make much of the GARP post-processing analysis optional at<br />this point in time (e.g. the statistical analysis of commision/omission, best<br />rule set, ROC analysis) The existing fullGARP workflow should be updated to show<br />this</p>
<p>Responsible Party: Higgins</p> Bug #1737 (Resolved): ENM III - ENM/GARP Calculationshttps://projects.ecoinformatics.org/ecoinfo/issues/17372004-10-22T18:04:34ZDan Higginshiggins@nceas.ucsb.edu
<p>This bug/task represents the section of the workflow after the ocurrence data<br />and environment layers have been prepared in the desired format. It includes<br />post-processing of GARP output, climate variations for a single species, and<br />iterations over different species.</p> Bug #1720 (Resolved): Create an example Kepler workflow for ENM/GARP calculationshttps://projects.ecoinformatics.org/ecoinfo/issues/17202004-10-21T22:12:49ZDan Higginshiggins@nceas.ucsb.edu
<p>The SEEK BEAM group needs to create a Environmental Niche Model (ENM) workflow<br />for predicting species occurence distributions using the GARP model (Eventually<br />also other algorithms). This is the top-level task. A number of other distinct<br />tasks, corresponding to Kepler actors, will also be defined.</p>
<p>The tasks linked to this bug are a result of interactions in a phone conference<br />held on 20 Oct 2004. See the attached memo.</p>
<p>Tasks will be characterized in three major areas, labeled I), II), and III):</p>
<p>I) - Creation of known species occurence data tables (GARP Input)</p>
<p>II) - Collection/preparation of data layers to be used to predict species<br />distribution (climate, geography, etc) (GARP Input)</p>
<p>III) - Execution of GARP calculations, using results of I) and II) as inputs,<br />creating ruleSets; Use of rulesets to predict occurence distributions for<br />alternative climate scenarios</p>