Kepler: Issueshttps://projects.ecoinformatics.org/ecoinfo/https://projects.ecoinformatics.org/ecoinfo/ecoinfo/favicon.ico?14691340362012-08-17T23:39:07ZEcoinformatics Redmine
Redmine Bug #5686 (New): datasets produced by archiveDataturbineDataToMetacat workflow show samples 'slid...https://projects.ecoinformatics.org/ecoinfo/issues/56862012-08-17T23:39:07ZDerik Barseghianbarseghian@nceas.ucsb.edu
<p>Data sample times slowly increase, e.g. second by second, over time, instead of staying on expected markers. This makes for irregular, messy datasets. I would guess the problem lies with SPAN.</p>
<p>A dataset available here shows examples of this:<br /><a class="external" href="http://dev2.nceas.ucsb.edu/knb/metacat?action=read&qformat=default&sessionid=&docid=doc.1345243176506661893.1&displaymodule=entity&entitytype=dataTable&entityindex=1">http://dev2.nceas.ucsb.edu/knb/metacat?action=read&qformat=default&sessionid=&docid=doc.1345243176506661893.1&displaymodule=entity&entitytype=dataTable&entityindex=1</a></p>
<p>Some example problematic changes found within:</p>
<p>timestamps data<br />2012-08-16 00:10:09 13.302116394<br />2012-08-16 00:10:39 13.302242279<br />[SNIP]<br />2012-08-16 02:46:09 13.295635223<br />2012-08-16 02:46:39 13.295513153<br />2012-08-16 02:47:10 13.295863152 # <== slip<br />2012-08-16 02:47:40 13.296142578<br />[SNIP]<br />2012-08-16 08:09:10 13.290791512<br />2012-08-16 08:09:40 13.290988922<br />2012-08-16 08:10:11 13.290698051 # <== slip<br />2012-08-16 08:10:41 13.290913582<br />...</p> Bug #5685 (New): data isn't always chunked properlyhttps://projects.ecoinformatics.org/ecoinfo/issues/56852012-08-17T23:29:49ZDerik Barseghianbarseghian@nceas.ucsb.edu
<p>Jing ran the workflow in Windows XP today, and it produced at least one datapackage with data from different sampling rates. See:<br /><a class="external" href="http://dev2.nceas.ucsb.edu/knb/metacat?action=read&qformat=default&sessionid=&docid=doc.1345243176506661893.1&displaymodule=entity&entitytype=dataTable&entityindex=1">http://dev2.nceas.ucsb.edu/knb/metacat?action=read&qformat=default&sessionid=&docid=doc.1345243176506661893.1&displaymodule=entity&entitytype=dataTable&entityindex=1</a></p>
<p>Example change section from data from above link:</p>
<p>2012-08-16 22:18:14 13.28463459<br />2012-08-16 22:18:44 13.284519196<br />2012-08-16 22:19:14 13.284427643<br />2012-08-16 22:19:44 13.284352303<br />2012-08-16 22:20:19 13.284294128<br />2012-08-16 22:38:14 13.28584671<br />2012-08-16 22:38:15 13.28584671<br />2012-08-16 22:38:16 13.291329384</p>
<p>These were the results Jing got from the workflow for this particular sensor:</p>
<p>Sensor Name: gpp-data/CR800_Batt_Volt<br />Document URL: <a class="external" href="http://dev2.nceas.ucsb.edu/knb/metacat?action=read&qformat=default&docid=doc.1345243176506661893.1">http://dev2.nceas.ucsb.edu/knb/metacat?action=read&qformat=default&docid=doc.1345243176506661893.1</a><br />Time Range: 2012-08-16 00:10:09 ~ 2012-08-16 22:38:32<br />Number of Records: 2680<br />Sensor Name: gpp-data/CR800_Batt_Volt<br />Document URL: <a class="external" href="http://dev2.nceas.ucsb.edu/knb/metacat?action=read&qformat=default&docid=doc.1345243308947314582.1">http://dev2.nceas.ucsb.edu/knb/metacat?action=read&qformat=default&docid=doc.1345243308947314582.1</a><br />Time Range: 2012-08-16 22:38:34 ~ 2012-08-16 22:39:54<br />Number of Records: 75<br />Sensor Name: gpp-data/CR800_Batt_Volt<br />Document URL: <a class="external" href="http://dev2.nceas.ucsb.edu/knb/metacat?action=read&qformat=default&docid=doc.1345243447776683166.1">http://dev2.nceas.ucsb.edu/knb/metacat?action=read&qformat=default&docid=doc.1345243447776683166.1</a><br />Time Range: 2012-08-16 22:39:55 ~ 2012-08-17 22:37:21<br />Number of Records: 8186</p> Bug #5684 (Resolved): Add warning to archiving workflow that it must be run on one machine to not...https://projects.ecoinformatics.org/ecoinfo/issues/56842012-08-17T23:23:20ZDerik Barseghianbarseghian@nceas.ucsb.edu
<p>See <a class="external" href="http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5401">http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5401</a></p> Bug #5683 (Resolved): archiveDataturbineDataToMetacat.kar can fail with an IOException on win7 an...https://projects.ecoinformatics.org/ecoinfo/issues/56832012-08-17T21:11:12ZDerik Barseghianbarseghian@nceas.ucsb.edu
<p>When trying to run the archival workflow on win7, I got this error. Jing got it today in XP.</p>
<p>ptolemy.kernel.util.IllegalActionException: Error: java.io.IOException: The filename, directory name, or volume label syntax is incorrect<br />in .archiveDataturbineDataToMetacat.GenerateEMLAndDataFile.CreateDataSets<br /> at org.kepler.actors.CreateDataSets.createFile(CreateDataSets.java:325)<br /> at org.kepler.actors.CreateDataSets.fire(CreateDataSets.java:234)<br /> at ptolemy.actor.process.ProcessThread.run(ProcessThread.java:229)</p> Bug #5582 (New): DataTurbine server crashing the JREhttps://projects.ecoinformatics.org/ecoinfo/issues/55822012-03-29T03:19:05ZDerik Barseghianbarseghian@nceas.ucsb.edu
<p>I don't think this is a Kepler bug but I want to record my notes somewhere:</p>
<p>I was able to get the DataTurbine server to crash the JRE around 10 times today. Sometimes the JRE would leave a log file, I submitted one to Sun. I was using DT 3.2b5 but moved to 3.2b6, but this didn't seem to help. Likewise I moved the client (Kepler DT actor) from 3.2b5 to 3.2b6, to no avail. I was able to crash the server using both Ubuntu w/ java 1.6.0_26-b03 and osX 10.6 w/ java 1.6.0_29-b11-402-10M3527. I'm in the process of doing some cleanup on the DT actor, but I verified these crashes would occur with none of the new changes. Generally the procedure was I'd run the growingDegreeDays workflow requesting 1 day of data. After about 6-10 iterations of this workflow in succession the server would crash.</p>
<p>This hasn't been well vetted but what seems to have "fixed" it is starting the server with a larger archiveSize -- I bumped from 500000 to 1000000. Recently I loaded a lot more data to the reap02 channels, so perhaps that's why this has only recently cropped up. I've now run the GDD workflow requesting <strong>100</strong> days of data 10 times, and no crash w/ the 3.2b6 DT server on my mac. I'm going to run a long stress test over night and see what happens.</p> Bug #5377 (New): DataTurbine actor fetch time outs (blockTimeout param not used?)https://projects.ecoinformatics.org/ecoinfo/issues/53772011-04-11T19:45:15ZDerik Barseghianbarseghian@nceas.ucsb.edu
<p>Sometimes when attempting to connect to my gumstix DT server w/ the DT actor, I get a fetch timeout message, and no channels (i.e. no data output ports):</p>
<p>MC 12:33:02,094: [WARN]: WARNING: fetch timed out. Try increasing the blockTimeOut value [org.kepler.data.datasource.datasource.dataturbine.DataTurbine]<br />MC 12:33:02,095: [ERROR]: DataTurbine actor Error: only 0 channels were returned [org.kepler.data.datasource.datasource.dataturbine.DataTurbine]</p>
<p>The server does have channels and RDV 2.2.1 is sometimes able to show them in just 2-3 seconds. Increasing the blockTimeout actor parameter from 15s to higher doesn't seem to change anything - it still takes 15s before it times out. Either the actor is not using the parameter, or 15s is a max somewhere.</p> Bug #5364 (New): DataTurbine server memory errorhttps://projects.ecoinformatics.org/ecoinfo/issues/53642011-04-04T18:11:30ZDerik Barseghianbarseghian@nceas.ucsb.edu
<p>Sometimes my gumstix DT server gives a memory error, and then stops working -- the process keeps running, but I cannot connect to it. The most recent time this happened i had been running span and spanToDT for a little under a day. I think for most of this period i sampled 1 sensor every 5s, and 2 every 30s.</p>
<p>Startup:<br />----------<br /><01-Apr-2011 Pacific Daylight Time 11:56:36.927> <dataturbine><br /> RBNB (Ring Buffered Network Bus) V3.2B5 build 2953 (built Tue Aug 31 05:47:41 Pacific Daylight Time 2010)<br /> Copyright 2006 Creare Inc.<br /> Started at address tcp://gpp.msi.ucsb.edu:3333.<br /> Archive home directory: "/var/rbnb_archive" <br /><01-Apr-2011 Pacific Daylight Time 12:50:54.179> <KeplerClient><br /> Started for sink running V3.2B5 build 2953 from Tue Aug 31 05:47:41 Pacific Daylight Time 2010.<br /><01-Apr-2011 Pacific Daylight Time 12:51:41.568> <gpp><br /> Started for source running V3.2B5 build 2953 from Tue Aug 31 05:47:41 Pacific Daylight Time 2010.<br /><01-Apr-2011 Pacific Daylight Time 12:51:41.622> <gpp><br /> Set up cache with 2 framesets of 1 frames/set (total frames = 2).<br /> Set up archive with 10 filesets of 100000 framesets/set (total frames = 1000000)<br />-----</p>
<p>Error, at the very end of log:<br />------<br />Exception in thread "gpp" java.lang.OutOfMemoryError<br /> <<No stacktrace available>><br /><02-Apr-2011 Pacific Daylight Time 10:53:49.549> <RCO com.rbnb.api.TCPRCO@40d55860><br /> java.lang.IllegalStateException: Cannot reconnect to existing client handler /dataturbine/gpp.<br /> at com.rbnb.api.RCO.login(RCO.java:1373)<br /> at com.rbnb.api.SerializingRCO.login(SerializingRCO.java:351)<br /> at com.rbnb.api.TCPRCO.login(TCPRCO.java:461)<br /> at com.rbnb.api.RCO.process(RCO.java:1653)<br /> at com.rbnb.api.RCO.run(RCO.java:2210)<br /> at java.lang.Thread.run(Thread.java:743)<br />--------</p> Bug #5362 (New): Create a GUI so user can easily archive a senosr's data from a dataturbine to a ...https://projects.ecoinformatics.org/ecoinfo/issues/53622011-03-31T21:39:54ZJing Taotao@nceas.ucsb.eduBug #5347 (New): DataTurbine actor should support Start Time with sub-second accuracyhttps://projects.ecoinformatics.org/ecoinfo/issues/53472011-03-10T01:15:13ZDerik Barseghianbarseghian@nceas.ucsb.edu
<p>DT actor Start TIme parameter should support sub-second accuracy. It would be nice if it accepted dates specified in a variety of formats, including [milli]seconds since epoch, and made apparent at least one of these formats - currently the user has no clue.</p> Bug #5341 (New): Belkin f5u409 usb=>serial dongle periodically stops workinghttps://projects.ecoinformatics.org/ecoinfo/issues/53412011-03-05T02:35:09ZDerik Barseghianbarseghian@nceas.ucsb.edu
<p>Sometimes the Belkin dongle gets in a state where all 3 of its LEDs stay off. This corresponds with being unable to run cr1k_d over it. I'm not sure if this is the sole reason for bug#5332, or if that sometimes happens even when the lights are on.</p>
<p>When this state occurs, and you try to use cr1k_d, dmesg starts showing errors like:</p>
<p>usb 1-1: cr1k_d timed out on ep0out len=0/1<br />usb 1-1: Set MODEM CTRL 0xb failed (error = -110)<br />usb 1-1: cr1k_d timed out on ep0out len=0/1<br />usb 1-1: Set LINE CTRL 0x3 failed (error = -110)<br />usb 1-1: cr1k_d timed out on ep0in len=0/1<br />usb 1-1: Get MODEM STATus failed (error = -110)<br />usb 1-1: cr1k_d timed out on ep0out len=0/1</p>
<p>I've tried this procedure to see if I could get the dongle working again, but it didn't work:<br />lsmod | grep -i usb<br />sudo rmmod mct_u232; <br />lsmod | grep -i usb; <br />sudo modprobe mct_u232</p>
<p>Rebooting consistently makes it work again.</p>
<p>Nick recommends trying a Keyspan dongle, and updating from ubuntu 9.04 to 10.04 (new kernel, possible hardware fixes). <br />I'll try the dongle he recommends when I get a chance.</p> Bug #5116 (In Progress): add lat/lon to sensor metadata and use during importhttps://projects.ecoinformatics.org/ecoinfo/issues/51162010-08-02T23:19:25ZDaniel Crawldanielcrawl@gmail.com
<p>The site layout in engineering workflows can reflect the physical layout of sensors at a deployment site. If lat/lon metadata were stored on the sensor server, this location information could be used when importing a workflow into Kepler.</p>
<p>The tasks for this bug are:</p>
<p>1. see if SPAN allows lat/lon metadata per sensor<br />2. if not, try to add it<br />3. use lat/lon metadata during import</p> Bug #4773 (New): Data processing on the GPPhttps://projects.ecoinformatics.org/ecoinfo/issues/47732010-02-08T23:51:25ZIlkay Altintasaltintas@sdsc.edu
<p>Do we allow data processing on the GPP unit? Could it be a Kepler workflow, e.g., for quick QA?QC.</p> Bug #4763 (New): create gui to access datalogger programhttps://projects.ecoinformatics.org/ecoinfo/issues/47632010-02-05T18:57:27ZDaniel Crawldanielcrawl@gmail.com
<p>Provide an interface within engineering view to download, view, upload, and install the program running on the datalogger. Note: this may not be possible when using the SPAN software.</p> Bug #4750 (New): create wiring table guihttps://projects.ecoinformatics.org/ecoinfo/issues/47502010-02-05T18:34:25ZDaniel Crawldanielcrawl@gmail.com
<p>Create a wiring table interface that can describe all the connections between sensors and datalogger(s) at a sensor deployment. Note that this table is for documentation purposes only.</p>
<p>See figure 3 in <a class="external" href="https://kepler-project.org/developers/incubation/kepler-engineering-view-for-reap/engineering-view-plans">https://kepler-project.org/developers/incubation/kepler-engineering-view-for-reap/engineering-view-plans</a></p> Bug #4741 (New): create icons for engineering view componentshttps://projects.ecoinformatics.org/ecoinfo/issues/47412010-02-05T18:18:45ZDaniel Crawldanielcrawl@gmail.com
<p>Create icons for engineering view components in Kepler. This includes: sensor, datalogger, dataturbine, etc. Also, need to have different colors of the icons to show status: green, red, yellow.</p>
<p>See figures 1 and 4 in <a class="external" href="https://kepler-project.org/developers/incubation/kepler-engineering-view-for-reap/engineering-view-plans">https://kepler-project.org/developers/incubation/kepler-engineering-view-for-reap/engineering-view-plans</a></p>