Ecoinformatics Redmine: Tom Parrishttps://projects.ecoinformatics.org/ecoinfo/https://projects.ecoinformatics.org/ecoinfo/ecoinfo/favicon.ico?14691340362011-06-24T21:40:10ZEcoinformatics Redmine
Redmine Kepler - Bug #4827: RExpression does not handle matrix tokens correctly.https://projects.ecoinformatics.org/ecoinfo/issues/4827#change-166022011-06-24T21:40:10ZTom Parrisparris@isciences.com
<p>I just re-tested this with the latest trunk. The problem is still there.</p> Kepler - Bug #4859: GDALTranslate does not work on Windows 7 (64bit)https://projects.ecoinformatics.org/ecoinfo/issues/4859#change-167382010-03-08T21:10:44ZTom Parrisparris@isciences.com
<p>Yes. But for the moment it has to take second place to meeting our own rather extreme schedule commitments. I think we can have something in deliverable shape by late April/early May.</p> Kepler - Bug #4859: GDALTranslate does not work on Windows 7 (64bit)https://projects.ecoinformatics.org/ecoinfo/issues/4859#change-167362010-03-08T19:36:27ZTom Parrisparris@isciences.com
<p>OK. We're in the process of building the new suite of java language bindings now. We've gotten them to compile on a 32 bit platform and our next step is to port to the 64bit platform.</p>
<p>We are doing this in preparation of writing some new Kepler GDAL actors. The new GDAL actors will read images into matrix tokens so we can create pipelines of actors that act on the data without having to read/write it for each operation. (Please let me know if I've missed something and this capability already exists elsewhere in Kepler).</p>
<p>I've poked around the current suite of Kepler GDAL actors. They make use of a custom coded JNI (not the generic java language bindings). So, I think we just need to provide the GDAL dll's for each platform and the dll for the JNI for the 64bit platform.</p>
<p>Our first priority is getting our new actors up and running on the 64bit platform. We'll try to get the existing GDAL actors up and running on 64bits after that.</p>
<p>-- Tom</p> Kepler - Bug #4859: GDALTranslate does not work on Windows 7 (64bit)https://projects.ecoinformatics.org/ecoinfo/issues/4859#change-167342010-03-08T13:45:44ZTom Parrisparris@isciences.com
<p>Apologies for the delay. I've been travelling.</p>
<p>We obviously have the requisite hardware and can compile the GDAL C libraries. But we need some instructions on how to package things appropriately. Can someone send a recipe?</p>
<p>We are separately trying to compile the new GDAL JAVA language bindings to build some new actors. Stay tuned for progress on that. Assuming we are successful, this would eliminate the need for the Kepler provided JNI interface that only supports a couple of the GDAL methods.</p> Kepler - Bug #4859 (In Progress): GDALTranslate does not work on Windows 7 (64bit)https://projects.ecoinformatics.org/ecoinfo/issues/48592010-03-02T15:41:15ZTom Parrisparris@isciences.com
<p>GDALJniGlue fails to intialize on Windows 64bit operating systems.</p>
<p>ptolemy.kernel.util.IllegalActionException: in .GDAL_Translate_test_oak.manager<br />Because:<br />Could not initialize class org.ecoinformatics.seek.gis.gdal.GDALJniGlue<br />at ptolemy.actor.Manager.execute(Manager.java:472)<br />at ptolemy.actor.Manager.run(Manager.java:1119)<br />at ptolemy.actor.Manager$3.run(Manager.java:1160)<br />Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.ecoinformatics.seek.gis.gdal.GDALJniGlue<br />at org.ecoinformatics.seek.gis.gdal.GDALTranslateActor.fire(GDALTranslateActor.java:207)<br />at ptolemy.actor.AtomicActor.iterate(AtomicActor.java:469)<br />at ptolemy.actor.sched.StaticSchedulingDirector.fire(StaticSchedulingDirector.java:188)<br />at ptolemy.actor.CompositeActor.fire(CompositeActor.java:458)<br />at ptolemy.actor.Manager.iterate(Manager.java:714)<br />at ptolemy.actor.Manager.execute(Manager.java:349)<br />... 2 more<br />Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.ecoinformatics.seek.gis.gdal.GDALJniGlue<br />at org.ecoinformatics.seek.gis.gdal.GDALTranslateActor.fire(GDALTranslateActor.java:207)<br />at ptolemy.actor.AtomicActor.iterate(AtomicActor.java:469)<br />at ptolemy.actor.sched.StaticSchedulingDirector.fire(StaticSchedulingDirector.java:188)<br />at ptolemy.actor.CompositeActor.fire(CompositeActor.java:458)<br />at ptolemy.actor.Manager.iterate(Manager.java:714)<br />at ptolemy.actor.Manager.execute(Manager.java:349)<br />at ptolemy.actor.Manager.run(Manager.java:1119)<br />at ptolemy.actor.Manager$3.run(Manager.java:1160)</p> Kepler - Bug #3205: gdalTranslate actor crashes on bad format and won't work with all gdal formatshttps://projects.ecoinformatics.org/ecoinfo/issues/3205#change-109982010-03-02T15:31:21ZTom Parrisparris@isciences.com
<p>We're having a similar problem on the 2.0.0 trunk. This is a trivial workflow that attempts to convert an ERDAS IMG file to Arc Info ASCII Grid on Windows XP (32 bit).</p>
<p>[run] Opened file:/Y:/Kepler/Workflows/GDAL_Translate_test.xml in 18407 ms.<br />[run] Size Statistics for .GDAL_Translate_test<br />[run] AtomicEntities: 6<br />[run] CompositeEntities: 0<br />[run] Relations: 4<br />[run] Attributes: 88<br />[run] Atomic Actor Names and Counts:<br />[run] ptolemy.actor.lib.gui.Display 4<br />[run] org.ecoinformatics.seek.gis.gdal.GDALTranslateActor 1<br />[run] ptolemy.actor.lib.StringConst 1<br />[run]<br />[run] Composite Entity Depths and Counts:<br />[run]<br />[run] Opened Y:\Kepler\Workflows\GDAL_Translate_test.xml in 18968 ms.<br />[run] Input file size is 720, 360<br />[run] 0...10...20...30...40...50...60...70...80...90...java.lang.NullPointerException<br />[run] 8688 ms. Memory: 66708K Free: 16133K (24%)</p> Kepler - Bug #4010: RExpression2 - tracking for backwards compatibilityhttps://projects.ecoinformatics.org/ecoinfo/issues/4010#change-135562010-02-23T20:58:26ZTom Parrisparris@isciences.com
<p>First ... apologies. I just submitted an RExpression bug that you've already fixed in Rexpression2 (column major matrices).</p>
<blockquote>
<p>SR2 - It's best to "Instantiate" the RExpression actor from the menu bar rather<br />than editing the actor on the canvas. I know you can hand edit the full class<br />name in the MOML file of your workflow and then open it in Kepler, but that is<br />a hassle and prone to typos. Use the menu option, definitely.</p>
</blockquote>
<p>OK. Getting ready to test with R-2.10.1, but not sure I understand SR2 above (and this may be related to why I am still having problems with matrices). If I search for RExpression in the "Search Components" box, the only actor I get is RExpression (no RExpression2). So the only way I can get to RExpression2 is to instantiate it from the Tools menu. When I do that, I get a bunch of character escaping errors due to Windows style back slashes.</p>
<p>Error in png(filename = "C:Usersparris.keplercache-2.0.0modules\rR_Matrix_Test_1266958503103RExpression1.png", : <br /> unable to start device<br />In addition: Warning messages:<br />1: \U used without hex digits <br />2: '\p' is an unrecognized escape in a character string <br />3: '\.' is an unrecognized escape in a character string <br />4: '\c' is an unrecognized escape in a character string <br />5: '\m' is an unrecognized escape in a character string <br />6: '\R' is an unrecognized escape in a character string <br />7: '\R' is an unrecognized escape in a character string</p>
<p>If I change the specification of the working directory for the actor to use forward slashes, I get the same problem.</p> Kepler - Bug #4827 (New): RExpression does not handle matrix tokens correctly.https://projects.ecoinformatics.org/ecoinfo/issues/48272010-02-23T20:38:25ZTom Parrisparris@isciences.com
<p>I've attached a simple workflow that demonstrates a bug in the way the RExpression actor translates Kepler/Ptolemy matrices to R matrices. There seems to be a problem in the autmatically generated "matrix" statement in R that is likely due to confusing row and column counts.</p>
<p>A simple 2 row x 5 column constant matrix ([1, 2, 3, 4, 5; 101, 102, 103, 104, 105]) is passed to Rexpression on the min port. The R script simply displays the matrix and passes it along to the mout port.</p>
<p>There are three displays.</p>
<p>1. min displays the matrix before it enters the RExpression actor. It is fine.</p>
<p>2. mout displays the matrix after it leaves the RExpression actor. It too is fine (indicating that the translation problem inverts itself when baorcasting to the output port).</p>
<p>3. The problem is in the Routput display.</p>
<blockquote>
<p>min</p>
</blockquote>
[,1] [,2] [,3] [,4] [,5]<br />[1,] 1 3 5 102 104<br />[2,] 2 4 101 103 105
<p>Note how the matrix is scrambled. The automatically generated R script line that reads:</p>
<pre><code>`min` <- matrix(c(1, 2, 3, 4, 5, 101, 102, 103, 104, 105), nrow=2,ncol=5)</code></pre>
<p>should read</p>
<pre><code>`min` <- matrix(c(1, 101, 2, 102, 3, 103, 4, 104, 5, 105), nrow=2,ncol=5)</code></pre> Kepler - Bug #4826: intermittent exceptions displaying matrix tokenshttps://projects.ecoinformatics.org/ecoinfo/issues/4826#change-165972010-02-23T18:23:34ZTom Parrisparris@isciences.com
<p>Forgot to specify platform.</p> Kepler - Bug #4826 (Resolved): intermittent exceptions displaying matrix tokenshttps://projects.ecoinformatics.org/ecoinfo/issues/48262010-02-23T18:22:46ZTom Parrisparris@isciences.com
<p>I experience intermittent exceptions displaying matrix tokens in a simple workflow. The problems are more frequent when using the runtime window, but I can generate the exceptions either way.</p>
<p>The offending workflow is attached to this message.</p>
<p>Sample of problem while repeatedely pressing the "run workflow " button (normal presentation window).<br /> [run] 16 ms. Memory: 58332K Free: 18864K (32%)<br /> [run] 15 ms. Memory: 58332K Free: 17878K (31%)<br /> [run] 16 ms. Memory: 58332K Free: 17001K (29%)<br /> [run] 16 ms. Memory: 58332K Free: 16048K (28%)<br /> [run] 16 ms. Memory: 58332K Free: 18800K (32%)<br /> [run] 16 ms. Memory: 58332K Free: 17876K (31%)<br /> [run] Exception in thread "AWT-EventQueue-0" java.lang.ArrayIndexOutOfBoundsException: 3 >= 2<br /> [run] at java.util.Vector.elementAt(Vector.java:427)<br /> [run] at javax.swing.table.DefaultTableColumnModel.getColumn(DefaultTableColumnModel.java:277)<br /> [run] at javax.swing.plaf.basic.BasicTableUI.paintCells(BasicTableUI.java:1967)<br /> [run] at javax.swing.plaf.basic.BasicTableUI.paint(BasicTableUI.java:1767)<br /> [run] at javax.swing.plaf.ComponentUI.update(ComponentUI.java:143)<br /> [run] at javax.swing.JComponent.paintComponent(JComponent.java:751)<br /> [run] at javax.swing.JComponent.paint(JComponent.java:1017)<br /> [run] at javax.swing.JComponent.paintToOffscreen(JComponent.java:5112)<br /> [run] at javax.swing.RepaintManager$PaintManager.paintDoubleBuffered(RepaintManager.java:1475)<br /> [run] at javax.swing.RepaintManager$PaintManager.paint(RepaintManager.java:1406)<br /> [run] at javax.swing.RepaintManager.paint(RepaintManager.java:1220)<br /> [run] at javax.swing.JComponent._paintImmediately(JComponent.java:5060)<br /> [run] at javax.swing.JComponent.paintImmediately(JComponent.java:4870)<br /> [run] at javax.swing.RepaintManager.paintDirtyRegions(RepaintManager.java:803)<br /> [run] at javax.swing.RepaintManager.paintDirtyRegions(RepaintManager.java:714)<br /> [run] at javax.swing.RepaintManager.seqPaintDirtyRegions(RepaintManager.java:694)<br /> [run] at javax.swing.SystemEventQueueUtilities$ComponentWorkRequest.run(SystemEventQueueUtilities.java:125)<br /> [run] at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:209)<br /> [run] at java.awt.EventQueue.dispatchEvent(EventQueue.java:597)<br /> [run] at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:269)<br /> [run] at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:184)<br /> [run] at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:174)<br /> [run] at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:169)<br /> [run] at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:161)<br /> [run] at java.awt.EventDispatchThread.run(EventDispatchThread.java:122)</p>
<p>Sample of problem while repeatedly pressing the "go" button in the "run time window"</p>
<p>[run] 15 ms. Memory: 58332K Free: 15914K (27%)<br />[run] 16 ms. Memory: 58332K Free: 11679K (20%)<br />[run] 0 ms. Memory: 58332K Free: 10640K (18%)<br />[run] Exception in thread "AWT-EventQueue-0" java.lang.ArrayIndexOutOfBoundsException: 2 >= 2<br />[run] at java.util.Vector.elementAt(Vector.java:427)<br />[run] at javax.swing.table.DefaultTableColumnModel.getColumn(DefaultTableColumnModel.java:277)<br />[run] 16 ms. Memory: 58332K Free: 13221K (23%)<br />[run] at javax.swing.plaf.basic.BasicTableUI.paintGrid(BasicTableUI.java:1928)<br />[run] at javax.swing.plaf.basic.BasicTableUI.paint(BasicTableUI.java:1764)<br />[run] at javax.swing.plaf.ComponentUI.update(ComponentUI.java:143)<br />[run] at javax.swing.JComponent.paintComponent(JComponent.java:751)<br />[run] at javax.swing.JComponent.paint(JComponent.java:1017)<br />[run] at javax.swing.JComponent.paintToOffscreen(JComponent.java:5112)<br />[run] at javax.swing.RepaintManager$PaintManager.paintDoubleBuffered(RepaintManager.java:1475)<br />[run] at javax.swing.RepaintManager$PaintManager.paint(RepaintManager.java:1406)<br />[run] at javax.swing.RepaintManager.paint(RepaintManager.java:1220)<br />[run] at javax.swing.JComponent._paintImmediately(JComponent.java:5060)<br />[run] at javax.swing.JComponent.paintImmediately(JComponent.java:4870)<br />[run] at javax.swing.RepaintManager.paintDirtyRegions(RepaintManager.java:803)<br />[run] at javax.swing.RepaintManager.paintDirtyRegions(RepaintManager.java:714)<br />[run] at javax.swing.RepaintManager.seqPaintDirtyRegions(RepaintManager.java:694)<br />[run] at javax.swing.SystemEventQueueUtilities$ComponentWorkRequest.run(SystemEventQueueUtilities.java:125)<br />[run] at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:209)<br />[run] at java.awt.EventQueue.dispatchEvent(EventQueue.java:597)<br />[run] at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:269)<br />[run] at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:184)<br />[run] at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:174)<br />[run] at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:169)<br />[run] at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:161)<br />[run] at java.awt.EventDispatchThread.run(EventDispatchThread.java:122)</p> Kepler - Bug #4010: RExpression2 - tracking for backwards compatibilityhttps://projects.ecoinformatics.org/ecoinfo/issues/4010#change-135542010-01-25T22:26:38ZTom Parrisparris@isciences.com
<p>Ben ... sorry to ask stupid questions (SQs).</p>
<p>SQ1: I just removed R 2.9.0 from my system and now have R 2.10.1. How do I change the Kepler definition of R_HOME to match.</p>
<p>SQ2: I'm assuming that the way to get to RExpression2 is to add the regular RExpression to a workflow and than change the class path ro RExpression2. Correct?</p>
<p>Thanks. I have some specific tests that I'll run regarding treatment of matrices (and large ones at that).</p>
<p>Cheers,<br />Tom</p>