Memory use problems with multiple EML Datasources
The workflow 'IPCC_Base_Layers.xml' contains 10 eml2 datasource actors. It is
thus a severe test of multiple datasources. [#16 in ENM_Workflows.html] In
Alpha8, the workflow will open and eventually download all the datasources
(although slowly) with the java heap set to 512M (i.e. the java command line
switch is '-Xmx512m' ) on a Windows machine with 2G of ram.
However, when using the head of CVS (on 1/11/2006), java 'out of memory' errors
occur when running on the same machine. Out of memeory errors sometimes also
occur when the java memory request is increased to '-Xmx1024m'. [Loading of some
of the data souces appears much faster with the head of CVS; out of memory
problem appears to be a thread problem because not exactly repeatable.]
Updated by Kevin Ruland about 18 years ago
In r1.26 of org.ecoinformatics.seek.datasource.EcogridDataCacheItem, I added a
simple mutex mechanism which only allows a single download to proceed at a time.
This is functionally similar to the exclusion mechanism in the Alpha8 code
base. This mutex needs to be replaced with a more flexible thread pool and
analyzed for thread-safety.