Bug #2328
closedMemory use problems with multiple EML Datasources
0%
Description
The workflow 'IPCC_Base_Layers.xml' contains 10 eml2 datasource actors. It is
thus a severe test of multiple datasources. [#16 in ENM_Workflows.html] In
Alpha8, the workflow will open and eventually download all the datasources
(although slowly) with the java heap set to 512M (i.e. the java command line
switch is '-Xmx512m' ) on a Windows machine with 2G of ram.
However, when using the head of CVS (on 1/11/2006), java 'out of memory' errors
occur when running on the same machine. Out of memeory errors sometimes also
occur when the java memory request is increased to '-Xmx1024m'. [Loading of some
of the data souces appears much faster with the head of CVS; out of memory
problem appears to be a thread problem because not exactly repeatable.]
Dan Higgins
Updated by Kevin Ruland over 18 years ago
In r1.26 of org.ecoinformatics.seek.datasource.EcogridDataCacheItem, I added a
simple mutex mechanism which only allows a single download to proceed at a time.
This is functionally similar to the exclusion mechanism in the Alpha8 code
base. This mutex needs to be replaced with a more flexible thread pool and
analyzed for thread-safety.
Updated by Dan Higgins over 18 years ago
The change referred to by Kevin (single download at a time) does indeed fix the
memory problems referred to. (Dan Higgins)
Updated by Kevin Ruland about 18 years ago
Committed change to ecogrid seek repository adding EcogridGetToStreamClient
class which greatly reduces memory requirements during downloads. This client
is used in Kepler for all ecogrid get operations.