Project

General

Profile

Bug #2328

Memory use problems with multiple EML Datasources

Added by Dan Higgins over 13 years ago. Updated over 13 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
actors
Target version:
Start date:
01/11/2006
Due date:
% Done:

0%

Estimated time:
Bugzilla-Id:
2328

Description

The workflow 'IPCC_Base_Layers.xml' contains 10 eml2 datasource actors. It is
thus a severe test of multiple datasources. [#16 in ENM_Workflows.html] In
Alpha8, the workflow will open and eventually download all the datasources
(although slowly) with the java heap set to 512M (i.e. the java command line
switch is '-Xmx512m' ) on a Windows machine with 2G of ram.

However, when using the head of CVS (on 1/11/2006), java 'out of memory' errors
occur when running on the same machine. Out of memeory errors sometimes also
occur when the java memory request is increased to '-Xmx1024m'. [Loading of some
of the data souces appears much faster with the head of CVS; out of memory
problem appears to be a thread problem because not exactly repeatable.]

Dan Higgins

History

#1 Updated by Kevin Ruland over 13 years ago

In r1.26 of org.ecoinformatics.seek.datasource.EcogridDataCacheItem, I added a
simple mutex mechanism which only allows a single download to proceed at a time.
This is functionally similar to the exclusion mechanism in the Alpha8 code
base. This mutex needs to be replaced with a more flexible thread pool and
analyzed for thread-safety.

#2 Updated by Dan Higgins over 13 years ago

The change referred to by Kevin (single download at a time) does indeed fix the
memory problems referred to. (Dan Higgins)

#3 Updated by Kevin Ruland over 13 years ago

Committed change to ecogrid seek repository adding EcogridGetToStreamClient
class which greatly reduces memory requirements during downloads. This client
is used in Kepler for all ecogrid get operations.

#4 Updated by Redmine Admin over 6 years ago

Original Bugzilla ID was 2328

Also available in: Atom PDF