Project

General

Profile

Bug #5433

use 1024m instead of 512m for reporting suite

Added by Derik Barseghian about 8 years ago. Updated about 8 years ago.

Status:
Resolved
Priority:
Normal
Category:
reporting
Target version:
Start date:
06/28/2011
Due date:
% Done:

0%

Estimated time:
Bugzilla-Id:
5433

Description

Kepler has a max memory setting of 512m by default. Increasing this to 1024m when using the reporting suite enables us to increase the hsqldb.cache_scale setting to 18 (which greatly improves workflow execution time performance when provenance is on by using more memory). Without making this change kepler will lock up with heap space errors after a few executions of data intensive workflows.

History

#1 Updated by Derik Barseghian about 8 years ago

Just increasing to 768m is enough. In fact the one set of tests i did (below) surprisingly shows this as having better performance than 1024.

I made the change to use 768m at r27813 (note this change is just for users of the provenance suite, not regular kepler). If the kepler jvm max mem setting is < 768, it's increased. Note this means: 1) 1st kepler session will not have this memory setting in effect, and 2) this will be annoying for a user intentionally trying to set memory <768, it will get reset each time. I don't think these are big deals though. I considered and decided against only changing to 768 if the default of 512 is in use (i.e. exactly == 512).

I ran the growingDegreeDays workflow (see bug#5429) with 200days of data on my linux box with provenance in file-mode (remove "localhost" from provenance configuration.xml). After each memory change I quit kepler, and removed KeplerData and .kepler.

512m
-------------
heap space errors on 4th run

768m
-------------
1:26, 2:01, 2:23, 3:05, 2:55, 3:14, 3:15, 4:14, 3:15, 3:38, 3:37, 5:32, 4:13, 3:57, 3:25, 5:50, 4:05, 4:17, 3:48, 7:09, 4:51, 4:31, 4:20, 7:54

total: 5815s or 96.9min

no heap space errors

1024m
-------------
1:24, 2:00, 2:33, 3:02, 2:46, 4:24, 3:04, 4:27, 3:03, 3:18, 3:13, 5:28, 4:34, 4:05, 4:08, 7:02, 4:40, 4:30, 4:43, 9:37, 7:04, 6:42, 5:10, 11:08

total: 6725s or 112.08min

no head space errors

#2 Updated by Redmine Admin over 6 years ago

Original Bugzilla ID was 5433

Also available in: Atom PDF