Bug #954
closed
cache should not expunge temp objects that are needed later in an analysis
Added by Chad Berkley almost 22 years ago.
Updated over 21 years ago.
Category:
monarch - general bugs
Description
need to make sure that the cache does not expunge any temporary data that is
stored by a previous stepgroup in a pipeline and may be needed later in the
pipeline. need to have some way for a pipeline to specify that a temporary data
object should persist until the entire pipeline has executed.
callers of the cache need to be able to control the fate of objects submitted.
there needs to be expungeNow(cacheid), neverExpunge(cacheid), and
allowExpunge(cacheid) methods so that callers have more precise control of what
is in the cache. the expungeNow method would force the cache to remove the
specified object. neverExpunge would stop the cache from expunging the object
alltogether and allowExpunge would allow the cache to expunge the object the
next time it needed to without requiring it to do so immediately.
the methods expungeNow, allowExpunge and neverExpunge are now implemented in the
cache. It occured to me while doing this that adding a feature to set a time
out on a cached file would also be useful. For instance, if the pipeline
produces an image, you could set the time out to be 5 minutes, then the file
would be removed from the server. five minutes would be plenty of time for a
client to download an image or for someone to download it from a browser.
I also discussed with Matt making the cache manager the ultimate decision maker
on what stays and what goes. So I think I need to make an upper absolute limit
on the number of MB that can be in the cache but a lower purge limit for
expunging uneeded files. The cache would absolutely disallow the cache size to
go over the max, but would start expunging the files that it could when it got
to the lower limit.
The other thing i still need to do is to make
ScriptLanguagePlugin.createOutputData intelligently set the allowPurge and
neverPurge attributes of a cacheitemdescription.
The cache manager now has two parameter settings which are set in the config.xml
file (replaced by ant in build.xml). maxdisksize is the upper bound over which
the cache will never go. purgedisksize is the size at which the cache will
start purging items.
now just need to make ScriptLanguagePlugin.createOutputData intelligently set
the allowPurge and neverPurge attributes of a cacheitemdescription.
The cache now has funtionality to expunge items after a certain amount of time.
The expunge can be cancelled before the time limit expires. if the time limit
expires, the cache item is expunged immediately.
Original Bugzilla ID was 954
Also available in: Atom
PDF