Project

General

Profile

Actions

Bug #5796

closed

D1Object in Morpho still uses byte array as data source

Added by Jing Tao about 11 years ago. Updated about 11 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
morpho - general
Target version:
Start date:
01/18/2013
Due date:
% Done:

0%

Estimated time:
Bugzilla-Id:
5796

Description

When morpho reads and saves the data package to the DataONE network, it handle the D1Objects. The entities, metadata and ore document will be transformed to the D1Object. However, the D1Object is still using the byt arrary as data source. That means we may put a big data file into the memory. This may cause some memory issue. The orignal design is that the D1Object will have a class attribute - a DataStore interface. The DataStore interface get and set methods. Those methods are handling InputStream and OutputStream, rather than byte array.

Actions #1

Updated by ben leinfelder about 11 years ago

We discussed the following changes to D1Object.
-add: getDataInputStream() and setDataInputStream()
-modify: getData() and setData() to operate without actually storing the byte[]
-use the stream-based methods in Morpho.

D1Object will likely defer to the D1Client (with cache enabled) to retrieve the data input stream initially.

Actions #2

Updated by Jing Tao about 11 years ago

I am thinking about the two methods we will add (in comment 1):

getDataInputStream() and setDataInputStream()

I think we should call getDataOutputStream()

So we should have a way to persistent the data in D1Object.

I believe the DataStore interface concept is still very useful. You may find it at;
https://repository.dataone.org/documents/Projects/cicore/architecture/api-documentation/source/design/morpho/images/morpho-class-diagram.png

DataStore has two methods:

OutputStream get(Identifier id);
void set(Identifier id, InputStream input);

Actions #3

Updated by Jing Tao about 11 years ago

create an issue on the dataone redmine. So the dataone persons can know what we will do.
https://redmine.dataone.org/issues/3523

Actions #4

Updated by Jing Tao about 11 years ago

Now we use DataSource for D1Object to replace the byte[]

Actions #5

Updated by Redmine Admin almost 11 years ago

Original Bugzilla ID was 5796

Actions

Also available in: Atom PDF