Bug #5796
closedD1Object in Morpho still uses byte array as data source
0%
Description
When morpho reads and saves the data package to the DataONE network, it handle the D1Objects. The entities, metadata and ore document will be transformed to the D1Object. However, the D1Object is still using the byt arrary as data source. That means we may put a big data file into the memory. This may cause some memory issue. The orignal design is that the D1Object will have a class attribute - a DataStore interface. The DataStore interface get and set methods. Those methods are handling InputStream and OutputStream, rather than byte array.
Updated by ben leinfelder about 11 years ago
We discussed the following changes to D1Object.
-add: getDataInputStream() and setDataInputStream()
-modify: getData() and setData() to operate without actually storing the byte[]
-use the stream-based methods in Morpho.
D1Object will likely defer to the D1Client (with cache enabled) to retrieve the data input stream initially.
Updated by Jing Tao about 11 years ago
I am thinking about the two methods we will add (in comment 1):
getDataInputStream() and setDataInputStream()
I think we should call getDataOutputStream()
So we should have a way to persistent the data in D1Object.
I believe the DataStore interface concept is still very useful. You may find it at;
https://repository.dataone.org/documents/Projects/cicore/architecture/api-documentation/source/design/morpho/images/morpho-class-diagram.png
DataStore has two methods:
OutputStream get(Identifier id);
void set(Identifier id, InputStream input);
Updated by Jing Tao about 11 years ago
create an issue on the dataone redmine. So the dataone persons can know what we will do.
https://redmine.dataone.org/issues/3523
Updated by Jing Tao about 11 years ago
Now we use DataSource for D1Object to replace the byte[]