Bug #1548

consolidating data access user interfaces

Added by Matt Jones over 14 years ago. Updated over 9 years ago.

In Progress
data access
Target version:
Start date:
Due date:
% Done:


Estimated time:


Currently Kepler contains several distinct methods for binding data sources to a
workflow. These include the EML200DataSource actor, the JDBC data source
actor(s), the incipient EcoGrid access interfaces, the GridFTP actor, and
probably others. Each of these exposes the data in a different way, and is
therefore multiply representing data in a confusing way. We need to consolidate
these approaches to find a single UI that can encapsualte all of the data access

This proposal is to use and adapt the user interface described in
kepler/docs/dev/screenshots and related design documents to data access in
EcoGrid, GridFTP, JDBC, and other sources. This would allow a user to view data
uniformly in the workflow, regardless of which data access protocol is used to
get the data. This would also allow the user to specify subsetting constraints
(WHERE clause) uniformly, and to choose which attributes from the joined
relations are exposed to the workflow. Finally, it would allow us to use richer
metadata descriptions of underspecified data sources (like those found at the
other end of JDBC connections) so that the user (and ultimately the SEEK SMS
system) can reason about these data sources effectively.

Related issues

Is duplicate of Kepler - Bug #1887: Consolidate grid actorsResolved2005-01-20


#1 Updated by Chad Berkley almost 14 years ago

uniform put and get will be looked at by jing and efrat.

#2 Updated by Jing Tao almost 14 years ago

The get function should be extened to accept sql query. So user can get partial
data object.

#3 Updated by Chad Berkley almost 14 years ago

  • Bug 1887 has been marked as a duplicate of this bug. ***

#4 Updated by Jing Tao over 13 years ago

Here is the summary after the meeting in March 11:

Federate Metadata across different communities: Create a unified metadata
object, called DataProxy. The DataProxy can get the metadata
(probably\uffffusing a\uffffDataSystem class\uffffas described below) and
parse\uffffit using different metadata formats interpreter, such as EML, Darwin
Core, ADN, FGDC, etc... After parsing the\uffffmetadata, the DataProxy object
will have the info to download\uffffthe\uffffdata object\uffffas described by
the metadata specification\uffffand pass the info to proper DataSystem class to
download the data. The API will include the following
InputStream getFullMetadata(String id, String endPoints);
DataSystem parseMetadata(InpuStream metadata);
void downloadData(DataSystem object);
The DataSystem class:\uffffa generic class to handle get data object (including
metadata object) from different data sources (data system).
API requirements:
Inputstream getData(String identifier, String endPoints);
InputStream getData(other signatures).\uffff
Extending classes: EcoGridDataSystem, MetacatDataSystem(for metacats\uffffthat
don't implement the ecogrid interface), JDBCDataSystem, etc...
Certificate authority: Create a single centralized certificate authority to
provide a shared infrastructure to access and maintain
different sites' certificate authorities, e.g., the GEON portal, the seek
different sites' CAs. - follow up with Karan for more
information about the Grid Account Management Architecture (GAMA) used in the
GEON portal authentication.
A unified web service access to the datasources:
- In order to support other clients than the Kepler interface to\uffffaccess the
various datasources.
- A web service access to datasources with no additional requirements (such as
registering). Communities can benefits from accessing each
other datasources directly.
The GEON and SEEK datasources\uffffaccess architectures\uffffare very similar -
a follow up meeting\uffffis required\uffffon consolidating datasources access
through a unified web service with Kai, Ashraf, Karan, Sandeep, Efrat, Chaitan
from GEON and folks from SEEK.
Unified query for data sources in Kepler either by
adding\uffffmore\uffffdatasources querying\uffffclasses (besides EMLDataSource
and DCDataSource), or
once there is a unified web service access, using a\uffffgeneric web service
actor to query all the data sources.

#5 Updated by Matt Jones about 13 years ago

put/get should be implemented for 1.0.0, but the fully unified data access
interface will not be, so changing milestone.

#6 Updated by Redmine Admin over 5 years ago

Original Bugzilla ID was 1548

Also available in: Atom PDF