Bug #1070
openpropose a flexible system for transformation steps
0%
Description
In any workflow, linking any two steps is likely to require some sort of
transformation to get the outputs of one step to match the inputs of the next.
This transformation is sometimes a simple cast (e.g., int -> float), sometimes a
more complex cast (e.g., string -> date), sometimes will require more complex
processing (e.g., statistical processing), and sometimes will require schema
transformations. In all cases, the transformation between step A and step Z can
be represented as one or more intermediate tansformation steps:
A -> T1 -> T2 -> TN -> Z
Our challenge includes:
1) use the SMS to locate candidate transformation steps T1..TN based on type
signature and ontologies
2) determine how to generate transformation steps automatically for simple
transforms such as unit conversions
3) create a simple GUI for creating transformation steps that map between two
existing steps
4) determine the pros and cons of having transformation steps be directly
associated with links (e.g., a link property) rather than simply introducing new
transform steps that do the same tasks directly into the pipeline
So, the objective of this bug is to:
1) propose a flexible system for discovering and generating transformation steps
2) identify the suite of complex issues involved in generating a mapping from
one step to another (especially when this might require several intermediate steps)
3) propose a plan of work for completing this plan