Project

General

Profile

Bug #3473

allow user to attach, replace, or delete data table on existing entity description

Added by Matt Jones almost 11 years ago. Updated over 8 years ago.

Status:
Resolved
Priority:
Immediate
Category:
morpho - general
Target version:
Start date:
08/20/2008
Due date:
% Done:

0%

Estimated time:
Bugzilla-Id:
3473

Description

Currently users can describe a data table in Morpho without importing the data. This causes the attributeList to be filled out, but the data table is not associated with the package. This feature used to exist, but apparently has been removed, and we need to add it back. Its a relatively simple matter of:

1) Add the data table to the profiles/data directory with an appropriate id number
2) Add an online distribution URL to the physical element of the entity description

This would look like this:
<distribution>
<online><url>ecogrid://knb/jones.354.1</url></online>
</distribution>

If Morpho was used to describe the data, there may already be an empty offline distribution description, which should be deleted if it is empty:
<distribution>
<offline><mediumName> </mediumName></offline>
</distribution>

It's not clear to me why this is being created in the first place -- we should probably stop creating it, but that is a separate issue.

3) This procedure assumes that the newly attached entity matches the existing entity and attribute list, which may be a bad assumption. We should probably test if they are in agreement (e.g., right # and types of columns)

4) Add a GUI element that allows one to attach a data table for an entity that currently doesn't have an attached table

That should do it.

History

#1 Updated by Jing Tao almost 11 years ago

move to 1.6.2

#2 Updated by Jing Tao almost 11 years ago

move to 1.7

#3 Updated by Jing Tao almost 10 years ago

move to 1.7.1

#4 Updated by Jim Regetz over 9 years ago

This should be evident from the bug summary line, but just to clarify: Morpho should be able to handle not only attaching a data table to existing attribute metadata in cases where a data table wasn't previously attached (Matt's description), but also replacing an existing data table with a different one. I imagine the procedure would be largely the same for the replacement case, but with incrementing the revision number in the table id rather than assigning a new id altogether?

Deleting an attached data table (without deleting the attribute metadata) could be useful, but is a lower priority enhancement. Most use cases should be covered by the fact that users can already delete both data table and its metadata, or modify access rules for the data table to make it private.

#5 Updated by ben leinfelder over 9 years ago

reassign/target for semtools

#6 Updated by ben leinfelder about 9 years ago

"Data->Replace Current Data Table..." presents a dialog where you can select a different local data file.
The file is saved in the Morpho cache, and the following metadata is updated:
-"online" url (with the ecogrid:// prefix)
-object name (set as file name)
-file size in bytes
-number of records (if text file)

Current assumptions:
-you are replacing a nearly identical file (attributes are not checked). mismatches don't cause a failure, but the display is odd (say you have three columns, but the new data only has two, then you see the third column as empty).
-you are not "adding" the data to metadata that was previously there just describing non-existent data.
I need to create a test scenario where we only have descriptions of the data and did not actually import a file. I imagine we can handle this case in the same command.

Notes:
At this point I'm not supporting the manually entered URL for data file references (like you can get with the new data import wizard). This may prove to be an area we wish to improve upon.

#7 Updated by Jim Regetz about 9 years ago

Great start! This will be a fantastic enhancement. A couple of comments:

1. When a table is replaced, it would make more sense for Morpho to increment the rev number rather than assign a new docid, as this operation really constitutes 'data revision' rather addition of a new table. (Unless of course this is the first time the data is being attached to existing attribute metadata, if/when that capability is added.)

2. In the Data menu, a user might assume 'Replace Current Data Table..." and "Delete Current Data Table" sound like parallel operations, but they're not. The former replaces only the data, whereas the latter deletes both data and metadata. Perhaps modify the confirmation text for the Delete operation to emphasize that the table and its attribute metadata will be deleted.

#8 Updated by ben leinfelder about 9 years ago

for #1:
I've incremented the data file id now.
I have reservations about doing this - especially in a networked environment - since we historically have a horrible time keeping track of where/when doc revisions exist locally and networked.
But since Jing put in all the extra handling to deal with docid conflicts, we may escape unscathed!

(In reply to comment #7)

Great start! This will be a fantastic enhancement. A couple of comments:

1. When a table is replaced, it would make more sense for Morpho to increment
the rev number rather than assign a new docid, as this operation really
constitutes 'data revision' rather addition of a new table. (Unless of course
this is the first time the data is being attached to existing attribute
metadata, if/when that capability is added.)

2. In the Data menu, a user might assume 'Replace Current Data Table..." and
"Delete Current Data Table" sound like parallel operations, but they're not.
The former replaces only the data, whereas the latter deletes both data and
metadata. Perhaps modify the confirmation text for the Delete operation to
emphasize that the table and its attribute metadata will be deleted.

#9 Updated by Jim Regetz about 9 years ago

(In reply to comment #8)

I've incremented the data file id now.
I have reservations about doing this - especially in a networked environment -
since we historically have a horrible time keeping track of where/when doc
revisions exist locally and networked.
But since Jing put in all the extra handling to deal with docid conflicts, we
may escape unscathed!

Yeah I thought about this, but there is a precedent for incrementing the rev number insofar as that's how it works when you modify (delimited text) data using the Morpho table editor. As far as I know, the docid conflict resolution step will handle any issues when the document is saved. I've never actually tested it when saving to metacat, but I know it works as expected when you modify a previous version of a table and try to save locally (i.e. changing foo.1.1 and trying to save it when foo.1.2 exists already).

#10 Updated by ben leinfelder about 9 years ago

I double checked that the data docid conflict was caught and that you are prompted for action (locally). Still works!

#11 Updated by ben leinfelder about 9 years ago

First section of "Data" menu now has these options:
--------------------------
Create/Import New Data Table...
Replace Current Data...
Convert Data to Table...
Delete Current Data Entity
Edit Data Access

#12 Updated by ben leinfelder about 9 years ago

changing the Entity Name to match the Object Name. I actually think I'll add at
text field so that you can enter a more descriptive name to the entity if you
wish. Otherwise it's just duplicated information (filename).

#13 Updated by ben leinfelder about 9 years ago

"Data->Replace Current Data..." can now be used to add data that has previously only been described.

#14 Updated by ben leinfelder about 9 years ago

I believe I've addressed the various permutations described in this bug - reopen as necessary.

#15 Updated by ben leinfelder over 8 years ago

At TFRI they brought up a very good point:
Say I duplicate a DP as a means of creating a metadata template - then I replace the data on the copy.
I end up incrementing the data file id - but it's still the old data id in the original DP that I started from.
So now I have the following 2 DPs:
metadata.1.1 with data.1.1
and
metadata.2.1 with data.1.2

There is a conflict now! Either we need an option to make new data ID or we just always make a new data id.

(In reply to comment #8)

for #1:
I've incremented the data file id now.
I have reservations about doing this - especially in a networked environment -
since we historically have a horrible time keeping track of where/when doc
revisions exist locally and networked.
But since Jing put in all the extra handling to deal with docid conflicts, we
may escape unscathed!

(In reply to comment #7)

Great start! This will be a fantastic enhancement. A couple of comments:

1. When a table is replaced, it would make more sense for Morpho to increment
the rev number rather than assign a new docid, as this operation really
constitutes 'data revision' rather addition of a new table. (Unless of course
this is the first time the data is being attached to existing attribute
metadata, if/when that capability is added.)

2. In the Data menu, a user might assume 'Replace Current Data Table..." and
"Delete Current Data Table" sound like parallel operations, but they're not.
The former replaces only the data, whereas the latter deletes both data and
metadata. Perhaps modify the confirmation text for the Delete operation to
emphasize that the table and its attribute metadata will be deleted.

#16 Updated by ben leinfelder over 8 years ago

Added a checkbox to specify that a new ID should be generated for the replaced data.
If left unchecked, the data id only has it's revision incremented, but if checked a completely new id is created.

#17 Updated by Redmine Admin about 6 years ago

Original Bugzilla ID was 3473

Also available in: Atom PDF