Bug #748: Systematic approach to rectification - VegBank - Ecoinformatics Redmine

we need a plan for how we will rectify parties, plants, etc. to VegBank records.
VegBranch has a tool for this as a starting point, perhaps. This will be
nearly impossible to automate (in that no computer will be able to judge
ambiguous cases), so it will have to be a step-by-step process.

This plan needs to be created very soon. This bug can be marked resolved as
soon as the plan is finished.

Actions

Copy link

Updated by Robert Peet almost 21 years ago

"Create a plan" -- This must be the mother of all bugs.

So I suppose it was intended to be "create a plant". I think we decided to
omit this option from the loader and require that any plants rejected by the
validation/rectification layer be returned to the submitter to enter through
forms. Ultimately we will provide a mechanism for mass insertion of plants
(perhaps via the Access tool), but for now no. The forms for loading plants
will need to search for matches in the concept/names/references tables and
report misses and close misses. The level at which this gets done will
increase in sophistication as we move beyond version 1.0 to 2.0

I turn this bug over to Michael for refinement, and he will in turn hand it to
Gabe for implementation

Actions

Copy link

Updated by Michael Lee almost 21 years ago

Indeed, this is the mother of all bugs (MOAB). Well, at least all bugs that
pertain to rectification. We sort of made a plan in Santa Barbara last meeting,
which was essentially, that rectification would be a 1.0.1. issue for us to
resolve. How we are going to do rectification remains to be solved, as well as
the different types of things that need rectification.

Currently, I believe our status with recitification is that your value (be that
a closed list value, a party, a plant, a placeNAme) must match exactly or else
your plot gets rejected. We need to "Create a plan" to make rectification work,
but this should probably wait until we have created a framework for web pages to
talk to the db and vice versa. Quite crucial for rectification to work.

So for now, I'll accept this bug as mine, and I will hopefully be able to get us
moving on this for 1.0.1

Actions

Copy link

Updated by Michael Lee over 20 years ago

one of 5 major bugs (or so) and thus critical and highlighted red.

We will use the notes tables to populate information which requires
rectification. Thus we will need to add a note type "rectification Request".

When data are imported into Vegbank via the XML loader, several things may need
rectification.

1) closed lists with values that don't match will need to either:
a) user changes to match a value
b) user changes to make null (if nulls are allowed)

2) [more complex] certain entities that are referenced may need checking to make
sure they are not duplicates (party, reference, plantConcept, commConcept, a few
others). IF a user submits a new one of the above, we might want to do one of
the following:
a) add the new thing, but also add a note requesting that the user go back
and check that this is indeed new. Options for user are then: a) confirm it's
new, b) match to another eleemnt (ie party). IF b) we run into problems because
it would be nice to delete the extraneous party that was created.
b) not add the element, but instead store the XML as the note, and when the
user deals with rectification requests, the XML is reparsed and info presented.
If user says, yes this is new, it's added, if they say, no this is something
else, then the something else gets referenced and the first thing goes away.
This could get complex if very nested elements, but we may be able to use this
route. Would have to have some rectification-placeholder elements, probably, so
that taxonInterpretations could be stored, pointing to uncertain plantConcepts.
This is complex, but almost all elements to be rectified are not too complex:
party, plant, comm, reference, methods.

--------------
anyway, we need to develop a plan that will extend easily to new pieces of data
that we'd like to rectify.

Actions

Copy link