Kepler

One of the reasons DiGIR is slow is because we ALWAYS hit all 10 or providers
and have to wait for all the resultsets to come back before it can return any
results.

This "should" be improved once we have metadata in the regisrty and enable the
users to be more selective about where they want to search.

If they don't want to be selective and want to get all the data from everywhere
then it will take a while. This has partically to do with setting the user's
expectations about the task they are performing. We know what it is doing behind
the scenes so we have a different set of expectations.

They have no idea the amount of data that is being searched and how it is
collected and combined and returned.

Also, as we enable more providers the slowness with only get worse.

Giving some further input on this to help the debug progess. When some of the
users couldn't get any search results returned on their machines, they moved
to a neighbor's machine (that had previously been successful in getting some
search results) and performed the search there and some users were able to get
results.

The same problems occured when the search term was "test" or "Datos
Meteorologicos." With the latter, people tried various pieces of the search
term. Some used the entire search term, others used just "datos" or "Datos"
and still were unable to get any search results.

Still other users tried re-booting Kepler and at least one user was successful
in getting search results after doing that.

A few users tried to limit the search to just metacat and were still
unsuccessful in getting any search results returned.

Regardless of whether users are searching across a large data set or several
large data sets, their expectation was some type of results, not a time out
(or the progress indicator just stopping with no results returned). Obviously
the moving progress indicator is what lets users know that things are still
working, and it is a nice feature. However, if we anticipate even more
slowness in the future, we will need to design some additional feedback, such
as letting the user know the system is searching data set 1, then data set 2
etc.

Initial fixes introduced using the Factory method to fix some of these issues.
Need to verify that they work, and adapt to the decision to move to plain web
services in the next EcoGrid release. Then test, and close this bug. Higgins
agreed to develop a test case for this demonstrating data access failures.

This bug is related to Bug # 2174 in SEEK Bugzilla which describes details of
problems with mutliple datasources.

A test for this is the workflows/eco/IPCC_Base_Layers.xml workflow with tries to
load 10 IPCC datasource files. This reliably 'hangs' on the last of the
datasources (which remains 'red').

After the ecogrid switch to pure axis, the issues seems get better. In the
workshop at Jan. 2006, about 30 people kept searching and getting data from
ecogrid and it worked well.

I used 3 machine to do a stress testing in ecogrid and it worked well.

But the stress testing by the cluster in NCEAS (which has 14 machines),
sometimes failed - some search didn't get result set back.

Although there are some performance issues, the ecogrid connection now seems much more robust. I am thus removing this general bug as FIXED. Specific other problems should be added as needed. (Dan Higgins)

In 2008 workshop in New Mexico, the ecogrid search worked fine. Dan and I think we can close this bug.

Original Bugzilla ID was 1836