https://projects.ecoinformatics.org/ecoinfo/https://projects.ecoinformatics.org/ecoinfo/ecoinfo/favicon.ico?14691340362011-02-07T20:57:46ZEcoinformatics RedmineKepler - Bug #5297: Workflow Run Manager - no indication of progress during preview runs downloadhttps://projects.ecoinformatics.org/ecoinfo/issues/5297?journal_id=183482011-02-07T20:57:46ZMatt Jonesjones@nceas.ucsb.edu
<ul></ul><p>Derik,</p>
<p>When we discussed this before, I think we concluded that the search time was largely due to Kepler issuing an EcoGrid get() request for every result record, which obviously doesn't scale. The Metacat search itself should be subsecond for up to thousands of records. Metacat needs to be configured properly for that to work though -- in that the karxml fields need to be indexed for search to be fast. Other than that, its probably a client-side issue in its decision to download all of the KARs.</p> Kepler - Bug #5297: Workflow Run Manager - no indication of progress during preview runs downloadhttps://projects.ecoinformatics.org/ecoinfo/issues/5297?journal_id=183492011-02-07T22:36:29ZDerik Barseghianbarseghian@nceas.ucsb.edu
<ul></ul><p>This bug is just for fetching data needed to build "preview" WorkflowRuns (bits from the karXML, not the kars or entire karXMLs). I get all records in one query to metacat. I did some quick timing tests of WRM with kepler-dev (there are a few reasons why these shouldn't be considered exact, but they give a good rough idea) :</p>
<p>The ~30s that it currently takes to show all ~131 preview runs for a first time switch of WRM to kepler-dev breaks down roughly:<br />1) "anon" authentication via kepler gui: ~6s<br />2) query to fetch all data needed to build workflowRuns: ~4s<br />3) locally building runs, writing to provenance and tracker file: ~17s</p>
<p>For a subsequent Refresh action:<br />1) ~0s (already authenticated)<br />2) ~3s<br />3) ~0s</p>
<p>So I'll look into improving <a class="issue tracker-1 status-3 priority-2 priority-default closed" title="Bug: some required elements should be optional (Resolved)" href="https://projects.ecoinformatics.org/ecoinfo/issues/3">#3</a>, the big cost must be the write actions.</p> Kepler - Bug #5297: Workflow Run Manager - no indication of progress during preview runs downloadhttps://projects.ecoinformatics.org/ecoinfo/issues/5297?journal_id=183502011-02-09T00:20:02ZDerik Barseghianbarseghian@nceas.ucsb.edu
<ul></ul><p>"fixed" at r27053.<br />Step <a class="issue tracker-1 status-3 priority-2 priority-default closed" title="Bug: some required elements should be optional (Resolved)" href="https://projects.ecoinformatics.org/ecoinfo/issues/3">#3</a> now takes about 2s instead of 17. <br />Refactoring to show the preview runs as they're imported would take a fair bit of work, especially to avoid putting us right back at 17s+, and at 2s wait it's not worth it.</p> Kepler - Bug #5297: Workflow Run Manager - no indication of progress during preview runs downloadhttps://projects.ecoinformatics.org/ecoinfo/issues/5297?journal_id=183512013-03-27T21:29:55ZRedmine Admin
<ul></ul><p>Original Bugzilla ID was 5297</p>