Bug #5276: component search performance is poor - Kepler - Ecoinformatics Redmine

Custom queries

Tasks-hierarchical
Tickets grouped by Target version

Actions

Copy link

Bug #5276

open

component search performance is poor

Added by Derik Barseghian about 13 years ago. Updated over 8 years ago.

Status:

New

Priority:

Normal

Assignee:

Jing Tao

Category:

general

Target version:

2.X.Y

Start date:

01/24/2011

Due date:

% Done:

Estimated time:

Bugzilla-Id:

5276

Description

Searching for remote KARs through the Components pane is very slow -- search downloads and deals with karXML, each generally about twice as big as the kar it represents. During search the GUI is locked, and there is no progress indicator letting you know search is happening.

The query and related code probably need to be refactored to only download and utilize the bare minimum to show the results. This refactoring may necessitate some changes to what actions are possible on items in the results tree.

Jing and Matt will discuss this bug.

Related issues

Is duplicate of Kepler - Bug #5229: Poor component search performance

Resolved

Chad Berkley

10/28/2010

Actions

Blocked by Kepler - Bug #4981: RIO pdfs don't show up in the Component Library

In Progress

Derik Barseghian

05/06/2010

Actions

Issue # Delay: days Cancel

History
Notes
Property changes

Actions

Copy link

Updated by Derik Barseghian about 13 years ago

Components search can also fail because of a java.lang.OutOfMemoryError. Here are two examples stacktraces

searching chico1 and keplerDev for "test" (after logging in as anonymous):

Exception in thread "AWT-EventQueue-0" java.lang.OutOfMemoryError: Java heap space
at org.apache.xml.utils.SuballocatedIntVector.setElementAt(SuballocatedIntVector.java:404)
at org.apache.xml.dtm.ref.dom2dtm.DOM2DTM.addNode(DOM2DTM.java:246)
at org.apache.xml.dtm.ref.dom2dtm.DOM2DTM.nextNode(DOM2DTM.java:520)
at org.apache.xml.dtm.ref.DTMDefaultBase._nextsib(DTMDefaultBase.java:563)
at org.apache.xml.dtm.ref.DTMDefaultBase.getNextSibling(DTMDefaultBase.java:1140)
at org.apache.xml.dtm.ref.DTMDefaultBaseTraversers$ChildTraverser.next(DTMDefaultBaseTraversers.java:461)
at org.apache.xpath.axes.AxesWalker.getNextNode(AxesWalker.java:333)
at org.apache.xpath.axes.AxesWalker.nextNode(AxesWalker.java:361)
at org.apache.xpath.axes.WalkingIterator.nextNode(WalkingIterator.java:192)
at org.apache.xpath.axes.NodeSequence.nextNode(NodeSequence.java:281)
at org.apache.xpath.axes.NodeSequence.runTo(NodeSequence.java:435)
at org.apache.xml.dtm.ref.DTMNodeList.<init>(DTMNodeList.java:79)
at org.apache.xpath.objects.XNodeSet.nodelist(XNodeSet.java:339)
at org.apache.xpath.jaxp.XPathExpressionImpl.getResultAsType(XPathExpressionImpl.java:351)
at org.apache.xpath.jaxp.XPathExpressionImpl.eval(XPathExpressionImpl.java:98)
at org.apache.xpath.jaxp.XPathExpressionImpl.evaluate(XPathExpressionImpl.java:178)
at org.kepler.kar.karxml.KarXml.parse(KarXml.java:272)
at org.kepler.kar.karxml.KarXml.of(KarXml.java:183)
at org.kepler.objectmanager.repository.EcogridRepositoryResults.parseKarXml(EcogridRepositoryResults.java:97)
at org.kepler.objectmanager.repository.EcogridRepository.search(EcogridRepository.java:169)
at org.kepler.objectmanager.repository.EcogridRepositoryLibrarySearcher.search(EcogridRepositoryLibrarySearcher.java:230)
at org.kepler.gui.SimpleLibrarySearcher.search(SimpleLibrarySearcher.java:200)
at org.kepler.gui.ComponentLibraryTab$SearchButtonActionHandler.actionPerformed(ComponentLibraryTab.java:435)
at org.kepler.gui.SimpleSearchUIPane$1.actionPerformed(SimpleSearchUIPane.java:91)
at javax.swing.JTextField.fireActionPerformed(JTextField.java:492)
at javax.swing.JTextField.postActionEvent(JTextField.java:705)
at javax.swing.JTextField$NotifyAction.actionPerformed(JTextField.java:820)
at javax.swing.SwingUtilities.notifyAction(SwingUtilities.java:1639)
at javax.swing.JComponent.processKeyBinding(JComponent.java:2851)
at javax.swing.JComponent.processKeyBindings(JComponent.java:2886)
at javax.swing.JComponent.processKeyEvent(JComponent.java:2814)
at java.awt.Component.processEvent(Component.java:6129)

Searching keper-dev for "test" (after logging in as anonymous):

Exception in thread "AWT-EventQueue-0" java.lang.OutOfMemoryError: Java heap space
at org.apache.xerces.xni.XMLString.toString(Unknown Source)
at org.apache.xerces.parsers.AbstractDOMParser.characters(Unknown Source)
at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanContent(Unknown Source)
at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown Source)
at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
at org.apache.xerces.parsers.DOMParser.parse(Unknown Source)
at org.apache.xerces.jaxp.DocumentBuilderImpl.parse(Unknown Source)
at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:124)
at org.kepler.kar.karxml.KarXml.parseXmlStream(KarXml.java:351)
at org.kepler.kar.karxml.KarXml.of(KarXml.java:171)
at org.kepler.objectmanager.repository.EcogridRepositoryResults.parseKarXml(EcogridRepositoryResults.java:97)
at org.kepler.objectmanager.repository.EcogridRepository.search(EcogridRepository.java:169)
at org.kepler.objectmanager.repository.EcogridRepositoryLibrarySearcher.search(EcogridRepositoryLibrarySearcher.java:230)
at org.kepler.gui.SimpleLibrarySearcher.search(SimpleLibrarySearcher.java:200)
at org.kepler.gui.ComponentLibraryTab$SearchButtonActionHandler.actionPerformed(ComponentLibraryTab.java:435)
at org.kepler.gui.SimpleSearchUIPane$1.actionPerformed(SimpleSearchUIPane.java:91)
at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:2028)
at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2351)
at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:387)
at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:242)
at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:236)
at java.awt.Component.processMouseEvent(Component.java:6352)
at javax.swing.JComponent.processMouseEvent(JComponent.java:3267)
at java.awt.Component.processEvent(Component.java:6117)
at java.awt.Container.processEvent(Container.java:2085)
at java.awt.Component.dispatchEventImpl(Component.java:4714)
at java.awt.Container.dispatchEventImpl(Container.java:2143)
at java.awt.Component.dispatchEvent(Component.java:4544)
at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4621)
Exception in thread "HSQLDB Connection @6b0ede6" java.lang.NullPointerException
at org.hsqldb.ServerConnection.close(Unknown Source)
at org.hsqldb.ServerConnection.run(Unknown Source)
at java.lang.Thread.run(Thread.java:680)
Exception in thread "HSQLDB Connection @19e06de8" java.lang.NullPointerException
at org.hsqldb.ServerConnection.close(Unknown Source)
at org.hsqldb.ServerConnection.run(Unknown Source)
at java.lang.Thread.run(Thread.java:680)
Exception in thread "HSQLDB Connection @1b332109" java.lang.NullPointerException
at org.hsqldb.ServerConnection.close(Unknown Source)
at org.hsqldb.ServerConnection.run(Unknown Source)
at java.lang.Thread.run(Thread.java:680)
Exception in thread "HSQLDB Connection @42101da9" java.lang.NullPointerException
at org.hsqldb.ServerConnection.close(Unknown Source)
at org.hsqldb.ServerConnection.run(Unknown Source)
at java.lang.Thread.run(Thread.java:680)
Exception in thread "HSQLDB Connection @3b46ad8b" java.lang.NullPointerException
at org.hsqldb.ServerConnection.close(Unknown Source)
at org.hsqldb.ServerConnection.run(Unknown Source)
at java.lang.Thread.run(Thread.java:680)
Exception in thread "HSQLDB Connection @4c4936f3" java.lang.NullPointerException
at org.hsqldb.ServerConnection.close(Unknown Source)
at org.hsqldb.ServerConnection.run(Unknown Source)
at java.lang.Thread.run(Thread.java:680)

Actions

Copy link

Updated by Jing Tao about 13 years ago

<resultset>
<pagestart>0</pagestart>
<pagesize>0</pagesize>
<nextpage>1</nextpage>
<previouspage>-1</previouspage>
<query><pathquery version="1.2"><meta_file_id>test.1.1</meta_file_id><querytitle>kepler query</querytitle><returndoctype>http://www.kepler-project.org/kar-2.1.0</returndoctype><returndoctype>http://www.kepler-project.org/kar-2.0.0</returndoctype><returnfield>kar/karEntry/karEntryAttributes/Name</returnfield><returnfield>kar/karEntry/karEntryAttributes/type</returnfield><returnfield>kar/karEntry/karEntryAttributes/handler</returnfield><querygroup operator="INTERSECT"><queryterm casesensitive="false" searchmode="contains"><value>%test%</value></queryterm></querygroup></pathquery></query> <document><docid>878.72.1</docid><docname>kar</docname><doctype>http://www.kepler-project.org/kar-2.0.0</doctype><createdate>2010-05-13</createdate><updatedate>2010-05-13</updatedate><param name="kar/karEntry/karEntryAttributes/Name">

tpc09-plant-dynamics-woody.urn.lsid.gamma.msi.ucsb.edu.OpenAuth..8016.19.1.xml
            &lt;/param&gt;&lt;param name="kar/karEntry/karEntryAttributes/handler"&gt;
                org.kepler.kar.handlers.ActorMetadataKAREntryHandler
            &lt;/param&gt;&lt;param name="kar/karEntry/karEntryAttributes/type"&gt;
                org.kepler.moml.CompositeClassEntity
            &lt;/param&gt;&lt;param name="kar/karEntry/karEntryAttributes/Name"&gt;
                tpc09-plant-dynamics-woody_ROML.xml
            &lt;/param&gt;&lt;param name="kar/karEntry/karEntryAttributes/handler"&gt;
                org.kepler.kar.handlers.ReportLayoutKAREntryHandler
            &lt;/param&gt;&lt;param name="kar/karEntry/karEntryAttributes/type"&gt;
                org.kepler.reporting.roml.ReportLayout
            &lt;/param&gt;&lt;/document&gt;
&lt;/resutset&gt;

Actions

Copy link

Updated by Derik Barseghian about 13 years ago

just reformatting jing's above example as we discuss it:

<document>
<docid>878.72.1</docid>
<docname>kar</docname>
<doctype>http://www.kepler-project.org/kar-2.0.0</doctype>
<createdate>2010-05-13</createdate>
<updatedate>2010-05-13</updatedate>
<param name="kar/karEntry/karEntryAttributes/Name">tpc09-plant-dynamics
woody.urn.lsid.gamma.msi.ucsb.edu.OpenAuth..8016.19.1.xml
</param>
<param name="kar/karEntry/karEntryAttributes/handler">
org.kepler.kar.handlers.ActorMetadataKAREntryHandler
</param>
<param name="kar/karEntry/karEntryAttributes/type">
org.kepler.moml.CompositeClassEntity
</param>
<param name="kar/karEntry/karEntryAttributes/Name">
tpc09-plant-dynamics-woody_ROML.xml
</param>
<param name="kar/karEntry/karEntryAttributes/handler">
org.kepler.kar.handlers.ReportLayoutKAREntryHandler
</param>
<param name="kar/karEntry/karEntryAttributes/type">
org.kepler.reporting.roml.ReportLayout
</param>
</document>
</resutset>

Actions

Copy link

Updated by Derik Barseghian about 13 years ago

Discussed this with Jing. We think the schema is too loose right now, and we should change it so that we can rely on every karEntry having a name, type, and handler. We need to verify that existing karEntries all have these fields, but we're pretty sure this is true.
The schedule currently uses the karXml to determine if a moml has a director (for telling if it's an actor or a regular workflow). This may no longer be possible to do without having the karXml, but this may be an acceptable loss for speeding up search.
Additional features currently offered by the components search (e.g. drag and drop of workflow contained in kar, etc) will have to be looked at in detail, search may have to lose a few features in order to increase performance.

Actions

Copy link