Project

General

Profile

Bug #5276

component search performance is poor

Added by Derik Barseghian over 8 years ago. Updated about 4 years ago.

Status:
New
Priority:
Normal
Assignee:
Category:
general
Target version:
Start date:
01/24/2011
Due date:
% Done:

0%

Estimated time:
Bugzilla-Id:
5276

Description

Searching for remote KARs through the Components pane is very slow -- search downloads and deals with karXML, each generally about twice as big as the kar it represents. During search the GUI is locked, and there is no progress indicator letting you know search is happening.

The query and related code probably need to be refactored to only download and utilize the bare minimum to show the results. This refactoring may necessitate some changes to what actions are possible on items in the results tree.

Jing and Matt will discuss this bug.


Related issues

Is duplicate of Kepler - Bug #5229: Poor component search performanceResolved10/28/2010

Blocked by Kepler - Bug #4981: RIO pdfs don't show up in the Component LibraryIn Progress05/06/2010

History

#1 Updated by Derik Barseghian over 8 years ago

Components search can also fail because of a java.lang.OutOfMemoryError. Here are two examples stacktraces

searching chico1 and keplerDev for "test" (after logging in as anonymous):

Exception in thread "AWT-EventQueue-0" java.lang.OutOfMemoryError: Java heap space
at org.apache.xml.utils.SuballocatedIntVector.setElementAt(SuballocatedIntVector.java:404)
at org.apache.xml.dtm.ref.dom2dtm.DOM2DTM.addNode(DOM2DTM.java:246)
at org.apache.xml.dtm.ref.dom2dtm.DOM2DTM.nextNode(DOM2DTM.java:520)
at org.apache.xml.dtm.ref.DTMDefaultBase._nextsib(DTMDefaultBase.java:563)
at org.apache.xml.dtm.ref.DTMDefaultBase.getNextSibling(DTMDefaultBase.java:1140)
at org.apache.xml.dtm.ref.DTMDefaultBaseTraversers$ChildTraverser.next(DTMDefaultBaseTraversers.java:461)
at org.apache.xpath.axes.AxesWalker.getNextNode(AxesWalker.java:333)
at org.apache.xpath.axes.AxesWalker.nextNode(AxesWalker.java:361)
at org.apache.xpath.axes.WalkingIterator.nextNode(WalkingIterator.java:192)
at org.apache.xpath.axes.NodeSequence.nextNode(NodeSequence.java:281)
at org.apache.xpath.axes.NodeSequence.runTo(NodeSequence.java:435)
at org.apache.xml.dtm.ref.DTMNodeList.<init>(DTMNodeList.java:79)
at org.apache.xpath.objects.XNodeSet.nodelist(XNodeSet.java:339)
at org.apache.xpath.jaxp.XPathExpressionImpl.getResultAsType(XPathExpressionImpl.java:351)
at org.apache.xpath.jaxp.XPathExpressionImpl.eval(XPathExpressionImpl.java:98)
at org.apache.xpath.jaxp.XPathExpressionImpl.evaluate(XPathExpressionImpl.java:178)
at org.kepler.kar.karxml.KarXml.parse(KarXml.java:272)
at org.kepler.kar.karxml.KarXml.of(KarXml.java:183)
at org.kepler.objectmanager.repository.EcogridRepositoryResults.parseKarXml(EcogridRepositoryResults.java:97)
at org.kepler.objectmanager.repository.EcogridRepository.search(EcogridRepository.java:169)
at org.kepler.objectmanager.repository.EcogridRepositoryLibrarySearcher.search(EcogridRepositoryLibrarySearcher.java:230)
at org.kepler.gui.SimpleLibrarySearcher.search(SimpleLibrarySearcher.java:200)
at org.kepler.gui.ComponentLibraryTab$SearchButtonActionHandler.actionPerformed(ComponentLibraryTab.java:435)
at org.kepler.gui.SimpleSearchUIPane$1.actionPerformed(SimpleSearchUIPane.java:91)
at javax.swing.JTextField.fireActionPerformed(JTextField.java:492)
at javax.swing.JTextField.postActionEvent(JTextField.java:705)
at javax.swing.JTextField$NotifyAction.actionPerformed(JTextField.java:820)
at javax.swing.SwingUtilities.notifyAction(SwingUtilities.java:1639)
at javax.swing.JComponent.processKeyBinding(JComponent.java:2851)
at javax.swing.JComponent.processKeyBindings(JComponent.java:2886)
at javax.swing.JComponent.processKeyEvent(JComponent.java:2814)
at java.awt.Component.processEvent(Component.java:6129)

Searching keper-dev for "test" (after logging in as anonymous):

Exception in thread "AWT-EventQueue-0" java.lang.OutOfMemoryError: Java heap space
at org.apache.xerces.xni.XMLString.toString(Unknown Source)
at org.apache.xerces.parsers.AbstractDOMParser.characters(Unknown Source)
at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanContent(Unknown Source)
at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown Source)
at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
at org.apache.xerces.parsers.DOMParser.parse(Unknown Source)
at org.apache.xerces.jaxp.DocumentBuilderImpl.parse(Unknown Source)
at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:124)
at org.kepler.kar.karxml.KarXml.parseXmlStream(KarXml.java:351)
at org.kepler.kar.karxml.KarXml.of(KarXml.java:171)
at org.kepler.objectmanager.repository.EcogridRepositoryResults.parseKarXml(EcogridRepositoryResults.java:97)
at org.kepler.objectmanager.repository.EcogridRepository.search(EcogridRepository.java:169)
at org.kepler.objectmanager.repository.EcogridRepositoryLibrarySearcher.search(EcogridRepositoryLibrarySearcher.java:230)
at org.kepler.gui.SimpleLibrarySearcher.search(SimpleLibrarySearcher.java:200)
at org.kepler.gui.ComponentLibraryTab$SearchButtonActionHandler.actionPerformed(ComponentLibraryTab.java:435)
at org.kepler.gui.SimpleSearchUIPane$1.actionPerformed(SimpleSearchUIPane.java:91)
at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:2028)
at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2351)
at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:387)
at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:242)
at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:236)
at java.awt.Component.processMouseEvent(Component.java:6352)
at javax.swing.JComponent.processMouseEvent(JComponent.java:3267)
at java.awt.Component.processEvent(Component.java:6117)
at java.awt.Container.processEvent(Container.java:2085)
at java.awt.Component.dispatchEventImpl(Component.java:4714)
at java.awt.Container.dispatchEventImpl(Container.java:2143)
at java.awt.Component.dispatchEvent(Component.java:4544)
at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4621)
Exception in thread "HSQLDB Connection @6b0ede6" java.lang.NullPointerException
at org.hsqldb.ServerConnection.close(Unknown Source)
at org.hsqldb.ServerConnection.run(Unknown Source)
at java.lang.Thread.run(Thread.java:680)
Exception in thread "HSQLDB Connection @19e06de8" java.lang.NullPointerException
at org.hsqldb.ServerConnection.close(Unknown Source)
at org.hsqldb.ServerConnection.run(Unknown Source)
at java.lang.Thread.run(Thread.java:680)
Exception in thread "HSQLDB Connection @1b332109" java.lang.NullPointerException
at org.hsqldb.ServerConnection.close(Unknown Source)
at org.hsqldb.ServerConnection.run(Unknown Source)
at java.lang.Thread.run(Thread.java:680)
Exception in thread "HSQLDB Connection @42101da9" java.lang.NullPointerException
at org.hsqldb.ServerConnection.close(Unknown Source)
at org.hsqldb.ServerConnection.run(Unknown Source)
at java.lang.Thread.run(Thread.java:680)
Exception in thread "HSQLDB Connection @3b46ad8b" java.lang.NullPointerException
at org.hsqldb.ServerConnection.close(Unknown Source)
at org.hsqldb.ServerConnection.run(Unknown Source)
at java.lang.Thread.run(Thread.java:680)
Exception in thread "HSQLDB Connection @4c4936f3" java.lang.NullPointerException
at org.hsqldb.ServerConnection.close(Unknown Source)
at org.hsqldb.ServerConnection.run(Unknown Source)
at java.lang.Thread.run(Thread.java:680)

#3 Updated by Jing Tao over 8 years ago


<resultset>
<pagestart>0</pagestart>
<pagesize>0</pagesize>
<nextpage>1</nextpage>
<previouspage>-1</previouspage>
<query><pathquery version="1.2"><meta_file_id>test.1.1</meta_file_id><querytitle>kepler query</querytitle><returndoctype>http://www.kepler-project.org/kar-2.1.0&lt;/returndoctype&gt;&lt;returndoctype&gt;http://www.kepler-project.org/kar-2.0.0&lt;/returndoctype&gt;&lt;returnfield&gt;kar/karEntry/karEntryAttributes/Name&lt;/returnfield&gt;&lt;returnfield&gt;kar/karEntry/karEntryAttributes/type&lt;/returnfield&gt;&lt;returnfield&gt;kar/karEntry/karEntryAttributes/handler&lt;/returnfield&gt;&lt;querygroup operator="INTERSECT"><queryterm casesensitive="false" searchmode="contains"><value>%test%</value></queryterm></querygroup></pathquery></query> <document><docid>878.72.1</docid><docname>kar</docname><doctype>http://www.kepler-project.org/kar-2.0.0&lt;/doctype&gt;&lt;createdate&gt;2010-05-13&lt;/createdate&gt;&lt;updatedate&gt;2010-05-13&lt;/updatedate&gt;&lt;param name="kar/karEntry/karEntryAttributes/Name">

tpc09-plant-dynamics-woody.urn.lsid.gamma.msi.ucsb.edu.OpenAuth..8016.19.1.xml
&lt;/param&gt;&lt;param name="kar/karEntry/karEntryAttributes/handler"&gt;
org.kepler.kar.handlers.ActorMetadataKAREntryHandler
&lt;/param&gt;&lt;param name="kar/karEntry/karEntryAttributes/type"&gt;
org.kepler.moml.CompositeClassEntity
&lt;/param&gt;&lt;param name="kar/karEntry/karEntryAttributes/Name"&gt;
tpc09-plant-dynamics-woody_ROML.xml
&lt;/param&gt;&lt;param name="kar/karEntry/karEntryAttributes/handler"&gt;
org.kepler.kar.handlers.ReportLayoutKAREntryHandler
&lt;/param&gt;&lt;param name="kar/karEntry/karEntryAttributes/type"&gt;
org.kepler.reporting.roml.ReportLayout
&lt;/param&gt;&lt;/document&gt;
&lt;/resutset&gt;

#4 Updated by Derik Barseghian over 8 years ago

just reformatting jing's above example as we discuss it:

<document>
<docid>878.72.1</docid>
<docname>kar</docname>
<doctype>http://www.kepler-project.org/kar-2.0.0&lt;/doctype>
<createdate>2010-05-13</createdate>
<updatedate>2010-05-13</updatedate>
<param name="kar/karEntry/karEntryAttributes/Name">tpc09-plant-dynamics
woody.urn.lsid.gamma.msi.ucsb.edu.OpenAuth..8016.19.1.xml
</param>
<param name="kar/karEntry/karEntryAttributes/handler">
org.kepler.kar.handlers.ActorMetadataKAREntryHandler
</param>
<param name="kar/karEntry/karEntryAttributes/type">
org.kepler.moml.CompositeClassEntity
</param>
<param name="kar/karEntry/karEntryAttributes/Name">
tpc09-plant-dynamics-woody_ROML.xml
</param>
<param name="kar/karEntry/karEntryAttributes/handler">
org.kepler.kar.handlers.ReportLayoutKAREntryHandler
</param>
<param name="kar/karEntry/karEntryAttributes/type">
org.kepler.reporting.roml.ReportLayout
</param>
</document>
</resutset>

#5 Updated by Derik Barseghian over 8 years ago

Discussed this with Jing. We think the schema is too loose right now, and we should change it so that we can rely on every karEntry having a name, type, and handler. We need to verify that existing karEntries all have these fields, but we're pretty sure this is true.
The schedule currently uses the karXml to determine if a moml has a director (for telling if it's an actor or a regular workflow). This may no longer be possible to do without having the karXml, but this may be an acceptable loss for speeding up search.
Additional features currently offered by the components search (e.g. drag and drop of workflow contained in kar, etc) will have to be looked at in detail, search may have to lose a few features in order to increase performance.

#6 Updated by Redmine Admin over 6 years ago

Original Bugzilla ID was 5276

#7 Updated by Daniel Crawl about 4 years ago

  • Target version changed from 2.5.0 to 2.X.Y

Also available in: Atom PDF