Project

General

Profile

Actions
Copy link

Bug #5276

open

component search performance is poor

Added by Derik Barseghian almost 14 years ago. Updated over 9 years ago.

Status:
New
Priority:
Normal
Assignee:
Jing Tao
Category:
general
Target version:
2.X.Y
Start date:
01/24/2011
Due date:
% Done:

0%

Estimated time:
Bugzilla-Id:
5276

Description

Searching for remote KARs through the Components pane is very slow -- search downloads and deals with karXML, each generally about twice as big as the kar it represents. During search the GUI is locked, and there is no progress indicator letting you know search is happening.

The query and related code probably need to be refactored to only download and utilize the bare minimum to show the results. This refactoring may necessitate some changes to what actions are possible on items in the results tree.

Jing and Matt will discuss this bug.

Related issues

Is duplicate of Kepler - Bug #5229: Poor component search performanceResolvedChad Berkley10/28/2010

Actions
Blocked by Kepler - Bug #4981: RIO pdfs don't show up in the Component LibraryIn ProgressDerik Barseghian05/06/2010

Actions
Actions
Copy link
 #1

Updated by Derik Barseghian almost 14 years ago

Components search can also fail because of a java.lang.OutOfMemoryError. Here are two examples stacktraces

searching chico1 and keplerDev for "test" (after logging in as anonymous):

Exception in thread "AWT-EventQueue-0" java.lang.OutOfMemoryError: Java heap space
at org.apache.xml.utils.SuballocatedIntVector.setElementAt(SuballocatedIntVector.java:404)
at org.apache.xml.dtm.ref.dom2dtm.DOM2DTM.addNode(DOM2DTM.java:246)
at org.apache.xml.dtm.ref.dom2dtm.DOM2DTM.nextNode(DOM2DTM.java:520)
at org.apache.xml.dtm.ref.DTMDefaultBase._nextsib(DTMDefaultBase.java:563)
at org.apache.xml.dtm.ref.DTMDefaultBase.getNextSibling(DTMDefaultBase.java:1140)
at org.apache.xml.dtm.ref.DTMDefaultBaseTraversers$ChildTraverser.next(DTMDefaultBaseTraversers.java:461)
at org.apache.xpath.axes.AxesWalker.getNextNode(AxesWalker.java:333)
at org.apache.xpath.axes.AxesWalker.nextNode(AxesWalker.java:361)
at org.apache.xpath.axes.WalkingIterator.nextNode(WalkingIterator.java:192)
at org.apache.xpath.axes.NodeSequence.nextNode(NodeSequence.java:281)
at org.apache.xpath.axes.NodeSequence.runTo(NodeSequence.java:435)
at org.apache.xml.dtm.ref.DTMNodeList.<init>(DTMNodeList.java:79)
at org.apache.xpath.objects.XNodeSet.nodelist(XNodeSet.java:339)
at org.apache.xpath.jaxp.XPathExpressionImpl.getResultAsType(XPathExpressionImpl.java:351)
at org.apache.xpath.jaxp.XPathExpressionImpl.eval(XPathExpressionImpl.java:98)
at org.apache.xpath.jaxp.XPathExpressionImpl.evaluate(XPathExpressionImpl.java:178)
at org.kepler.kar.karxml.KarXml.parse(KarXml.java:272)
at org.kepler.kar.karxml.KarXml.of(KarXml.java:183)
at org.kepler.objectmanager.repository.EcogridRepositoryResults.parseKarXml(EcogridRepositoryResults.java:97)
at org.kepler.objectmanager.repository.EcogridRepository.search(EcogridRepository.java:169)
at org.kepler.objectmanager.repository.EcogridRepositoryLibrarySearcher.search(EcogridRepositoryLibrarySearcher.java:230)
at org.kepler.gui.SimpleLibrarySearcher.search(SimpleLibrarySearcher.java:200)
at org.kepler.gui.ComponentLibraryTab$SearchButtonActionHandler.actionPerformed(ComponentLibraryTab.java:435)
at org.kepler.gui.SimpleSearchUIPane$1.actionPerformed(SimpleSearchUIPane.java:91)
at javax.swing.JTextField.fireActionPerformed(JTextField.java:492)
at javax.swing.JTextField.postActionEvent(JTextField.java:705)
at javax.swing.JTextField$NotifyAction.actionPerformed(JTextField.java:820)
at javax.swing.SwingUtilities.notifyAction(SwingUtilities.java:1639)
at javax.swing.JComponent.processKeyBinding(JComponent.java:2851)
at javax.swing.JComponent.processKeyBindings(JComponent.java:2886)
at javax.swing.JComponent.processKeyEvent(JComponent.java:2814)
at java.awt.Component.processEvent(Component.java:6129)

Searching keper-dev for "test" (after logging in as anonymous):

Exception in thread "AWT-EventQueue-0" java.lang.OutOfMemoryError: Java heap space
at org.apache.xerces.xni.XMLString.toString(Unknown Source)
at org.apache.xerces.parsers.AbstractDOMParser.characters(Unknown Source)
at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanContent(Unknown Source)
at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown Source)
at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
at org.apache.xerces.parsers.DOMParser.parse(Unknown Source)
at org.apache.xerces.jaxp.DocumentBuilderImpl.parse(Unknown Source)
at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:124)
at org.kepler.kar.karxml.KarXml.parseXmlStream(KarXml.java:351)
at org.kepler.kar.karxml.KarXml.of(KarXml.java:171)
at org.kepler.objectmanager.repository.EcogridRepositoryResults.parseKarXml(EcogridRepositoryResults.java:97)
at org.kepler.objectmanager.repository.EcogridRepository.search(EcogridRepository.java:169)
at org.kepler.objectmanager.repository.EcogridRepositoryLibrarySearcher.search(EcogridRepositoryLibrarySearcher.java:230)
at org.kepler.gui.SimpleLibrarySearcher.search(SimpleLibrarySearcher.java:200)
at org.kepler.gui.ComponentLibraryTab$SearchButtonActionHandler.actionPerformed(ComponentLibraryTab.java:435)
at org.kepler.gui.SimpleSearchUIPane$1.actionPerformed(SimpleSearchUIPane.java:91)
at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:2028)
at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2351)
at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:387)
at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:242)
at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:236)
at java.awt.Component.processMouseEvent(Component.java:6352)
at javax.swing.JComponent.processMouseEvent(JComponent.java:3267)
at java.awt.Component.processEvent(Component.java:6117)
at java.awt.Container.processEvent(Container.java:2085)
at java.awt.Component.dispatchEventImpl(Component.java:4714)
at java.awt.Container.dispatchEventImpl(Container.java:2143)
at java.awt.Component.dispatchEvent(Component.java:4544)
at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4621)
Exception in thread "HSQLDB Connection @6b0ede6" java.lang.NullPointerException
at org.hsqldb.ServerConnection.close(Unknown Source)
at org.hsqldb.ServerConnection.run(Unknown Source)
at java.lang.Thread.run(Thread.java:680)
Exception in thread "HSQLDB Connection @19e06de8" java.lang.NullPointerException
at org.hsqldb.ServerConnection.close(Unknown Source)
at org.hsqldb.ServerConnection.run(Unknown Source)
at java.lang.Thread.run(Thread.java:680)
Exception in thread "HSQLDB Connection @1b332109" java.lang.NullPointerException
at org.hsqldb.ServerConnection.close(Unknown Source)
at org.hsqldb.ServerConnection.run(Unknown Source)
at java.lang.Thread.run(Thread.java:680)
Exception in thread "HSQLDB Connection @42101da9" java.lang.NullPointerException
at org.hsqldb.ServerConnection.close(Unknown Source)
at org.hsqldb.ServerConnection.run(Unknown Source)
at java.lang.Thread.run(Thread.java:680)
Exception in thread "HSQLDB Connection @3b46ad8b" java.lang.NullPointerException
at org.hsqldb.ServerConnection.close(Unknown Source)
at org.hsqldb.ServerConnection.run(Unknown Source)
at java.lang.Thread.run(Thread.java:680)
Exception in thread "HSQLDB Connection @4c4936f3" java.lang.NullPointerException
at org.hsqldb.ServerConnection.close(Unknown Source)
at org.hsqldb.ServerConnection.run(Unknown Source)
at java.lang.Thread.run(Thread.java:680)

Actions
Copy link
 #3

Updated by Jing Tao almost 14 years ago


<resultset>
<pagestart>0</pagestart>
<pagesize>0</pagesize>
<nextpage>1</nextpage>
<previouspage>-1</previouspage>
<query><pathquery version="1.2"><meta_file_id>test.1.1</meta_file_id><querytitle>kepler query</querytitle><returndoctype>http://www.kepler-project.org/kar-2.1.0&lt;/returndoctype&gt;&lt;returndoctype&gt;http://www.kepler-project.org/kar-2.0.0&lt;/returndoctype&gt;&lt;returnfield&gt;kar/karEntry/karEntryAttributes/Name&lt;/returnfield&gt;&lt;returnfield&gt;kar/karEntry/karEntryAttributes/type&lt;/returnfield&gt;&lt;returnfield&gt;kar/karEntry/karEntryAttributes/handler&lt;/returnfield&gt;&lt;querygroup operator="INTERSECT"><queryterm casesensitive="false" searchmode="contains"><value>%test%</value></queryterm></querygroup></pathquery></query> <document><docid>878.72.1</docid><docname>kar</docname><doctype>http://www.kepler-project.org/kar-2.0.0&lt;/doctype&gt;&lt;createdate&gt;2010-05-13&lt;/createdate&gt;&lt;updatedate&gt;2010-05-13&lt;/updatedate&gt;&lt;param name="kar/karEntry/karEntryAttributes/Name">

tpc09-plant-dynamics-woody.urn.lsid.gamma.msi.ucsb.edu.OpenAuth..8016.19.1.xml
            &lt;/param&gt;&lt;param name="kar/karEntry/karEntryAttributes/handler"&gt;
                org.kepler.kar.handlers.ActorMetadataKAREntryHandler
            &lt;/param&gt;&lt;param name="kar/karEntry/karEntryAttributes/type"&gt;
                org.kepler.moml.CompositeClassEntity
            &lt;/param&gt;&lt;param name="kar/karEntry/karEntryAttributes/Name"&gt;
                tpc09-plant-dynamics-woody_ROML.xml
            &lt;/param&gt;&lt;param name="kar/karEntry/karEntryAttributes/handler"&gt;
                org.kepler.kar.handlers.ReportLayoutKAREntryHandler
            &lt;/param&gt;&lt;param name="kar/karEntry/karEntryAttributes/type"&gt;
                org.kepler.reporting.roml.ReportLayout
            &lt;/param&gt;&lt;/document&gt;
&lt;/resutset&gt;
Actions
Copy link
 #4

Updated by Derik Barseghian almost 14 years ago

just reformatting jing's above example as we discuss it:

<document>
<docid>878.72.1</docid>
<docname>kar</docname>
<doctype>http://www.kepler-project.org/kar-2.0.0&lt;/doctype>
<createdate>2010-05-13</createdate>
<updatedate>2010-05-13</updatedate>
<param name="kar/karEntry/karEntryAttributes/Name">tpc09-plant-dynamics
woody.urn.lsid.gamma.msi.ucsb.edu.OpenAuth..8016.19.1.xml
</param>
<param name="kar/karEntry/karEntryAttributes/handler">
org.kepler.kar.handlers.ActorMetadataKAREntryHandler
</param>
<param name="kar/karEntry/karEntryAttributes/type">
org.kepler.moml.CompositeClassEntity
</param>
<param name="kar/karEntry/karEntryAttributes/Name">
tpc09-plant-dynamics-woody_ROML.xml
</param>
<param name="kar/karEntry/karEntryAttributes/handler">
org.kepler.kar.handlers.ReportLayoutKAREntryHandler
</param>
<param name="kar/karEntry/karEntryAttributes/type">
org.kepler.reporting.roml.ReportLayout
</param>
</document>
</resutset>

Actions
Copy link
 #5

Updated by Derik Barseghian almost 14 years ago

Discussed this with Jing. We think the schema is too loose right now, and we should change it so that we can rely on every karEntry having a name, type, and handler. We need to verify that existing karEntries all have these fields, but we're pretty sure this is true.
The schedule currently uses the karXml to determine if a moml has a director (for telling if it's an actor or a regular workflow). This may no longer be possible to do without having the karXml, but this may be an acceptable loss for speeding up search.
Additional features currently offered by the components search (e.g. drag and drop of workflow contained in kar, etc) will have to be looked at in detail, search may have to lose a few features in order to increase performance.

Actions
Copy link
 #6

Updated by Redmine Admin over 11 years ago

Original Bugzilla ID was 5276

Actions
Copy link
 #7

Updated by Daniel Crawl over 9 years ago

  • Target version changed from 2.5.0 to 2.X.Y
Actions
Copy link

Also available in: Atom PDF