Bug #5276
opencomponent search performance is poor
Added by Derik Barseghian almost 14 years ago. Updated over 9 years ago.
0%
Description
Searching for remote KARs through the Components pane is very slow -- search downloads and deals with karXML, each generally about twice as big as the kar it represents. During search the GUI is locked, and there is no progress indicator letting you know search is happening.
The query and related code probably need to be refactored to only download and utilize the bare minimum to show the results. This refactoring may necessitate some changes to what actions are possible on items in the results tree.
Jing and Matt will discuss this bug.
Related issues
Updated by Derik Barseghian almost 14 years ago
Components search can also fail because of a java.lang.OutOfMemoryError. Here are two examples stacktraces
searching chico1 and keplerDev for "test" (after logging in as anonymous):
Exception in thread "AWT-EventQueue-0" java.lang.OutOfMemoryError: Java heap space
at org.apache.xml.utils.SuballocatedIntVector.setElementAt(SuballocatedIntVector.java:404)
at org.apache.xml.dtm.ref.dom2dtm.DOM2DTM.addNode(DOM2DTM.java:246)
at org.apache.xml.dtm.ref.dom2dtm.DOM2DTM.nextNode(DOM2DTM.java:520)
at org.apache.xml.dtm.ref.DTMDefaultBase._nextsib(DTMDefaultBase.java:563)
at org.apache.xml.dtm.ref.DTMDefaultBase.getNextSibling(DTMDefaultBase.java:1140)
at org.apache.xml.dtm.ref.DTMDefaultBaseTraversers$ChildTraverser.next(DTMDefaultBaseTraversers.java:461)
at org.apache.xpath.axes.AxesWalker.getNextNode(AxesWalker.java:333)
at org.apache.xpath.axes.AxesWalker.nextNode(AxesWalker.java:361)
at org.apache.xpath.axes.WalkingIterator.nextNode(WalkingIterator.java:192)
at org.apache.xpath.axes.NodeSequence.nextNode(NodeSequence.java:281)
at org.apache.xpath.axes.NodeSequence.runTo(NodeSequence.java:435)
at org.apache.xml.dtm.ref.DTMNodeList.<init>(DTMNodeList.java:79)
at org.apache.xpath.objects.XNodeSet.nodelist(XNodeSet.java:339)
at org.apache.xpath.jaxp.XPathExpressionImpl.getResultAsType(XPathExpressionImpl.java:351)
at org.apache.xpath.jaxp.XPathExpressionImpl.eval(XPathExpressionImpl.java:98)
at org.apache.xpath.jaxp.XPathExpressionImpl.evaluate(XPathExpressionImpl.java:178)
at org.kepler.kar.karxml.KarXml.parse(KarXml.java:272)
at org.kepler.kar.karxml.KarXml.of(KarXml.java:183)
at org.kepler.objectmanager.repository.EcogridRepositoryResults.parseKarXml(EcogridRepositoryResults.java:97)
at org.kepler.objectmanager.repository.EcogridRepository.search(EcogridRepository.java:169)
at org.kepler.objectmanager.repository.EcogridRepositoryLibrarySearcher.search(EcogridRepositoryLibrarySearcher.java:230)
at org.kepler.gui.SimpleLibrarySearcher.search(SimpleLibrarySearcher.java:200)
at org.kepler.gui.ComponentLibraryTab$SearchButtonActionHandler.actionPerformed(ComponentLibraryTab.java:435)
at org.kepler.gui.SimpleSearchUIPane$1.actionPerformed(SimpleSearchUIPane.java:91)
at javax.swing.JTextField.fireActionPerformed(JTextField.java:492)
at javax.swing.JTextField.postActionEvent(JTextField.java:705)
at javax.swing.JTextField$NotifyAction.actionPerformed(JTextField.java:820)
at javax.swing.SwingUtilities.notifyAction(SwingUtilities.java:1639)
at javax.swing.JComponent.processKeyBinding(JComponent.java:2851)
at javax.swing.JComponent.processKeyBindings(JComponent.java:2886)
at javax.swing.JComponent.processKeyEvent(JComponent.java:2814)
at java.awt.Component.processEvent(Component.java:6129)
Searching keper-dev for "test" (after logging in as anonymous):
Exception in thread "AWT-EventQueue-0" java.lang.OutOfMemoryError: Java heap space
at org.apache.xerces.xni.XMLString.toString(Unknown Source)
at org.apache.xerces.parsers.AbstractDOMParser.characters(Unknown Source)
at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanContent(Unknown Source)
at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown Source)
at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
at org.apache.xerces.parsers.DOMParser.parse(Unknown Source)
at org.apache.xerces.jaxp.DocumentBuilderImpl.parse(Unknown Source)
at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:124)
at org.kepler.kar.karxml.KarXml.parseXmlStream(KarXml.java:351)
at org.kepler.kar.karxml.KarXml.of(KarXml.java:171)
at org.kepler.objectmanager.repository.EcogridRepositoryResults.parseKarXml(EcogridRepositoryResults.java:97)
at org.kepler.objectmanager.repository.EcogridRepository.search(EcogridRepository.java:169)
at org.kepler.objectmanager.repository.EcogridRepositoryLibrarySearcher.search(EcogridRepositoryLibrarySearcher.java:230)
at org.kepler.gui.SimpleLibrarySearcher.search(SimpleLibrarySearcher.java:200)
at org.kepler.gui.ComponentLibraryTab$SearchButtonActionHandler.actionPerformed(ComponentLibraryTab.java:435)
at org.kepler.gui.SimpleSearchUIPane$1.actionPerformed(SimpleSearchUIPane.java:91)
at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:2028)
at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2351)
at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:387)
at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:242)
at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:236)
at java.awt.Component.processMouseEvent(Component.java:6352)
at javax.swing.JComponent.processMouseEvent(JComponent.java:3267)
at java.awt.Component.processEvent(Component.java:6117)
at java.awt.Container.processEvent(Container.java:2085)
at java.awt.Component.dispatchEventImpl(Component.java:4714)
at java.awt.Container.dispatchEventImpl(Container.java:2143)
at java.awt.Component.dispatchEvent(Component.java:4544)
at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4621)
Exception in thread "HSQLDB Connection @6b0ede6" java.lang.NullPointerException
at org.hsqldb.ServerConnection.close(Unknown Source)
at org.hsqldb.ServerConnection.run(Unknown Source)
at java.lang.Thread.run(Thread.java:680)
Exception in thread "HSQLDB Connection @19e06de8" java.lang.NullPointerException
at org.hsqldb.ServerConnection.close(Unknown Source)
at org.hsqldb.ServerConnection.run(Unknown Source)
at java.lang.Thread.run(Thread.java:680)
Exception in thread "HSQLDB Connection @1b332109" java.lang.NullPointerException
at org.hsqldb.ServerConnection.close(Unknown Source)
at org.hsqldb.ServerConnection.run(Unknown Source)
at java.lang.Thread.run(Thread.java:680)
Exception in thread "HSQLDB Connection @42101da9" java.lang.NullPointerException
at org.hsqldb.ServerConnection.close(Unknown Source)
at org.hsqldb.ServerConnection.run(Unknown Source)
at java.lang.Thread.run(Thread.java:680)
Exception in thread "HSQLDB Connection @3b46ad8b" java.lang.NullPointerException
at org.hsqldb.ServerConnection.close(Unknown Source)
at org.hsqldb.ServerConnection.run(Unknown Source)
at java.lang.Thread.run(Thread.java:680)
Exception in thread "HSQLDB Connection @4c4936f3" java.lang.NullPointerException
at org.hsqldb.ServerConnection.close(Unknown Source)
at org.hsqldb.ServerConnection.run(Unknown Source)
at java.lang.Thread.run(Thread.java:680)
Updated by Jing Tao almost 14 years ago
<resultset>
<pagestart>0</pagestart>
<pagesize>0</pagesize>
<nextpage>1</nextpage>
<previouspage>-1</previouspage>
<query><pathquery version="1.2"><meta_file_id>test.1.1</meta_file_id><querytitle>kepler query</querytitle><returndoctype>http://www.kepler-project.org/kar-2.1.0</returndoctype><returndoctype>http://www.kepler-project.org/kar-2.0.0</returndoctype><returnfield>kar/karEntry/karEntryAttributes/Name</returnfield><returnfield>kar/karEntry/karEntryAttributes/type</returnfield><returnfield>kar/karEntry/karEntryAttributes/handler</returnfield><querygroup operator="INTERSECT"><queryterm casesensitive="false" searchmode="contains"><value>%test%</value></queryterm></querygroup></pathquery></query> <document><docid>878.72.1</docid><docname>kar</docname><doctype>http://www.kepler-project.org/kar-2.0.0</doctype><createdate>2010-05-13</createdate><updatedate>2010-05-13</updatedate><param name="kar/karEntry/karEntryAttributes/Name">
tpc09-plant-dynamics-woody.urn.lsid.gamma.msi.ucsb.edu.OpenAuth..8016.19.1.xml
</param><param name="kar/karEntry/karEntryAttributes/handler">
org.kepler.kar.handlers.ActorMetadataKAREntryHandler
</param><param name="kar/karEntry/karEntryAttributes/type">
org.kepler.moml.CompositeClassEntity
</param><param name="kar/karEntry/karEntryAttributes/Name">
tpc09-plant-dynamics-woody_ROML.xml
</param><param name="kar/karEntry/karEntryAttributes/handler">
org.kepler.kar.handlers.ReportLayoutKAREntryHandler
</param><param name="kar/karEntry/karEntryAttributes/type">
org.kepler.reporting.roml.ReportLayout
</param></document>
</resutset>Updated by Derik Barseghian almost 14 years ago
just reformatting jing's above example as we discuss it:
<document>
<docid>878.72.1</docid>
<docname>kar</docname>
<doctype>http://www.kepler-project.org/kar-2.0.0</doctype>
<createdate>2010-05-13</createdate>
<updatedate>2010-05-13</updatedate>
<param name="kar/karEntry/karEntryAttributes/Name">tpc09-plant-dynamics
woody.urn.lsid.gamma.msi.ucsb.edu.OpenAuth..8016.19.1.xml
</param>
<param name="kar/karEntry/karEntryAttributes/handler">
org.kepler.kar.handlers.ActorMetadataKAREntryHandler
</param>
<param name="kar/karEntry/karEntryAttributes/type">
org.kepler.moml.CompositeClassEntity
</param>
<param name="kar/karEntry/karEntryAttributes/Name">
tpc09-plant-dynamics-woody_ROML.xml
</param>
<param name="kar/karEntry/karEntryAttributes/handler">
org.kepler.kar.handlers.ReportLayoutKAREntryHandler
</param>
<param name="kar/karEntry/karEntryAttributes/type">
org.kepler.reporting.roml.ReportLayout
</param>
</document>
</resutset>
Updated by Derik Barseghian almost 14 years ago
Discussed this with Jing. We think the schema is too loose right now, and we should change it so that we can rely on every karEntry having a name, type, and handler. We need to verify that existing karEntries all have these fields, but we're pretty sure this is true.
The schedule currently uses the karXml to determine if a moml has a director (for telling if it's an actor or a regular workflow). This may no longer be possible to do without having the karXml, but this may be an acceptable loss for speeding up search.
Additional features currently offered by the components search (e.g. drag and drop of workflow contained in kar, etc) will have to be looked at in detail, search may have to lose a few features in order to increase performance.
Updated by Daniel Crawl over 9 years ago
- Target version changed from 2.5.0 to 2.X.Y