https://projects.ecoinformatics.org/ecoinfo/https://projects.ecoinformatics.org/ecoinfo/ecoinfo/favicon.ico?14691340362009-04-06T19:27:33ZEcoinformatics RedmineKepler - Bug #3948: Create new configuration system supporting moduleshttps://projects.ecoinformatics.org/ecoinfo/issues/3948?journal_id=133052009-04-06T19:27:33ZMatt Jonesjones@nceas.ucsb.edu
<ul></ul><p>See notes on wiki about existing and proposed systems.</p>
<p><a class="external" href="https://kepler-project.org/developers/teams/framework/kepler-configuration">https://kepler-project.org/developers/teams/framework/kepler-configuration</a></p> Kepler - Bug #3948: Create new configuration system supporting moduleshttps://projects.ecoinformatics.org/ecoinfo/issues/3948?journal_id=133062009-04-21T00:19:35ZChristopher Brookscxh@eecs.berkeley.edu
<ul></ul><p>Last week, I spoke with Edward about runtime configuration and he <br />believes that it would be fairly easy to create an editor to develop<br />configurations.</p>
<p>For example, if you run<br />cd $PTII<br />$PTII/bin/vergil ptolemy/configs/full/configuration.xml</p>
<p>Then you can view the configuration. Using Graph -> Automatic layout<br />helps clean it up. However, currently, changes are not preserved<br />by saving. If we provide the user with an environment to create the<br />configuration, then the question of xml or not xml becomes moot.</p>
<p>We have funding to work on configurations, in particular to provide<br />an easy way for users to create custom configurations.</p>
<p>A few issues:<br />- There are plenty of things to configure: <br /> - menus<br /> - the right hand actor pane<br />- A large problem seems to be in overriding and removing features.<br /> For example, removing a menu choice is difficult<br />- The Vergil graph viewer should not necessarily expect to be a toplevel<br /> window. The issue is: what happens when we look inside or when we<br /> plot data? Right now, the position of the plots is encoded in the<br /> model. This would be an issue in an Eclipse View, were the plots<br /> should be inside an Eclipse view.</p>
<p>About internationalization, we should stick with the standards, see<br /><a class="external" href="http://java.sun.com/javase/technologies/core/basic/intl/">http://java.sun.com/javase/technologies/core/basic/intl/</a><br />and<br /><a class="external" href="http://java.sun.com/docs/books/tutorial/i18n/index.html">http://java.sun.com/docs/books/tutorial/i18n/index.html</a></p>
<p>The i18n tutorial suggests using ResourceBundles.</p> Kepler - Bug #3948: Create new configuration system supporting moduleshttps://projects.ecoinformatics.org/ecoinfo/issues/3948?journal_id=133072009-07-13T23:08:42ZChad Berkleyberkley@nceas.ucsb.edu
<ul></ul><ul>
<li>Bug 4088 has been marked as a duplicate of this bug. ***</li>
</ul> Kepler - Bug #3948: Create new configuration system supporting moduleshttps://projects.ecoinformatics.org/ecoinfo/issues/3948?journal_id=133082009-08-26T19:50:14ZMatt Jonesjones@nceas.ucsb.edu
<ul></ul><p>This bug now focuses exclusively on creating a new configuration system for<br />managing configuration properties. It does not include changing the portions<br />of kepler that are configurable, but rather only on the creation of a system<br />that allows modules to set system-wide and module-specific properties.</p>
<p>Bug 4336 was created for merging the existing (old) configuration files into<br />the new system.</p> Kepler - Bug #3948: Create new configuration system supporting moduleshttps://projects.ecoinformatics.org/ecoinfo/issues/3948?journal_id=133092009-09-11T17:27:48ZChad Berkleyberkley@nceas.ucsb.edu
<ul></ul><p>After looking at Matt's UML diagram (<a class="external" href="https://code.kepler-project.org/code/kepler-docs/trunk/teams-and-wg/2-infr-teams/framework/design/configuration/configuration-architecture.png">https://code.kepler-project.org/code/kepler-docs/trunk/teams-and-wg/2-infr-teams/framework/design/configuration/configuration-architecture.png</a>), I've been thinking about a couple ideas for serialization. The config system will need to serialize/deserialize to multiple different locations. I think this should use the build system's moduleTree functionality. The workflow would look something like this:</p>
<p>deserialize:<br />for each module in moduleTree
{<br /> get file module.config<br /> read module.config into common.config<br />}</p>
<p>serialize:<br />for each module.namespace in common.config
{<br /> find module.namespace in common.config<br /> write module.namespace to module.config<br />}</p>
<p>The moduleTree functionality is in the build system jar that is accessible from the kepler runtime.</p>
<p>Instead of using a String to represent a module in the ConfigurationProperty, we should probably use a Module object like the build system does. That way, the configuration system has all of the module's metadata that is read in at startup. This could be a useful way for other modules or the core to get information on modules at runtime.</p> Kepler - Bug #3948: Create new configuration system supporting moduleshttps://projects.ecoinformatics.org/ecoinfo/issues/3948?journal_id=133102009-10-05T18:55:58ZChad Berkleyberkley@nceas.ucsb.edu
<ul></ul><p>Configuration System Requirements agreed upon on 2009.10.02:</p>
<p>1) A module is able to add configuration properties to the configuration</p>
<p>2) A module is able to read its own configuration properties</p>
<p>3) A module is able to add or override configuration properties added by other modules.</p>
<p>4) A module is able to overwrite another module's configuration properties</p>
<p>5) A module can indicate whether a property is mutable (i.e., if property changes are noticed and incorporated without restarting the application). At runtime, only mutable properties can be overwritten. It is the responsibility of all modules to utilize mutable properties to monitor those properties and respond appropriately to changes.</p>
<p>6) A configuration property can have either or both of a single scalar value or a list of subordinate configuration properties. Subordinate configuration properties must have unique names within their parent property.</p>
<p>7) The configuration system is able to notify modules or other registered listeners that a configuration change has taken place</p>
<p>8) The configuration system is able to serialize each module's configuration properties as text</p>
<p>9) The configuration serialization should be able to store different language versions of each property file (internationalizable)</p>
<p>10) Each module can have its own configuration and can organize its configuration into any number of namespaces, allowing the module to organize its properties into groups. One use case for this is to allow a module to separate its UI strings from other configuration properties.</p>
<p>11) The configuration system is able to tell which module owns a configuration property</p>
<p>12) The configuration system is able to store default configuration properties separately from user-modified configuration properties. Consider whether user preference files, including loading and sharing of such files, is in scope or not.</p>
<p>13) The configuration system should be loaded under all variants of the Kepler application and should have minimal dependencies.</p>
<p>14) The configuration system should be able to store and make accessible documentation about configuration properties (name, description(s), etc.)</p>
<p>15) The configuration system stores strings only. It is up to a module to cast strings to implied value types.<br /> -- desire to include data types as as part of the model (rather than only accept strings)<br /> -- desire to have utility methods for doing casting (e.g., getValueAsInt)<br /> -- change this requirement to: The configuration system will support basic types. If no type is specified, the type will default to String.<br /> -- basic types TBD.</p>
<p>16) The config system should define the set of allowable characters for use in names (e.g., no spaces, or should be java identifiers)</p>
<p>17) The configuration system should specifically consider if and how configuration values or sets of values from the command-line, environment variables, module.txt and other sources should be merged or provided with values from configuration files</p> Kepler - Bug #3948: Create new configuration system supporting moduleshttps://projects.ecoinformatics.org/ecoinfo/issues/3948?journal_id=133112009-10-06T19:26:54ZChad Berkleyberkley@nceas.ucsb.edu
<ul></ul><p>Current configuration files in kepler:</p>
<p>Directory: common/configs/ptolemy/configs/kepler<br />authServicesBundle.properties<br />caseTableauFactory.xml<br />ConfigGUIAndCache.xml<br />ConfigGUINoCache.xml<br />ConfigNoGUIWithCache.xml<br />configuration.xml<br />graphTableauFactory.xml<br />jobLauncher.properties<br />repositoryBundle.properties<br />uiContextMenuMappings_en_US.properties<br />uiDisplayText_en_US.properties<br />uiMenuMappings_en_US.properties<br />uiSettings.properties<br />uiSVGIconMappingsByClass.properties<br />uiSVGIconMappingsByLSID.properties</p>
<p>Directory: common/resources/configurations</p>
<p>config.xml</p>
<p>Not sure if some of these are still used. Will have to determine that when converting to the new format (whatever that ends up being). This is just a list so we know what's out there. There are also other configuration files in various modules, but these are the main kepler ones.</p> Kepler - Bug #3948: Create new configuration system supporting moduleshttps://projects.ecoinformatics.org/ecoinfo/issues/3948?journal_id=133122009-10-19T20:56:29ZChad Berkleyberkley@nceas.ucsb.edu
<ul></ul><p>I've been working on prototyping and setting up unit tests for the configuration system. There are a few issues I've been running into. I'd like to get a general discussion of these issues and try to come to some conclusions.</p>
<p>The issue we left off with on the last call is what serialization format to use. The conclusion I've come to is that we really do not <strong>need</strong> to come up with one standard configuration format to use. If we think this is very important, we should, but the system I've been working on uses interfaces for serialization and deserialization making it very easy to plug different readers and writers in.</p>
<p>I've been experimenting with YAML and Apache Commons Configuration. Commons is much more flexible. It can read/write 9 different types of files. It supports highly nested structures very well. XML is the easiest format to use with Commons.</p>
<p>While the formatting of the YAML configuration file is very nice in its simplicity, nested structures are somewhat difficult to handle in the API. Since most of our current configuration is highly nested, it's my opinion that commons should be used for porting these configuration files to the new system. YAML could be used for new, or more simply structured files. The use of soft tabs to indicate nesting is somewhat fragile, imho, since accidentally deleting a couple spaces can give you a vastly different structure that what you intended.</p>
<p>Another issue is when a serialization should take place. There are a couple different choices. 1) Serialize the configuration file whenever a change is made. This is the Mac OSX style of doing things. 2) Wait for a calling class to initialize a serialization (basically calling save()). Number 1 makes it easier to keep a serialized config file in sync with the more abstract ConfigurationProperty class. Number 2 is simpler and will allow many properties to change then allow the entire file to be overwritten which is a simpler operation that doing a diff to just update the "dirty" properties.</p>
<p>One more issue that I've come up with is that requirement <a class="issue tracker-1 status-3 priority-2 priority-default closed" title="Bug: MCAT won't build under IRIX with Oracle 8.0.5 (Resolved)" href="https://projects.ecoinformatics.org/ecoinfo/issues/6">#6</a> should be changed. It states that all subproperties should have a unique name. This should not be a requirement since we have many instances of subproperties in our current config files that use the same name multiple times. This is not a requirement of XML or any other nested language that I know of. Anyone object to me changing this requirement?</p>
<p>I'll leave this open to discussion here on bugzilla and on kepler-dev. If we think we need a phone call to figure out these issues (or if anyone has any other issues) let me know and we can arrange that.</p> Kepler - Bug #3948: Create new configuration system supporting moduleshttps://projects.ecoinformatics.org/ecoinfo/issues/3948?journal_id=133132009-10-29T20:12:45ZChad Berkleyberkley@nceas.ucsb.edu
<ul></ul><p>The configuration system is now 99% done with only minor tweaks needed. I am still finding and adding utility methods to make accessing properties easier for programmers. I am now moving on to bug 4336. I'm starting with getting rid of the configxml system and replacing it with this. Then I'll move on to the configuration.xml file and the .properties files.</p>
<p>This system conforms to the requirements set out here:<br /><a class="external" href="https://kepler-project.org/developers/teams/framework/kepler-configuration/proposed-future-kepler-configuration-system">https://kepler-project.org/developers/teams/framework/kepler-configuration/proposed-future-kepler-configuration-system</a></p>
<p>I'm closing this bug, as I believe this is effectively done. We can reopen new bugs later if issues crop up.</p> Kepler - Bug #3948: Create new configuration system supporting moduleshttps://projects.ecoinformatics.org/ecoinfo/issues/3948?journal_id=133142013-03-27T21:25:04ZRedmine Admin
<ul></ul><p>Original Bugzilla ID was 3948</p>