Kepler

We seem to be running into this bug:
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=5082928

By not generating ptolemy javadocs, I get a lot farther but we need to get the <a.* out of the javadoc comments. This bug looks like it's been around for years and sun isn't fixing it.

This seems to run now from the command line, but not from ant. Might be a memory problem.

BTW - javadoc runs ok on the ptolemy tree.

Looking at the stack trace, it is almost like it

This is with:
[bldmastr@sisyphus build-area]$ javadoc -J-version
java version "1.5.0_19"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_19-b02)
Java HotSpot(TM) 64-Bit Server VM (build 1.5.0_19-b02, mixed mode)

[create-javadoc] Generating /home/bldmastr/kepler/javadoc/org/ecoinformatics/seek\
/querybuilder//DBSelectTableModelItem.html...

Running kill -3 on the process yields a stack trace:

[create-javadoc] "main" prio=1 tid=0x000000004e811210 nid=0x85a runnable [0x00007\
fff058be000..0x00007fff058c0b20]
[create-javadoc] at java.io.FileOutputStream.writeBytes(Native Method)
[create-javadoc] at java.io.FileOutputStream.write(FileOutputStream.java:260)
[create-javadoc] at java.io.BufferedOutputStream.write(BufferedOutputStream.java:105)
[create-javadoc] - locked <0x00002b91b17a71b8> (a java.io.BufferedOutputStream)
[create-javadoc] at java.io.PrintStream.write(PrintStream.java:412)
[create-javadoc] - locked <0x00002b91b17a6bb0> (a java.io.PrintStream)
[create-javadoc] at sun.nio.cs.StreamEncoder$CharsetSE.writeBytes(StreamEncoder.java:336)
[create-javadoc] at sun.nio.cs.StreamEncoder$CharsetSE.implFlushBuffer(StreamEncoder.java:404)
[create-javadoc] at sun.nio.cs.StreamEncoder$CharsetSE.implFlush(StreamEncoder.java:408)
[create-javadoc] at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:152)
[create-javadoc] - locked <0x00002b91b17a69e8> (a java.io.OutputStreamWriter)
[create-javadoc] at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:213)
[create-javadoc] at java.io.BufferedWriter.flush(BufferedWriter.java:236)
[create-javadoc] - locked <0x00002b91b17a69e8> (a java.io.OutputStreamWriter)
[create-javadoc] at java.io.PrintWriter.flush(PrintWriter.java:270)
[create-javadoc] - locked <0x00002b91b17a6948> (a java.io.BufferedWriter)
[create-javadoc] at com.sun.tools.javadoc.Messager.printWarning(Messager.java:242)

It is almost like javadoc is trying to write to stderr or something and can't.

I got the ptolemy tree running fine from the command line. I think the ant task is not attaching std.err correctly (as christopher said in the last comment). Trying to get it to attach correctly.

I got the javadoc command working through ant, but only if I exclude the ptolemy source. I'm not sure what is causing this problem. It doesn't throw any error, it just fails to build any html files. Running in debug mode shows that it loads all of the ptolemy source files, but then just dies with no error. I thought it might be memory related so I bumped the memory up to 2048m with no success. I also tried just running the ptolemy source with no kepler source and it still fails to run. There must be some source file in ptolemy that it doesn't like for some reason. It's annoying now because the generated documentation does not include the ptolemy class links. I'll keep looking into this a bit more...

There is slightly different code that execs a subprocess at
ptolemy/util/StreamExec.java
This code does not use much of Ptolemy, only ExecuteCommands and
a method from StringUtilities. It might be worth temporarily trying
to use this class to see if the "ant javadoc" problem is the
class in the build system that executes the subprocess. I had
a heck of a time getting subprocess execution to work, these classes
also list some resources.

Right now, ant javadoc has lots of errors:
[null] 100 errors
[null] 772 warnings

I wonder why it says [null]?

(Adding my comment to the right bug . . .)

I ported a version of ptolemy.util.StreamExec to
build-area/src/org/kepler/build/util/StreamSingleCommandExec.java
and modified CreateJavadoc.java to use it and the problem with
running javadoc on the ptolemy classes goes away.

I suspect that the problem is that both org.kepler.build.util.CommandLine
and org.apache.tools.ant.taskdefs.Javadoc are incorrectly handling
stderr output from the subprocess. StreamSingleCommandExec creates two
threads to read stdout and stderr.

I also added the Rating Taglet so that running javadoc on the ptolemy
code does not produce output like:
warning - @Pt.AcceptedRating is an unknown tag.

There are a few problems though:
1) We are not properly excluding directories like ptolemy/vendors.
If the ant taskdefs.Javadoc class worked properly, then maybe we could
use the ant exclude facility here?
2) Various third party packages such as the Apache Logging facility
are not included in the CLASSPATH so we get error messages.

The javadoc command in kepler uses -private which generates javadoc
for private methods. Previously, in Ptolemy, we were not using -private
because the private methods are not part of the public or protected interface
and thus are not of use to developers who are using the class. However,
over time we have been adding javadoc to private methods.

I just fixed over 120 javadoc warnings in the Ptolemy code.
I also added code that reads the ptolemy-excludes and .classpath.default files.

We are still missing certain third party jars needed by kepler .java files.
Right now we always include the javadoc for the ptolemy packages. This
might be a mistake, I don't understand -Dfull=true, which looks like it
reads classes from jar files?

"ant javadoc" now yields 2 errors and 901 warnings.
I'm not sure about the 2 errors.
Of those 901 warnings, 136 are related to ptolemy.

The classpath was not being added to the javadoc command so I'm trying that to see if the errors go away. A lot of the warning just come from outdated @param and @return statements. I'll try to clean this up a bit.

Most of our jars only contain class files so adding them to the classpath does not seem to get rid of the warnings associated with javadoc not finding a class. I don't see the need to add source files to kepler for these packages so I'm just going to leave this alone. There are various other warnings, mostly having to do with @param tags who's parameter does not match any parameter in the method signature. This could be fixed with a lot of manual editing, but I don't think it's worth the time at this point. I've got this down to a couple hundred warnings and 3 errors. Overall, this works so I'm going to close this bug as I don't think it's worth more of our time at this point. If this becomes an issue again, we can re-open this bug or open a new one.

Original Bugzilla ID was 4259