[Scons-dev] Java Development

Mark Flacy mflacy at verizon.net
Sat Aug 2 12:28:11 EDT 2014 

This is from my phone so sorry for the top posting. 

You get the extra class files from:

1. Defining an inner class
2. Creating an anonymous instance of an interface or abstract class.




On August 1, 2014 6:54:16 PM CDT, William Blevins <wblevins001 at gmail.com> wrote:
>The issue is that we are guessing as to what syntax causes the compiler
>to
>emit an extra class file.
>
>
>On Fri, Aug 1, 2014 at 7:49 PM, Mark A. Flacy <mflacy at verizon.net>
>wrote:
>
>> On Friday, August 01, 2014 05:22:39 PM William Blevins wrote:
>> > SCons Java doesn't need to be that fancy, but I think the root
>problems
>> can
>> > be solved. The SCons java tool simply doesn't get the love that
>some of
>> the
>> > other tools get.
>>
>> One common use case is to generate java source from the xjc tool
>(which
>> converts XML schema to java classes) for use by other java classes.
>>
>> The emitter in that case would need to parse a bindings file as well
>as
>> the XML
>> schema(s) that the binding file uses.  However, the resulting java
>files
>> should
>> have no dependencies on any external libraries, so they could be
>compiled
>> as a
>> unit independently from anything using them.
>>
>> And there are Annotations, which can be used to (and *are* used to)
>> generate
>> source.  See http://deors.wordpress.com/2011/09/26/annotation-types/
>for a
>> discussion of how to do that.
>>
>>
>>
>> >
>> > I don't have the experience with the code base to know if my ideas
>are
>> > workable, so I will need some guidance.
>> >
>> > On Thu, Jul 31, 2014 at 11:04 PM, Mark A. Flacy
><mflacy at verizon.net>
>> wrote:
>> > > On Sunday, July 27, 2014 03:10:34 PM William Deegan wrote:
>> > > > William,
>> > > > On July 25, 2014 at 8:27:02 PM, William Blevins (
>> wblevins001 at gmail.com)
>> > > > wrote:
>> > > >
>> > > > Team,
>> > > >
>> > > > I want to get another thread going for SCons Java development.
>> > > >
>> > > > The SCons Java tool has a ton of error reports on Tigris
>including 7
>> > > > priority 1 issues.  At the moment, this tool doesn't stand a
>chance
>> > >
>> > > against
>> > >
>> > > > other Java competitors, and not because they are great tools. 
>I
>> frankly
>> > > > hate ANT.  I have used Java support from SCons and it's
>seriously
>> > >
>> > > painful;
>> > >
>> > > > nothing like the C++ support.  Some other developers have made
>> > > > statements
>> > > > like "No one outside SCons builds Java programs more
>complicated than
>> > >
>> > > hello
>> > >
>> > > > world."  The SCons tool framework is great, and I would really
>like
>> to
>> > >
>> > > see
>> > >
>> > > > the Java toolkit see some love.  It has potential to be a
>hidden gem,
>> > >
>> > > and I
>> > >
>> > > > want help out with this, but I don't have the experience to do
>this
>> on
>> > > > my
>> > > > own, so firstly I'd like to list some of the biggest hurdles to
>users
>> > >
>> > > SCons
>> > >
>> > > > Java.  I'm not gonna try and propose any solutions at the
>moment.  I
>> > > > just
>> > > > want to see if I can get the group thinking about the problems.
>> > > >
>> > > > 1. Adding resource files to a jar causes SCons
>> > > > segfault: http://scons.tigris.org/issues/show_bug.cgi?id=2550
>> > > >
>> > > > I have firsthand experience with this bug.  The only way I
>could
>> figure
>> > >
>> > > out
>> > >
>> > > > how to workaround it was to make a separate jar just for
>resources.
>> > > >
>> > > > 2. Java emitter almost never gets the java output correct.
>> > > >
>> > > > One of the many things I hate about ANT is that ANT is stupid. 
>It
>> > > > always
>> > > > executes a build even if code is up-to-date and I usually have
>to
>> > > > explicitly clean.  SCons COULD resolve both problems if the
>emitters
>> > > > worked.  The only way to get remotely consistent working build
>is to
>> > > > call
>> > > > Jar( 'buildDir' ) when everyone wants to do Jar( [ 'class1',
>...
>> > >
>> > > 'classN' ]
>> > >
>> > > > ).
>> > > >
>> > > > 3. Dependencies:
>> > > >
>> > > > SCons does not automatically add classpath items as
>dependencies.
>>  Why
>> > >
>> > > do I
>> > >
>> > > > need to do this manually?  This is what SCons does!  It's the
>heart
>> and
>> > > > soul!
>> > > >
>> > > > I believe this is because of item #6.
>> > > >
>> > > > 4. Consistency:
>> > > >
>> > > > Classpath tokens (among other items) do not behave the same as
>other
>> > > > builders. Example: I cannot use "#jar/item.jar" in the
>classpath
>> without
>> > > > expanding via something like File(...).get_path().
>> > > >
>> > > > 5. Interfaces:
>> > > >
>> > > > Java(...) parameters and internal handling aren't intuitive and
>only
>> > >
>> > > handles
>> > >
>> > > > sources = 'directory' correctly.  It doesn't do lists of java
>files
>> or
>> > >
>> > > list
>> > >
>> > > > of directories in a sane manner.
>> > > > http://scons.tigris.org/issues/show_bug.cgi?id=1772
>> > > >
>> > > > Personally, I don't think that Java and Jar should be separate
>> > > > functions.
>> > > >
>> > > >  How do you get to the classes then?  What about Javah!  I have
>an
>> idea,
>> > > >
>> > > > but that's outside the scope of this rant.
>> > > >
>> > > > 6. Performance:
>> > > >
>> > > > The dependency structure for Java exposes class files in a way
>that
>> > >
>> > > creates
>> > >
>> > > > tons of false positives.
>> > > >
>> > > > SCons current:
>> > > >
>> > > > classes1 = Java(...)
>> > > >
>> > > > classes2 = Java(...)
>> > > >
>> > > > Depends( classes2, classes1 ) # O( N^2 ) dependency graph with
>tons
>> of
>> > >
>> > > false
>> > >
>> > > > positives
>> > > >
>> > > > SCons if I have anything useful to say about it
>> > > >
>> > > > jar1 = Jar( classes1 )
>> > > >
>> > > > jar2 = Jar( classes2 )
>> > > >
>> > > > Depends( jar2, jar1 ) # O( 1 ) which obviously fails in
>parallel
>> builds
>> > > > currently.
>> > > >
>> > > > I am currently data mining a production Java codebase to prove
>my
>> point.
>> > > > Dirk and I have already discussed this issue somewhat; thanks
>Dirk :)
>> > > >
>> > > > This actually causes the Task Master thread to get blocked on
>large
>> jars
>> > > > reducing parallel efficiency in builds to None.
>> > > >
>> > > >
>> > > > The main issue here (if I understand SCons’ internals enough)
>is that
>> > > > SCons’s doing all dependencies on a per file basis. For many
>types of
>> > > > builds that works fine. For Java (building jars, and other
>issues)
>> and
>> > >
>> > > some
>> > >
>> > > > other types of builds, that’s very inefficient.
>> > > >
>> > > > There’s no “blob” of files where you have N inputs and M
>outputs, and
>> > >
>> > > thus
>> > >
>> > > > you get the N*M arcs in the graph.
>> > > >
>> > > > Currently the only similar but not really similar enough is the
>Dir()
>> > >
>> > > Node
>> > >
>> > > > type.  But that has it’s own problems, which could be solved by
>a N*M
>> > >
>> > > type
>> > >
>> > > > node.
>> > > >
>> > > >
>> > > >
>> > > > Other Java issues could likely be resolved building on top of
>such a
>> new
>> > > > Node type.
>> > > >
>> > > > Though resolving the anonymous and inner classes in a java file
>> creating
>> > > > more than one class file and what it might be named is also
>still an
>> > >
>> > > issue
>> > >
>> > > > which the scanner and emitter try to solve by parsing the java
>files
>> and
>> > > > figuring out the proper naming.  This of course is not (as I
>> understand
>> > >
>> > > it)
>> > >
>> > > > formally defined as part of the java language and thus is a per
>> compiler
>> > > > implementation detail.
>> > >
>> > > In my opinion, you are wasting your time.
>> > >
>> > > The way java produces artifacts is so different from the model
>that
>> SCons
>> > > "expects" that you are attempting to cram an elephant down a
>shrew's
>> > > throat.
>> > > (or attempting to feed a Great White shark with ants, if you
>prefer) to
>> > > get
>> > > this to work.
>> > >
>> > > SCons works *great* when there's one output per input, especially
>when
>> you
>> > > can
>> > > deduce the output name from the input name and little else.  It
>also
>> works
>> > > great when it is fairly cheap to determine what files a given
>file
>> depends
>> > > upon.
>> > >
>> > > Sadly, neither of those conditions exist with java source.
>> > >
>> > >
>> > > Between 2003 and 2010, I developed and maintained a python based
>build
>> > > tool
>> > > for a Java project that contained ~10K java source files.  It
>would
>> > > analyse the
>> > > java source files to figure out the package-level compile
>dependencies
>> and
>> > > then
>> > > send lists of files to be compiled as a unit to a persistent
>compile
>> > > server (in
>> > > fact, you could run multiple compile servers at once to get a
>large
>> degree
>> > > of
>> > > parallelism).  It would then analyze the generated class files to
>see
>> if
>> > > the
>> > > total public/protected interface for the package changed,
>preventing
>> > > recompiles if all you did was change an implementation detail
>versus a
>> > > visible
>> > > method or attribute.
>> > >
>> > > You could also register various hooks against packages (such as
>run
>> rmic)
>> > > if
>> > > the package compiled correctly.  We did some funky stuff with
>> > > autogenerating
>> > > WSDL from java source and then generate client classes from the
>WSDL
>> and
>> > > compiling those classes also.  Almost all of that was data
>driven.  It
>> did
>> > > other neat stuff that I don't remember off the top of my head,
>but I've
>> > > got all
>> > > the source sitting around in various tla/hg repositories.
>> > >
>> > > Nobody uses it any more.  Even me, and I thought about providing
>some
>> it
>> > > in an
>> > > open source build tool.
>> > >
>> > > For our Windows users, it was faster to nuke the output
>directories and
>> > > recompile everything in one invocation of javac. (Well, 2
>invocations
>> > > since
>> > > some of the funky stuff was to use the .class files from the
>first
>> compile
>> > > to
>> > > generate the wsdl and use *that* wsdl to generate java source as
>part
>> of
>> > > the
>> > > second bulk compile.) Most of the time difference had to do with
>the
>> way
>> > > the
>> > > python tool talked to the compile processes by sending the list
>of file
>> > > names
>> > > through a pipe to the other process and then flushing the pipe. 
>The
>> rest
>> > > had
>> > > to do with the time spent parsing the files (and I maintained a
>cache
>> of
>> > > parsed
>> > > file information that contained the last mtime of the source
>files and
>> > > would not
>> > > process a file whose mtime was the same as what was recorded).
>> > >
>> > > For our Linux users (me, for one), it was not quite a wash to
>change
>> the
>> > > behavior but the delta wasn't enough to worry about.
>> > >
>> > > I'm more than willing to discuss this and also to be proven
>wrong.
>> > >
>> > > --
>> > > Mark A. Flacy
>> > > _______________________________________________
>> > > Scons-dev mailing list
>> > > Scons-dev at scons.org
>> > > http://two.pairlist.net/mailman/listinfo/scons-dev
>> --
>> Mark A. Flacy
>> _______________________________________________
>> Scons-dev mailing list
>> Scons-dev at scons.org
>> http://two.pairlist.net/mailman/listinfo/scons-dev
>>
>
>
>------------------------------------------------------------------------
>
>_______________________________________________
>Scons-dev mailing list
>Scons-dev at scons.org
>http://two.pairlist.net/mailman/listinfo/scons-dev

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://two.pairlist.net/pipermail/scons-dev/attachments/20140802/ab4d0536/attachment-0001.html>

More information about the Scons-dev mailing list