[Scons-dev] SCons and Java

Fri May 30 19:59:17 EDT 2014

Dirk,

An after thought on my original reply:

Users might also accidentally delete "*.class" files in your output dir,
> between two builds. And they expect SCons to rebuild those intermediate
> targets on its next invocation. How do you want to handle this?

SCons wouldn't rebuild since no source had changed.  This would be an issue
>> if the user was trying to share *.class file(s) which would only be
>> relevant with something like SCons.JavaH, but it supports the directory
>> parameter(s). SCons could check that the directory contents had changed and
>> mark as requiring rebuild; I am grabbing at straws on this one...
>
>
In the case of SCons.JavaH, you would probably want to denormalize the
build.  This might sound terrible at first, but it work(s) well in some
database implementation(s) requiring high-speed throughput.

JavaH work make a separate Java compiler invocation.  It would require more
overall work, but the build would be correct, and the target directory
could be cleaned up after Java/JavaH call(s) since the file(s) are waste
product(s) of the process and wouldn't be tracked in the DAG.

V/R,
William

On Fri, May 30, 2014 at 7:39 PM, William Blevins <wblevins001 at gmail.com>
wrote:

> Dirk,
>
> Team,
>> I have a couple points I'd like to discuss, but for the sake of
>> organization, I intend to split them into separate emails.
>> Java Part #2
>> There has been some discussion on making Scons more ANT-like,
>> http://www.scons.org/wiki/JavaSupport, and that might solve some issues
>> like identifying Java targets.  There is merit here assuming that Java
>> support stays SCons-like at the same time; otherwise, why not just use ant
>> or maven?
>>
>
>
> From this I get that you'd like to have some form of Java support in SCons
>>> directly, and don't want to delegate things to an external program,
>>> correct? At least it's my assumption for the following remarks...
>>
>> SCons already has Java support (right?).  I have been a SCons user for a
> few years and am interested in trying to make the Java toolkit better
> because I dislike Ant-like build system(s), but I feel that SCons still has
> room for improvement.  I am braining storming ways to improve the setup.  I
> have a bunch of idea(s) that I would like to discuss and this just happens
> to be one of them (#2 in fact).  I know that the SCons teams has talked
> about refactoring the SCons Java toolkit, but outside the notes from the
> link above, I haven't seen any further discussions.  I am the new kid on
> the scons-dev mailing list block, so feel free to give me a tour.  I would
> be willing to work on SCons with some direction.
>
> Problem(s) this will solve:
>> 1. Determining the source -> target(s) from *.java -> *.class is painful,
>> possibly java version dependent, and performance intensive (possibly);
>> thus, we should optimize this out.
>> 2. Java developers are plagued requiring manual cleaning of a jar far too
>> often with ant-like system(s) and the current scon(s) methodology.
>>    1. Clean could be required when deleting or moving a file since the
>> last build output isn't removed otherwise; avoidable in current SCons
>> depending on how sources are specified (not obvious and/or intuitive coming
>> from ant-like system(s)).
>> 3. Simplify the SCons Java API to allow building jar file(s) without
>> having to interact with *.class file(s) at all.
>> 4. Simplify the SCons Java support for resource file(s); currently, you
>> have to do copying by hand (if it's supported I know not where or how).
>> 5. Reduce the amount of code required to build a jar (by a lot).
>>
>
>
> This sounds like an even more simplified Java Builder than what we have
>>> now. If you could come up with a SoSimpleJava() Builder, we'd probably add
>>> it to the core and let people have their own choice.
>>
>> That's fair, but since I am talking about not tracking *.class file(s) at
> all.  The old function(s) couldn't exist.
>
>  How I imagine this should work:
>
> 1. Treat class file(s) like side-effect(s) and do not include them (at
>> all) in the dependency tree.
>>     1. This means we don't have to scan the Java file(s) and determine
>> *.class targets.
>>
>
>
> I don't think that you can have both. If you treat files as SideEffects
>>> (in the SCons sense), they have to be part of the dependency DAG. See next
>>> comment...
>>
>> I meant this in the truer form of side-effect and not the
> SCons.SideEffects sense.  Getting from A -> C by passing through B, but B
> is just along the way.  The only way the *.class file(s) could be part of
> the DAG is if they were identified after the Java compiler had already
> created the *.class file(s) by basically Glob(*.class).
>
> I would really like to get away from SCons being required to scan Java
> source file(s) to determine the targets due to the 1 -> N nature of *.java
> to *.class.  This would resolve several issues with the SCons Java toolkit
> including incorrect dependency tree(s) with derived (generated) Java source
> file(s) and some build issue(s) that require manual clean operation(s) on
> Java code (which may be alleviated depending on how you setup your
> SConscript, but may not be obvious coming from an ant-like system).
>
> The Java DAG is already incomplete because SCons doesn't add item(s) in
> JAVACLASSPATH automatically, so user(s) are required to do it manually
> anyway.  At first I thought this was a bug, but it's even documented in the
> User's Guide, so I assume it's intentional which confuses the crap out of
> me?
>
> 2. Remove the Java(...) function from SCons; thus, do not interact with
>> *.class file(s) directly ever.  There isn't a *good* reason to do this.
>>
>
>
> Here is just one scenario, where dealing with *.class files actually makes
>>> sense: If you're using anonymous classes in a Java source, a bunch of
>>> additional class files get generated. How do you remove these files on a
>>> "clean", if they are located in a source folder (which people might prefer
>>> for some reason)?
>>
>> Remove all *.class file(s) generated within the directory tree were the
> Java compiler is expected to output it's source.  I'd almost prefer to
> require the build directory and just delete it.  Making the build
> configuration simple and clean is more important I think.
>
> More experienced Java developers can probably come up with a dozen more
>>> useful cases...I'm not sure how often people would want to integrate
>>> pre-built *.class files (coming from Scala perhaps?) into their builds, for
>>> example.
>>
>> Are you talking about coping *.class file(s) between build directories?
>  Why not use a *.jar file? At that point you are just moving code
> copy-pasta  from your source code to your build tool.
>
>  You may be able to work around the "clean" problem, by creating all
>>> "*.class" files in a separate folder structure (pretty much like a
>>> VariantDir). But what you'll end up with is a somewhat crippled DAG,
>>> forcing you to remove this output dir completely and rebuild everything
>>> from scratch...even when only a comment in a single Java source is changed.
>>> Only to "be sure", just like in make or Ant...there is not much of a win
>>> here for the user, I'm afraid.
>>
>> The win here is that Java compiler is much faster than the SCons source
> emitter (I think that's the right term) for Java. There is a performance
> bottleneck(s) in the Java toolkit which I will discuss in another email; I
> hope the team is alright with email attachments because I have performance
> logs.
>
>  Users might also accidentally delete "*.class" files in your output dir,
>>> between two builds. And they expect SCons to rebuild those intermediate
>>> targets on its next invocation. How do you want to handle this?
>>
>> SCons wouldn't rebuild since no source had changed.  This would be an
> issue if the user was trying to share *.class file(s) which would only be
> relevant with something like SCons.JavaH, but it supports the directory
> parameter(s). SCons could check that the directory contents had changed and
> mark as requiring rebuild; I am grabbing at straws on this one...
>
> Finally, you should take into account that users might want to create
>>> different versions of the same source code. Just as an example, let's
>>> assume they want to create an "original" and "obfuscated" target(s), or a
>>> "simple" and "extended" release where additional packages get included
>>> based on environment variables. And for maximum speedup they do this in
>>> parallel, by using the "-j" option...so a single dedicated output dir won't
>>> do probably.
>>
>> There would be 1 target directory per Java compiler invocation which I
> imagined would be 1 target directory per *.jar created.  The gain from
> threading packages in a *.jar file shouldn't be observable until well into
> 1000+ *.class file(s) per *.jar file.  At this point, I imagine there are
> bigger architecture concerns at hand than the compile time of 1 *.jar file.
>
> V/R,
> William
>
>
>
> On Fri, May 30, 2014 at 7:13 AM, Dirk Bächle <tshortik at gmx.de> wrote:
>
>>  Hi William,
>>
>> please find a few comments below.
>>
>>
>> On 30.05.2014 02:42, William Blevins wrote:
>>
>> Team,
>>
>>  I have a couple points I'd like to discuss, but for the sake of
>> organization, I intend to split them into separate emails.
>>
>>  Java Part #2
>>
>>  There has been some discussion on making Scons more ANT-like,
>> http://www.scons.org/wiki/JavaSupport, and that might solve some issues
>> like identifying Java targets.  There is merit here assuming that Java
>> support stays SCons-like at the same time; otherwise, why not just use ant
>> or maven?
>>
>>   From this I get that you'd like to have some form of Java support in
>> SCons directly, and don't want to delegate things to an external program,
>> correct? At least it's my assumption for the following remarks...
>>
>>
>>  Problem(s) this will solve:
>> 1. Determining the source -> target(s) from *.java -> *.class is painful,
>> possibly java version dependent, and performance intensive (possibly);
>> thus, we should optimize this out.
>> 2. Java developers are plagued requiring manual cleaning of a jar far too
>> often with ant-like system(s) and the current scon(s) methodology.
>>     1. Clean could be required when deleting or moving a file since the
>> last build output isn't removed otherwise; avoidable in current SCons
>> depending on how sources are specified (not obvious and/or intuitive coming
>> from ant-like system(s)).
>> 3. Simplify the SCons Java API to allow building jar file(s) without
>> having to interact with *.class file(s) at all.
>>  4. Simplify the SCons Java support for resource file(s); currently, you
>> have to do copying by hand (if it's supported I know not where or how).
>> 5. Reduce the amount of code required to build a jar (by a lot).
>>
>>   This sounds like an even more simplified Java Builder than what we
>> have now. If you could come up with a SoSimpleJava() Builder, we'd probably
>> add it to the core and let people have their own choice.
>>
>>  How I imagine this should work:
>> 1. Treat class file(s) like side-effect(s) and do not include them (at
>> all) in the dependency tree.
>>     1. This means we don't have to scan the Java file(s) and determine
>> *.class targets.
>>
>> I don't think that you can have both. If you treat files as SideEffects
>> (in the SCons sense), they have to be part of the dependency DAG. See next
>> comment...
>>
>>
>>  2. Remove the Java(...) function from SCons; thus, do not interact with
>> *.class file(s) directly ever.  There isn't a *good* reason to do this.
>>
>> Here is just one scenario, where dealing with *.class files actually
>> makes sense: If you're using anonymous classes in a Java source, a bunch of
>> additional class files get generated. How do you remove these files on a
>> "clean", if they are located in a source folder (which people might prefer
>> for some reason)?
>>
>> More experienced Java developers can probably come up with a dozen more
>> useful cases...I'm not sure how often people would want to integrate
>> pre-built *.class files (coming from Scala perhaps?) into their builds, for
>> example.
>>
>> You may be able to work around the "clean" problem, by creating all
>> "*.class" files in a separate folder structure (pretty much like a
>> VariantDir). But what you'll end up with is a somewhat crippled DAG,
>> forcing you to remove this output dir completely and rebuild everything
>> from scratch...even when only a comment in a single Java source is changed.
>> Only to "be sure", just like in make or Ant...there is not much of a win
>> here for the user, I'm afraid.
>>
>> Users might also accidentally delete "*.class" files in your output dir,
>> between two builds. And they expect SCons to rebuild those intermediate
>> targets on its next invocation. How do you want to handle this?
>>
>> Finally, you should take into account that users might want to create
>> different versions of the same source code. Just as an example, let's
>> assume they want to create an "original" and "obfuscated" target(s), or a
>> "simple" and "extended" release where additional packages get included
>> based on environment variables. And for maximum speedup they do this in
>> parallel, by using the "-j" option...so a single dedicated output dir won't
>> do probably.
>>
>> Best regards,
>>
>> Dirk
>>
>>
>> _______________________________________________
>> Scons-dev mailing list
>> Scons-dev at scons.org
>> http://two.pairlist.net/mailman/listinfo/scons-dev
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://two.pairlist.net/pipermail/scons-dev/attachments/20140530/acbe907b/attachment-0001.html>