[Scons-dev] Cross-language support
Jason Kenny
dragon512 at live.com
Wed Jul 29 22:56:44 EDT 2015
OK
I think we agree case 4 is best. Let me look at the idl test case. I am not sure what this would be an issue with the installer builder at the moment.
Jason
From: William Blevins
Sent: Wednesday, July 29, 2015 9:50 PM
To: SCons developer list
Subject: Re: [Scons-dev] Cross-language support
On Wed, Jul 29, 2015 at 10:05 PM, Jason Kenny <dragon512 at live.com> wrote:
Ya I see this...
So I trying to take a little time tonight and figure out what happens.. Could use some help.
I currently looking at test/CPPSUFFIXES.py
This fails at line 107. From what I can tell the test expects output of
C:\Python27\python.exe mycc.py test1.o test1.c
but instead we get
C:\Python27\python.exe mycc.py test1.o test1.c
Install file: "test1.c" as "test1_c"
Install file: "test1.h" as "test1_h"
This happens because foo.h was changed causing these files to be copied again.
So problems I have with this test.
1) no one would copy headers that depend on a header not passed to the install area. I am not sure if we should treat <> different from “” includes. It a common practice to use “” for headers in your library and <> for stuff outside your library. I don’t this really helps either way
2) the copy/install builder does not generate new file if the source is different. What we have here is a good scanner that would do the right thing if the “binary” needs to be rebuilt, but redundant is the copy has to happen.
Gary and I were of the same opinion. The functionality is correct and matches all other scanner/builder behavior, but it is inefficient...
The issue in a way is that SCons makes the installed test1_c out of date because foo.h is out of date, when in fact this is a case in which we would want to update logic to care more about the csig of the source being the same as what is stored and not doing anything as technically this is correct behavior for copy like builders and ignoring that the scanner header is out of date.
I see few solutions to this:
1) we say current behavior is correct and update the test.
This is the easiest, but I was hoping to resolve this performance issue.
2) we make a change to allow a install/copy builder to say the scanner items are Required() which remove the Depends() state
I looked into doing this, but I didn't see an easy way to do this. I made some notes: https://bitbucket.org/scons/scons/pull-requests/237/issue-2264-cross-language-scanner-support/diff#comment-8502930
3) we change builder to apply a custom sig check for the install nodes to ignore that a file dependent from the scanner.
This seems like a reasonable approach unless this means updating every scanner. The fundamental problem here is that language scanners already exist that exhibit correct behavior. I don't want to duplicate any part of the scanner code. I feel like going this route will require some level of duplication.
4) remove the scanner from the install builder as we only want to copy files that change
I assume this is functional equivalent to disabling recursive scanning in the install builder. I already considered this and think its the best choice. See some notes here about our previous conversation: https://bitbucket.org/scons/scons/pull-requests/237/issue-2264-cross-language-scanner-support/diff#comment-8521558
With that change only 3 tests are left failing for me. See copa-pasta snippet below:
test/IDL/IDLSUFFIXES.py # If I understand the test correctly, this test checks to see that the bug we are trying to fix works???
test/explain/basic.py # Contains code with build dependencies on installed files as sources: is this something SCons explicitly supports?
test/explain/save-info.py # Contains code with build dependencies on installed files as sources: is this something SCons explicitly supports?Please pay special attention to test/IDL/IDLSIFFIXES.py because this test requires the recursive scanner behavior to work which leads me to believe that the recursive scanner problem already existed, and there used to be a hack for this with the CScanner...
I lean toward 4) logic given I how I understand the CCopy builder in Parts works. and that the install builder for the most part is a copy builder that adds nodes to a list to be used by the Scons version of the packaging builders.
Agreed. See above.
Jason
From: William Blevins
Sent: Wednesday, July 29, 2015 7:19 PM
To: SCons developer list
Subject: Re: [Scons-dev] Cross-language support
Jason.
FYI. The failing tests you listed match the list from June. The action-test.py may be new.
https://bitbucket.org/scons/scons/pull-requests/237/issue-2264-cross-language-scanner-support/diff#comment-7323143
V/R,
William
On Wed, Jul 29, 2015 at 8:16 PM, William Blevins <wblevins001 at gmail.com> wrote:
Jason,
As previously stated in the pull request comments, I expect some tests to fail currently because of the recursive header install issue (topic #1): <lang>SUFFIXES, HeaderInstall.py. The other tests may or may not be related.
There is more value in looking at those tests first. It may also be valuable to go back 3-4 commits to the closest ancestor of the default branch and run tests there, so that we can see if some of the tests failed previously. Also, if we think its a problem of the current location of that divergent head, then we can see if a rebase fixes some of them.
V/R,
William
On Wed, Jul 29, 2015 at 7:09 PM, Jason Kenny <dragon512 at live.com> wrote:
ok not any better...
Failed the following 16 tests:
test\Batch\action-changed.py
test\CPPSUFFIXES.py
test\DSUFFIXES.py
test\Fortran\FORTRANSUFFIXES.py
test\HeaderInstall.py
test\MSVS\vs-9.0-exec.py
test\Win32\mingw.py
test\exitfns.py
test\implicit\IMPLICIT_COMMAND_DEPENDENCIES.py
test\import.py
test\long-lines\signature.py
test\scons-time\run\config\python.py
test\scons-time\run\option\python.py
test\sconsign\script\SConsignFile.py
test\sconsign\script\Signatures.py
test\sconsign\script\no-SConsignFile.py
I would need to dig down to see what is failing and why
Jason
From: Jason Kenny
Sent: Wednesday, July 29, 2015 5:38 PM
To: SCons developer list
Subject: Re: [Scons-dev] Cross-language support
ok I have it fixed... I am rerunning the tests now
Jason
From: Jason Kenny
Sent: Wednesday, July 29, 2015 5:30 PM
To: SCons developer list
Subject: Re: [Scons-dev] Cross-language support
I must have messed up. I only see debugCount and JniHeaderDir bookmarks at the moment.
Jason
From: William Blevins
Sent: Wednesday, July 29, 2015 4:50 PM
To: SCons developer list
Subject: Re: [Scons-dev] Cross-language support
That's a SCons fork. Not an HG branch (in case its a terminology problem). Make sure you are using the CrossLanguage bookmark.
V/R,
William
On Wed, Jul 29, 2015 at 5:45 PM, Jason Kenny <dragon512 at live.com> wrote:
branch SCons_20150323
I can will setup from scratch again and try again.
Jason
From: William Blevins
Sent: Wednesday, July 29, 2015 4:38 PM
To: SCons developer list
Subject: Re: [Scons-dev] Cross-language support
Jason,
Are you sure you are using the correct commit revision?
None of those tests are failing on my Linux distro, and I haven't modified any of those tests either. Plus, tests that I am expecting to fail are apparently passing.
V/R,
William
On Wed, Jul 29, 2015 at 5:06 PM, Jason Kenny <dragon512 at live.com> wrote:
So a quick pass of the tests has these failures
Failed the following 15 tests:
test\Batch\action-changed.py
test\MSVS\vs-9.0-exec.py
test\Win32\mingw.py
test\exitfns.py
test\implicit\IMPLICIT_COMMAND_DEPENDENCIES.py
test\import.py
test\long-lines\signature.py
test\option\debug-count.py
test\option\debug-multiple.py
test\option\debug-objects.py
test\scons-time\run\config\python.py
test\scons-time\run\option\python.py
test\sconsign\script\SConsignFile.py
test\sconsign\script\Signatures.py
test\sconsign\script\no-SConsignFile.py
From: Jason Kenny
Sent: Wednesday, July 29, 2015 2:32 PM
To: SCons developer list
Subject: Re: [Scons-dev] Cross-language support
Ok let me update the branch and re-run everything.
Jason
From: William Blevins
Sent: Wednesday, July 29, 2015 10:30 AM
To: SCons developer list
Subject: Re: [Scons-dev] Cross-language support
Jason,
I'm not sure. I remember that you were helping me look into the recursive Install behavior. Plus, possible parts incompatibility?
I haven't ran the tests on a windows box at all. I don't expect anything new, but it should be done before it ships :)
V/R,
William
On Wed, Jul 29, 2015 at 8:44 AM, Jason Kenny <dragon512 at live.com> wrote:
Which MS toolchain on windows do we need tested? I can do a test run this afternoon.
Jason
From: William Blevins
Sent: Wednesday, July 29, 2015 1:11 AM
To: SCons developer list
Subject: Re: [Scons-dev] Cross-language support
Happy with it*
Until the review is complete, and I can perhaps get a few more guinea pigs to try it out, it's hard to make a concrete projection. I have 3 items listed out that need to be given a thumbs up/down at a minimum:
* Item #1 is still mostly outstanding. I'm not sure how to address, please see the discussion Jason and I started a few weeks ago under the pull request comments.
* Item #2 requires no changes (to my knowledge), but someone with more QT knowledge may say otherwise.
* I may already have most of the patch for Item #3 (IE. no scanner for key) based on today's feedback.
I also still need to request test runs on Windows, and for toolchains that I may not have installed or instructions to install them (EG. D).
V/R,
William
On Wed, Jul 29, 2015 at 1:58 AM, William Blevins <wblevins001 at gmail.com> wrote:
As soon as we are happen with it :)
On Wed, Jul 29, 2015 at 12:30 AM, Bill Deegan <bill at baddogconsulting.com> wrote:
O.k. let me push 2.3.5 with the visual studio 2015 stuff.
Then we'll changed to 2.4 merge slots. stabilize. release.
Then this code? (2.5?)
-Bill
On Tue, Jul 28, 2015 at 6:33 PM, Jason Kenny <dragon512 at live.com> wrote:
I have been using the slots drop for a while with Parts. I think it is ready. It does have a notable improvement in speed and memory size. I would before getting this out as officially earlier than later.
Jason
From: William Blevins
Sent: Tuesday, July 28, 2015 8:31 PM
To: Dirk Bächle ; SCons developer list
Subject: Re: [Scons-dev] Cross-language support
On Tue, Jul 28, 2015 at 7:16 PM, Dirk Bächle <tshortik at gmx.de> wrote:
Hi guys,
sorry for chiming in so late.
On 28.07.2015 23:44, Gary Oberbrunner wrote:
Yes, that's how we've done it in the past. Sounds like doing it at the same time as slots would be perfect.
Doing this in parallel with the "slots" change sounds good to me too. +1
I'm not opposed to releasing in the same update as slots, but since the cross language code reviews haven't been finished, I don't want to delay slots since it is ready now.
The pessimist in me as sees doing two major enhancements in the same release as higher risk; I don't foresee any issues, but its worth the thought if the release overhead isn't too bad.
-- Gary
On Tue, Jul 28, 2015 at 2:01 PM, Bill Deegan <bill at baddogconsulting.com <mailto:bill at baddogconsulting.com>> wrote:
Gary,
For such a change we should bump the second digit?
2.4?
I agree we should not turn down a change because it will cause rebuilds where the past didn't as long as it is now more correct
(which it should be with this change).
Yes, "forward" is the way to go. ;)
Also agree we should be verbose in our notification of the impacts of the new change to avoid (as much as we can) "surprises".
I think (better: hope) we did a good enough job for the "slots" stuff on this. For the scanner changes, I see them more like a fix...so a single announcement should be sufficient?
Finally, and just in case I haven't done so already, I'd like to thank William for all the work he's done on this issue. I couldn't help as much as I would've liked, but with Gary's support you tackled this down and brought it to a good end. Kudos to you...bravo!
Truly appreciated. I have spent a lot of time on this issue despite what one might expect from the number of lines of code.
It was honestly my first time working in a "real" python environment outside of scripting, and the SCons code base is rather complex. It instills me greater appreciation for the work that has already done.
I also want to thank everyone here for their help past and future, but don't pat me on the back until its done. I might get lazy :)
Best regards,
Dirk
_______________________________________________
Scons-dev mailing list
Scons-dev at scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev
--------------------------------------------------------------
_______________________________________________
Scons-dev mailing list
Scons-dev at scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev
_______________________________________________
Scons-dev mailing list
Scons-dev at scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev
_______________________________________________
Scons-dev mailing list
Scons-dev at scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev
--------------------------------------------------------------------
_______________________________________________
Scons-dev mailing list
Scons-dev at scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev
_______________________________________________
Scons-dev mailing list
Scons-dev at scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev
----------------------------------------------------------------------
_______________________________________________
Scons-dev mailing list
Scons-dev at scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev
----------------------------------------------------------------------
_______________________________________________
Scons-dev mailing list
Scons-dev at scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev
_______________________________________________
Scons-dev mailing list
Scons-dev at scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev
------------------------------------------------------------------------
_______________________________________________
Scons-dev mailing list
Scons-dev at scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev
_______________________________________________
Scons-dev mailing list
Scons-dev at scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev
--------------------------------------------------------------------------
_______________________________________________
Scons-dev mailing list
Scons-dev at scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev
--------------------------------------------------------------------------
_______________________________________________
Scons-dev mailing list
Scons-dev at scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev
--------------------------------------------------------------------------
_______________________________________________
Scons-dev mailing list
Scons-dev at scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev
_______________________________________________
Scons-dev mailing list
Scons-dev at scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev
------------------------------------------------------------------------------
_______________________________________________
Scons-dev mailing list
Scons-dev at scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev
_______________________________________________
Scons-dev mailing list
Scons-dev at scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev
--------------------------------------------------------------------------------
_______________________________________________
Scons-dev mailing list
Scons-dev at scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist2.pair.net/pipermail/scons-dev/attachments/20150729/bc301e41/attachment-0001.html>
More information about the Scons-dev
mailing list