[Scons-dev] TEMPFILE spaces bug: your thoughts requested

[Kenny, Jason L]

I believe what he wants to fix this is this function
Action.py line 151
def escape_list(mylist, escape_func):
    """Escape a list of arguments by running the specified escape_func
    on every object in the list that has an escape() method."""
    def escape(obj, escape_func=escape_func):
        try:
            e = obj.escape
        except AttributeError:
            return obj
        else:
            return e(escape_func)
    return list(map(escape, mylist))

Which in the end calls the escape function. I believe a different way to fix is this is to make a new functions that respects "" in the command string. It is late for me at the moment... but if we fix the escape functions to return back

['"c:\\Program Files (x86)\\Microsoft<file:///\\Microsoft> Visual Studio 10.0\\VC\\bin\\AMD64\\lib.exe"', '/nologo', '/OUT:_build\\build_debug_win32-x86_64\\part1\\foo.lib', '_build\\build_debug_win32-x86_64\\part1\\foo.obj']

Instead of the current (this still happens with the current patch):
['"c:\\Program', 'Files', '(x86)\\Microsoft', 'Visual', 'Studio', '10.0\\VC\\bin\\AMD64\\lib.exe"', '/nologo', '/OUT:_build\\build_debug_win32-x86_64\\part1\\foo.lib', '_build\\build_debug_win32-x86_64\\part1\\foo.obj']

I think that will be a lot simpler and honestly make the Spawn function look more correct.

Jason

From: scons-dev-bounces at scons.org [mailto:scons-dev-bounces at scons.org] On Behalf Of Gary Oberbrunner
Sent: Sunday, September 02, 2012 5:24 PM
To: SCons developer list
Subject: Re: [Scons-dev] TEMPFILE spaces bug: your thoughts requested


I believe the patch in the issue solves it. The question is whether there's a simpler way.
On Sep 2, 2012 6:19 PM, "Kenny, Jason L" <jason.l.kenny at intel.com<mailto:jason.l.kenny at intel.com>> wrote:
I made a small test of this with Parts and reproduced it.
It seems that the issue is not the tempfile.. but the way the command get processed SCons and passed to the Spawn calls to spawn the command.

In the example we have here we get something like this as a command:
c:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\bin\AMD64\lib.exe /nologo /OUT:_build\build_debug_win32-x86_64\part1\foo.lib _build\build_debug_win32-x86_64\part1\foo.obj

If I have a command like:
'"$AR" $ARFLAGS /OUT:$TARGET $SOURCES'

I get SCons to print this out as a command:
"c:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\bin\AMD64\lib.exe" /nologo /OUT:_build\build_debug_win32-x86_64\part1\foo.lib _build\build_debug_win32-x86_64\part1\foo.obj

While what I get from the Spawn function looks like this:
Ie ..print cmd,args
"c:\Program ['"c:\\Program', 'Files', '(x86)\\Microsoft<file:///\\Microsoft>', 'Visual', 'Studio', '10.0\\VC\\bin\\AMD64\\lib.exe"', '/nologo', '/OUT:_build\\build_debug_win32-x86_64\\part1\\foo.lib', '"_build\\build_debug_win32-x86_64\\part1\\foo.obj"']

The only way we can get this work at in Scons ( or in any user Spawn functions) is to say something like this:
string.join(args)

This mean the cmd value that is passed in is useless. From what I can see looking at the code it looks like SCons makes a full command and splits it, as you will note the we get strings like:
"C:\Program
and the string of
10.0\\VC\\bin\\AMD64\\lib.exe"

If I use the escape function:
escape(string.join(args))
 I will get
""c:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\bin\AMD64\lib.exe" /nologo /OUT:_build\build_debug_win32-x86_64\part1\foo.lib "_build\build_debug_win32-x86_64\part1\foo.obj""

This leads to a double "" problem that leads to the failure the issue reports.

I need to look farther in to this, as I am not sure what a good solution is. Hopefully what I stated here helps explain the problem

I do know that SCons needs a tweak to deal with how it passes data to the Spawn function.

Jason

-----Original Message-----
From: scons-dev-bounces at scons.org<mailto:scons-dev-bounces at scons.org> [mailto:scons-dev-bounces at scons.org<mailto:scons-dev-bounces at scons.org>] On Behalf Of Gary Oberbrunner
Sent: Sunday, September 02, 2012 9:41 AM
To: SCons developer list
Subject: Re: [Scons-dev] TEMPFILE spaces bug: your thoughts requested

On Sat, Sep 1, 2012 at 11:27 PM, Kenny, Jason L <jason.l.kenny at intel.com<mailto:jason.l.kenny at intel.com>> wrote:

> We gave Steve the patch to this issue.. As I recall he did not take it 100% and changed it. I have updated that version here:

>

> http://parts.tigris.org/source/browse/*checkout*/parts/trunk/parts/par

> ts/mappers.py?revision=449&content-type=text%2Fplain

>

> end of the file line 694-800


At least just from looking at the code, I don't see how it quotes cmd[0] or does anything specifically to prevent this problem.


> As I recall Steve tweak the actions to help make it smarter ( I

> believe it was the addition of a print action value), to help separate

> the value the temp file will call from the case when it need to create

> the temp file


Yes, that part is already in SCons.


> I forget the tweak we made, but it fixed the generation issue we found in our builds. I believe the issue is fixed with left overs lnk files in the temp directory. We use TempFile heavily in Part, or tweak the mslink and msvc tool use TEMPFILEMUNGE to deal with CC CXX and LINK commands as we found cases in our build in which the command line of these go way to long.

>

> The issue in 1705 I have not seen in our usages of it with scon 2.1 or

> 2.2


Can you try the simple test case from the bug report?  I modified it a little bit to run on modern SCons versions.  This test fails for me on the trunk.  The fact that env['AR'] has spaces is what makes it fail.

=====
# Test MSVC tempfile handling

import os.path

d = os.path.join( "xxxxx", "yyyyyy", "zzzzzz", "aaaaaa" )
try:
   os.makedirs( d )
except OSError, e:
   pass


# Create a bunch of source files in xxxxx numSources = 100 sources = [] for x in range(numSources):
   source = os.path.join( d, str(x)+ ".cpp" )
   sources += [ source ]
   print "Making source: ", source
   open( source, 'w' ).close()

env = Environment()

# Path to alternative build tools:
env['AR'] = env.WhereIs("lib.exe")
print "Using %s for AR"%env['AR']

# If issue #1705 is still active, this will fail, # saying 'C:\Program' is not recognized as an internal or external command # because it doesn't get quoted properly.
env.StaticLibrary( os.path.join( d, "foo.lib" ), sources )

=====


--
Gary
_______________________________________________
Scons-dev mailing list
Scons-dev at scons.org<mailto:Scons-dev at scons.org>
http://two.pairlist.net/mailman/listinfo/scons-dev
_______________________________________________
Scons-dev mailing list
Scons-dev at scons.org<mailto:Scons-dev at scons.org>
http://two.pairlist.net/mailman/listinfo/scons-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://two.pairlist.net/pipermail/scons-dev/attachments/20120903/daa890a1/attachment.htm>