[Scons-dev] Binding Environments to Nodes

[Kenny, Jason L]

I am probably missing something.. but from I can see,

You are wanting to modify the SCons NodeInfoBase or the class based on it such as FileNodeInfo to has a new ext_sig value ( in addition to the current csig, timestamp and size values)
You then need tweaks to make sure the node class stores some value for ext_sig when we storing data about that is put in the ninfo member. ( I would need to look up the functions that do this in the node classes)

This way when you get the decider function called you can get data that was stored to see if something changed. I am unclear on what you need an environment object for here. I would guess that what you want to be able to attach some function to the object so that when the Scons can get the signature data to store, this function can be called and generate some data to be stored as this ext_sig. This suggests that you want to add some API, in your copy of SCons, to allow some sort of global or per node registration of some type of callable object.  


Just some quick thoughts..

Jason


-----Original Message-----
From: scons-dev-bounces at scons.org [mailto:scons-dev-bounces at scons.org] On Behalf Of Eric W. Anderson
Sent: Wednesday, September 19, 2012 3:10 PM
To: SCons developer list
Subject: Re: [Scons-dev] Binding Environments to Nodes

What I'm doing (the exact code is here
https://bitbucket.org/eanderso/scons-custom-sig/compare/default..scons/scons:default)
 is an internal change in Scons.Node.FS.File.  So in that context, I'm getting the environment with self.get_env().  That returns self.env if set, and otherwise the/a DefaultEnvironment.  I can't tell you why, but I can say from looking at the output that may operations are done on the Node without env having been set.  Here's a snippet of my SConstruct:

   env = Environment()

     ... bunch of stuff ...

   b=Builder(action="bash myscript.sh $SOURCE $TARGET")
   env.Append(BUILDERS={'B' : b})
   env.Export()

   i=File('myinput.txt')
   x=env.B(['myoutput.txt'],[i])

This leads to my extended signature function getting called for the following node, environment pairs, where ...d0 is the default and ...90 is env.

   Node 'SConstruct' in env <0x17997d0>
   Node 'SConstruct' in env <0x17997d0>
   Node 'myinput.txt' in env <0x17997d0>
   Node 'myinput.txt' in env <0x17997d0>
   Node 'bash' in env <0x17997d0>
   Node 'bash' in env <0x17997d0>
   Node 'myinput.txt' in env <0x17997d0>
   Node 'bash' in env <0x17997d0>
   Node 'myoutput.txt' in env <0x1799790> 
   Node 'myoutput.txt' in env <0x1799790>

There are several different stack traces that lead to getting the default env; here's one:

   SCons.Script.main() -> _exec_main(parser, values) -> _main(parser) -> nodes = _build_targets(fs, options, targets, target_top) -> jobs.run(postfunc = jobs_postfunc) -> self.job.start() -> task = self.taskmaster.next_task() -> task.make_ready() -> SCons.Taskmaster.OutOfDateTask.make_ready(self) -> t.visited()

The env I want happens when we get to my code with the following stacks (sorry for the data dump):

   SCons.Script.main() -> _exec_main(parser, values) -> _main(parser) -> nodes = _build_targets(fs, options, targets, target_top) -> jobs.run(postfunc = jobs_postfunc) -> self.job.start() -> task = self.taskmaster.next_task() -> task.make_ready() -> SCons.Taskmaster.OutOfDateTask.make_ready(self) -> t.visited()
   SCons.Script.main() -> _exec_main(parser, values) -> _main(parser) -> nodes = _build_targets(fs, options, targets, target_top) -> jobs.run(postfunc = jobs_postfunc) -> self.job.start() -> task.executed() -> SCons.Taskmaster.OutOfDateTask.executed(self) -> t.visited()

Any thoughts would be very welcome!
-Eric

Thus spake Kenny, Jason L (jason.l.kenny at intel.com):


> I don't believe you have this correct.

> 

> A Node in SCons has a reference to environment that created it.

> 

> So as I understand it and have seen it work a call like:

> 

> F=File("foo")

>  Will have a f.env that point to the DefaultEnvironment() active at 

> the time it was called ( ideally SCons makes only one default 

> environment, but there are way you might have this get re-created)

> 

> While

> 

> Env=Environment()

> f=Env.File("foo")

> f.env now is a reference to the env variable.

> 

> Keep in mind that I found that this might not be the same environment that the builder would use to do the actions to create the node.

> 

> I should point out the node.env is not a documented API.. so it could 

> break at some point. ( not any time soon)

> 

> Jason

> 

> 

> -----Original Message-----

> From: scons-dev-bounces at scons.org [mailto:scons-dev-bounces at scons.org] 

> On Behalf Of Eric W. Anderson

> Sent: Wednesday, September 19, 2012 1:10 PM

> To: SCons developer list

> Subject: [Scons-dev] Binding Environments to Nodes

> 

> It doesn't seem like anyone else is especially fired up about this, which is OK.  There's one technical point on which I'd really like suggestions:

> 

> I'd like to have the/a user-constructed environment associated with nodes when Node.visited() is called.  As I understand it, that can (and often does) happen outside the context of any particular Builder, meaning that the only environment available is the DefaultEnvironment.  Since I want to allow user customization of what happens at that time, it makes sense to do that through a user-constructed Environment.

> 

> Any suggestions?  I don't know the inner workings of SCons well enough to have a great idea at this point.

> 

> Thanks,

> Eric

> 

> Thus spake Eric W. Anderson (andersoe at ece.cmu.edu):

> 

> > Hi Jason (and list),

> > 

> > To be clear, I'm not proposing that that particular signature 

> > function should become part of SCons.  What I want to add to the 

> > "core" SCons code is the necessary set of hooks to accept and run 

> > user-supplied signature-generating functions.  Indeed, the whole 

> > point is to allow users to supply something which we wouldn't want 

> > _in general_, but is right for their needs.  I see this as the 

> > missing half of the user-specified Decider: You can write a much 

> > broader range of functions if you can ensure that whatever information your decider needs has been gathered.

> > 

> > In my particular situation, the build tools themselves cause 

> > "spurious" changes to the source files.  Unless those changes are 

> > somehow filtered out, the files are _always_ changed, so a false 

> > rebuild occurs every single time.  That's mostly an issue of poor tool design, but I'm stuck with it.

> > 

> > -Eric

> > 

> > Thus spake Kenny, Jason L (jason.l.kenny at intel.com):

> > 

> > > Interesting idea. However here are some cons to consider.

> > > 

> > > 1) this will take longer to process ( increase startup time) as 

> > > one has to parse the file for a certain type of comment(s)

> > > 2) a number of tools process values that are in the comments

> > > 

> > > In your case you point out how a comment that is updated by a VCS tool with a value of a new date, etc can cause a false rebuild of the code. That same value can be used by another tools to make documentation updates, or define many other values. Maybe in your case this is not so. You are suggesting that we have a tool based MD5 for a given node... i.e. I want a MD5 only of the content that the tool cares about, which could be one or more different signature per node.

> > > 

> > > Jason

> > > _______________________________________________

> > > Scons-dev mailing list

> > > Scons-dev at scons.org

> > > http://two.pairlist.net/mailman/listinfo/scons-dev

> > 

> > -- 

> > Eric W. Anderson                     Electrical and Computer Engineering

> > andersoe at ece.cmu.edu                          Carnegie Mellon University

> > phone: +1-412-268-1908                                  Roberts Hall 244

> > 

> >                           PGP key fingerprint:

> >        	   D3C5 D6FF EDED 9F1F C36D  53A3 74B7 53A6 3C74 5F12

> 

> 

> 

> > _______________________________________________

> > Scons-dev mailing list

> > Scons-dev at scons.org

> > http://two.pairlist.net/mailman/listinfo/scons-dev

> 

> 

> -- 

> Eric W. Anderson                     Electrical and Computer Engineering

> andersoe at ece.cmu.edu                          Carnegie Mellon University

> phone: +1-412-268-1908                                  Roberts Hall 244

> 

>                           PGP key fingerprint:

>        	   D3C5 D6FF EDED 9F1F C36D  53A3 74B7 53A6 3C74 5F12 

> _______________________________________________

> Scons-dev mailing list

> Scons-dev at scons.org

> http://two.pairlist.net/mailman/listinfo/scons-dev


-- 
Eric W. Anderson                     Electrical and Computer Engineering
andersoe at ece.cmu.edu                          Carnegie Mellon University
phone: +1-412-268-1908                                  Roberts Hall 244

                          PGP key fingerprint:
       	   D3C5 D6FF EDED 9F1F C36D  53A3 74B7 53A6 3C74 5F12