[Scons-dev] SCons-like build system
voidmb
voidmb at gmail.com
Thu Jan 15 05:01:37 EST 2015
Hi Jason,
/>> //I am interested in how you deal with target platforms and tool
section ( I see that you suggest that tool version selection exists)./
I will try to explain.
Aqualid tries to load all tools from following locations:
On Windows:
X:\PythonXX\Lib\site-packages\aqualid\tools
%USERPROFILE%\AppData\Roaming\Python\PythonXX\site-packages\aqualid\tools
%USERPROFILE%\.config\aqualid\tools
On Unix:
/usr/lib/pythonX.Y/site-packages/aqualid/tools
$PYTHONUSERBASE/lib/pythonX.Y/site-packages
$HOME/.config/aqualid/tools
User may specify additional paths in defualt config file
(~/.config/aqualid/defualt.cfg) or via CLI option '-I'
An example of g++ tool:
|@tool('c++', 'g++', 'gxx', 'cpp', 'cxx')||
class ToolGcc( ToolCommonCpp ):||
||||
|| @classmethod||
|| def setup( cls, options, env ):||
|| # it usually searches for programs and sets programs' options||
|| # if tools can't be found then it raises NotImplementedError||
||||
|| @classmethod
def options( cls ):
# returns tool's specific options
returns options
|| def __init__(self, options ):||
|| # intinitializes other tool's options after setup(..)
successfully finished||
||||
||#//-------------------------------------------------------//||
|| # tool's methods, which can be called from scripts||
||||
|| def Compile( self, options ):||
|| return GccCompiler( options )||
||||
|| def CompileResource( self, options ):||
|| return GccResCompiler( options )||
||||
|| def LinkStaticLibrary( self, options, target ):||
|| return GccArchiver( options, target )||
||||
|| def LinkSharedLibrary( self, options, target, def_file = None ):||
|| return GccLinker( options, target, shared = True )||
||||
|| def LinkProgram( self, options, target ):||
|| return GccLinker( options, target, shared = False )|
Decorator @tool is used to set names for the tool.
Any name can used from scripts to get access to tool's methods.
Different tools may share names. For example MS Visual C++:
|@tool('c++', 'msvcpp','msvc++', 'cpp', 'cxx')|||
|||class ToolMsVCpp( ToolMsvcCommon ):|||
||
Assume the following build script:
|||cpp = Tool('c++')|
In this line Aqualid calls *setup(..)* methods of each tool.
If no exceptions were raised then corresponding tool instance will be
returned to the build script.
Aqualid also checks that key properties do not conflict with user settings.
For example, if user set target_os to 'windows' but we have only gcc
compiler for Linux then this tool will be skipped.
Aqualid supports customization different environments for different
versions of a tool.
User may define special pre-setup methods for a tool.
For example below is pre-setup script for GCC on my system:
~/.config/aqualid/tools/setup_gcc.py
|@toolSetup('g++', 'gxx', 'gcc')||
||def setupGcc( options ):||
|| if options.cc_name.isSetNotTo( 'gcc' ):||
||||raise NotImplementedError()||
||||
|| if |||options.target_os||| == 'windows':||
||||
|| target_arch = options.target_arch||
||||
|| if not target_arch.isSet() or target_arch == 'x86-64':||
|| options.gcc_prefix="x86_64-w64-mingw32-"||
||||
|| elif target_arch == 'x86-32':||
|| options.gcc_prefix="i686-w64-mingw32-"||
||||
|| else:||
|| raise NotImplementedError()||
|
If I set target_arch to 'arm' via command line or in a build script then
no C++ tool will be found.
Pre-setup scripts can be global (for all users) or local (only for
current user).
The main purpose of pre-setup scripts to set OS environment (like
correct PATH),
But tool's setup method use the OS environment to find tool itself.
And Aqualid checks that key options don't conflict.
Tool selection logic:
0. Get a list of tools with the same name
1. Pick the next tool from the list
2. Get list of pre-setup methods for the tool
3. If exists pick the next pre-setup method from the list (2).
4. Run pre-setup method for a tool. if failed go to the step 3 or step 1
5. Run setup method of the tool. if failed go to the step 3 or step 1
6. Compare tool's key options with user's settings. if there is a
conflict go to step 3 or step 1
7. Create the tool instance. if failed go to the step 3 or step 1
8. Return the tool
/>> //As the build gets larger the time to rebuild when one file changed
take much longer./
I agree. Running Aqualid from the top directory could take much time,
but it supports sub-projects.
You may build only specific targets running Aqualid from a sub-project
directory.
/>> //I have number of other suggestion on what could be done in you DB
to store certain states to allow rebuild to get going very fast. Some
other these I have not done in Parts yet ( as I don’t have “easy”
control over the main execution logic )//
/I'm very interested. Can you share you suggestions?
Thank you.
Best regards,
Constantine
On 01/14/15 17:42, Kenny, Jason L wrote:
>
> HI,
>
> Looks nice. It will be nice to see more documentation on this. I am
> interested in how you deal with target platforms and tool section ( I
> see that you suggest that tool version selection exists).
>
> Another test which would be useful for me is how well does this scale.
> You show an example with lots of node in a given aql file. The issue I
> see with this is that in my case I have 1000+ build (parts) files that
> need to be loaded. The some of the total targets (objs,so|dll, etc)
> would be about 200K. One issue with this is that it takes time to load
> each file. What I have seem with SCons, and looks like you have a
> similar issue, is that to load a build file, means it has to be
> processed/executed by python. This takes time as user tend to scan the
> disk to get the source files and python code is executed. As the build
> gets larger the time to rebuild when one file changed take much
> longer. Parts ( currently) does some work to avoid loading build files
> to SCons to reduce the startup time to get a build going. What would
> take Scons 2-3 minute to get started with a build, now takes 3 to30
> second ( depending on states…).
>
> Just something to think about. I have number of other suggestion on
> what could be done in you DB to store cetain states to allow rebuild
> to get going very fast. Some other these I have not done in Parts yet
> ( as I don’t have “easy” control over the main execution logic )
>
> Jason
>
> *From:*Scons-dev [mailto:scons-dev-bounces at scons.org] *On Behalf Of
> *Constantine
> *Sent:* Wednesday, January 14, 2015 1:38 AM
> *To:* SCons developer list
> *Subject:* Re: [Scons-dev] SCons-like build system
>
> Hi Jason,
>
> >> /What I would like to know as I did not see it in the example right
> off is, how do you deal with lots of components./
>
> I guess you mean this
> example:https://github.com/aqualid/aqualid/wiki/An-introduction-to-Aqualid#build-sub-projects
>
> lib = Script('../lib/make.aql')['lib']
>
>
>
> dll = tools.cpp.LinkSharedLibrary( 'tool_api.cpp', lib, target = 'toolapi',
>
> cpppath = lib.options.api_cpppath,
>
> api_cpppath = '.',
>
> cppdefines = 'MAKING_LIBRARY' )
>
> We call function 'Script()' to build component 'lib'.
> This function returns a dictionary of all local variables defined
> inside the executed script. Then we get the 'lib' Node.
>
> To build a shared library we use the lib's CPPPATH:
> cpppath = lib.options.api_cpppath,
>
> Option 'api_cpppath' will have correct CPPPATH to use headers of this
> library.
>
> Inside component 'lib' we are free to change directories layout
> whenever we want.
>
>
> />> //Also your benchmarks… is this 32-bit or 64-bit./
>
> I used 64-bit Linux, 2 cores CPU and available 1.3 GiB of RAM, no swap
>
> Thank you.
>
> Best regards,
>
> Konstantin.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist2.pair.net/pipermail/scons-dev/attachments/20150115/2fba0dd5/attachment-0001.html>
More information about the Scons-dev
mailing list