[Scons-dev] Toolchain requirements

[Michael Haubenwallner]

On 05/25/14 19:14, Gary Oberbrunner wrote:
> I'd like to kick off a round of discussion about toolchains, so can make some progress toward a design.
> I have some preliminary thoughts.  Please comment.  Apologies for the HTML mail, let me know if this isn't readable for you.
> 
>   * Allow installing external tools (pip install or ...)
>   *
>       o scons --version (or similar) should list installed tools and toolchains
>       o missing external tools should give sensible errors
>   * Tool setup must happen before reading SConstruct somehow
>   *
>       o DefaultEnvironment and all new Environments should know about all tools
>       o alternative: lazy-construct DefaultEnvironment
>       o user-specified tools and toolchains need to be specifiable at beginning of build
>   * User should be able to set default tools and toolchains

Consider three types of "user" here:

* The upstream package source maintainer
   = Specify which language-tools are to be initialized: C89, C99, C++, C++11, JavaC, ...
     or let SCons auto-detect used languages along the list of source-files
   = Avoid the need to take care of platform- and tool-details:
     - Let the language-tool delegate to (or know about): cl.exe/gcc/xlc/hp-cc/...
     - Make additional resource-tools a no-op on platforms not using them
   = Specify package-default (used during development) language-/tool-options
     - warnings (as errors), debugging, optimization, ...
     - the language tool implementation filters unsupported flags
   = Don't care about a library to be static or shared - just define a Library:
     fex, AIX traditionally used just _one_ file target for both the static
     and the shared library (yes, shared objects can be linked statically).
   = (maybe more)

* The downstream (distro) package build/integration maintainer
   = Define the language-tool implementation to use: cl.exe/gcc/xlc/hp-cc/...
   = Define the actual tool-executable to use (PATH, ...)
   = Set the final language-/tool-options:
     - warnings (as errors), debugging, optimization, additional QA flags
     - override the package-default values
   = Define to build the Library as static and/or shared.
   = Define the search path for package dependencies.
   = (maybe more)

* The downstream (distro) package build/integration maintainer for SCons:
   = Define default language-tool implementation to use: cl.exe/gcc/xlc/hp-cc/...
     - To be used by upstream package source maintainer
   = Define the default tool-executable to use (PATH, ...)
   = Define the default search path for package dependencies.
   = (maybe more)

For each design decision, IMHO it is important to think of /who/:
- defines the defaults,
- overrides the defaults,
- prepends/appends to the defaults.

>   *
>       o unused tools shouldn't take any startup time
>   * Lazy init of tools and chains
>   *
>       o This is faster because unused tools don't matter
>       o It allows missing unused tools to not give errors, but missing used tools can (and should)
>       o But it makes configuring environments much harder for users, because they can't override or append to tool-provided variables until those exist.  This would break a lot of existing SConstructs.

Let a tool declare beforehand which variables it accepts on any platform,
without actually completely initializing the tool. Also, this allows to
define additional flags even when unused on the current platform.

>       o We need to find some kind of compromise here:
>       o
>           + Explicitly list tools required by build (where?): this should work well because only the needed tools will be initialized
>           + if nothing explicitly specified, fall back to current method
>   * Within a tool:
>   *
>       o specify dependencies on other tools
>       o detect existence on system reliably, and without modifying env
>       o
>           + need better error messaging: ability to probe silently, but also give sensible errors when needed
>       o constructor needs to allow args: version, path, ABI, etc. (this is important)
>       o allow for common setup (all C compilers, etc.) as now
>       o tools should be versioned so user can check if up to date, etc.
>   * Tool chains:
>   *
>       o either-or
>       o and
>       o collections

The Tool-/Toolchain- *architecture* should allow for using even libtool as the backend IMO.

>   * Platform
>   *
>       o How much do we need to know about the platform, for tools to initialize themselves?
>       o Cross-compilation comes into this, but may be too much to include as a general part of this project.
>       o It may be useful to define toolchains and enable/disable them by platform
>       o Of course the default toolchains need to be different by platform
>       o It may be possible for a default toolchain to just search for all tools in a particular order and pick the first, as long as the tool-dependency system is robust enough.
>   * Usability
>   *
>       o $CC etc. must never be left blank (without a prior tool-missing error message at least) - this is a common problem
>       o Must be backward compatible, at least for all common cases.
>       o Must not require any new user files (e.g. something in site_scons) for normal operation
>       o Need a clear guide on requirements for new tools
>       o
>           + how to make a a tool
>           + how to include tests
>   * Considerations
>   *
>       o "batteries included?"
>       o
>           + Each tool should do its best to set itself up, find executables, etc.
>           + What about SCons policy of not relying on $PATH?  Maybe we should relax that or have an option?

These decisions IMHO should be left open to the downstream distro build/integration maintainer,
without the need to patch SCons source by the downstream build/integration SCons maintainer.

>   *
>       o minimum magic, maximum flexibility
>       o what about single tools?  Should every tool be required to be part of a toolchain (even if it's just one tool)?  Maybe this doesn't matter much.
>   * Non-goals:
>   *
>       o New command-line args like autotools (platform, install paths, etc.).  We should build something that would enable that, but it's too much to bite off now.

Agreed: Think (design) globally, act (implement) locally.

>       o Persistence -- remember configuration on disk between runs.  This is a performance enhancement which we should address only once we know it's needed.  Better if we can design a system that's fast without needing this.
>   * References:
>   *
>       o http://www.scons.org/wiki/PlatformToolConfig (Greg's original proposal)
>       o http://www.scons.org/wiki/RevampToolsSubsystem
>       o http://www.scons.org/wiki/PlatformToolConfigAlt (my proposal from 2008)
>       o http://www.scons.org/wiki/EvanEnhancedConfigurationPackageProposal

my 2 cents, thank you!
/haubi/