[Scons-dev] SCons performance investigations

[Bill Deegan]

See comments inline..

On Thu, Jul 27, 2017 at 10:39 AM, Russel Winder <russel at winder.org.uk>
wrote:

> On Thu, 2017-07-27 at 08:57 -0700, Bill Deegan wrote:
> > Russel,
> >
> > You're forgetting (I think) that SCons does implement (some) of
> > AutoConf/AutoMake's functionality.  Could it be better/more complete? Of
> > course. Is it usable (and used)? very much so.
>
> From what I can see the end user has to handle pkgconfig more or less
> manually, i.e. ParseConfig is an unsubtle tool, and may well put people
> off.
> Meson's way of doing this is somewhat better. Also, the config.h file
> stuff is
> there but a tad awkward. Again Meson
> seems to do this better. I spotted https://bitbucket.org/scons/
> scons/wiki/Usin
> gPkgConfig
> but code should never be on a wiki, it should always be in a DVCS hence
> https:
> //bitbucket.org/russel/sconspkgconfig
>
> Other tools have a GNU install toolkit so I started
> https://bitbucket.org/russ
> el/sconsgnuinstallutils
> <https://bitbucket.org/russ%0Ael/sconsgnuinstallutils>
>
> And the SCons platform determination is not sufficiently fine grained. I
> started something here: https://bitbucket.org/russel/sconsplatformutils
>
> But no-one else in SCons-land seems interested in these things that come as
> standard in CMake and Meson. Thus small but sometimes decisive negative
> points
> for SCons.
>

I don't think that's a fair statement.


>
> > Here's how I see the next two releases:
> > 3.0 - PY2/3 compat, VS 2017, some minor performance improvements.
> > Post 3.0 - Migrate to GitHub
>
> I think the move from Python 2 to Python 3 has, for me anyway, been a huge
> success. Being able to use Python 3.5 constructs in SConstruct and
> SConscript
> files is a huge plus.
>
> I am not entirely sure about a switch to GitHub. Having bookmarks and
> BitBucket supporting them for pull requests has re-enamoured me of
> Mercurial.
> The single biggest issue is "hg pull" doesn't update the working tree
> unlike
> "git pull". I have really come to hate "git pull" exactly because of this!
>

git fetch is what you're looking for.  From the manpages..

       git-pull - Fetch from and integrate with another repository or a
local
       branch
       git-fetch - Download objects and refs from another repository



>
> > 3.1 - Focus on performance. I think (per the email/wiki page), there's
> lots
> > of opportunities to speed up SCons, without losing it's SCons'ness..  (If
> > we swap out sconsign implementations we will need to have the version
> > string be 4.0 as we're breaking compatibility)
>
> I am entirely up for speed improvement, but all my projects are really
> rather
> small in comparison to those who comment on project size on the lists, so I
> have never has this as a frustration.
>
> If the .sconsign.dblite is replaced by something faster then yes I think we
> are making SCons 4.
>
> > Of course everything depends on how much time/effort the community (and
> > myself) are able to provide towards improving SCons.
>
> Ever the same for FOSS projects with no big corporate backers.
>
> For myself I have to prioritise the rewrite of the ACCU conference
> submission
> and review system now, so I have to work effectively full time on that. I
> can
> always chip in bits on SCons stuff, but nothing even close to full time.
>

Sure.. Any help is welcome, even if it's not code but making the community
aware of other tools and their relative strengths/weeknesses..


>
> > Things I'd really like to see happen in the not too distant future:
> > * Revamp platform/toolchain - Right now it's left to the user to
> implement
> > logic to say, if I have these 3 tools, then use all of them, otherwise
> use
> > these tools.  Think mingw vs msvc on win32 for example
>
> Not sure what this is saying. Sorry, I am probably just missing something
> obvious.
>

Right now you can add a list of tools for SCons to load and if they are
available they'll be configured.
For example you end up with mingw linker and msvc compiler.. which I'm not
sure would work.

So the idea is that you can specify a set of tools which all must be
present to use, if one is not, then go to the next list of tools, say clang
and if not all the clang tools you need are there then use gcc based tools.


> I wonder if there is some need to look at the C, C++, Fortran, D
> integration,
> especially the linker. I think there are a lot of hacks in there.
>

If you're talking about the smart linker which attempts to set up the link
line based on what type of objects are feeding into the link, I'm sure
there's room for improvement. Though it doe work at this point if you're
using the "batteries" included tools.



>
> > * Add support for components - For example a library can specify what
> > include paths, defines, libraries, compiler flags would be used by
> programs
> > (or components) which use itself.  Parts provides some of this I beleive.
> > I would want this as an available API, but not mandatory to use SCons
>
>
> > * Handle pulling components from repos.. cmake,gradle, etc can do this, I
> > think it's pretty darn useful especially where you may have a project
> with
> > many repos and you focus on just one.
>
> Has the "integration" with CVCS repositories as locations of sources been
> deprecated/removed? Should DVCS integration be included? Go and Cargo both
> support getting code for compilation from Git, Mercurial and Bazaar (soon
> to
> be replaced by Breeze) repositories.
>

Yes deprecated a long time ago. Remaining dregs removed in 3.0.
This would require a rethink and a better approach, and would hopefully
support git, hg, and be extendable to most other CVS and DVCS's.


>
> C++ now has Conan supported by JFrog. Without support for this, SCons will
> fall behind – I believe CMake and Meson are gaining built in support for
> this
> C++ source package repository.
>
> I was trying to create a tool to handle Dub repository packages for D
> projects, and it looks like the idea has legs. I have had to stop working
> on
> it due to having to work on the ACCU Conference stuff. I am sure something
> could be put together for Bintray packages. Cargo/Rust is arguably the
> better
> model to follow that Go.
>
> > * Taskmaster improvements (see the need for speed wiki page, and also
> > comments by Jason about Greg Noel's TaskmasterNG which Greg (who's long
> > since left the project worked on, but never contributed, or shared
> sources
> > with anyone).
>
> I had assumed this had all gone with Greg when he departed the project.
>

Yes. Nothing above contradicts that.


>
> > Most of these bullets are in the nice to have.
> > I think improving performance is in the must have as it's the top
> complaint
> > I hear about SCons.
>
> I still believe getting the issues out of Tigris and into BitBucket (or
> GitHub
> if the project is switched from Mercurial to Git and moved).
>

Yes once we switch to github.


>
> I think making use of some of the CI tools such as Travis-CI, Codeship,
> etc.
> would be good for the SCons project. This is as much about publicity and
> marketing as it is about CI.
>

We're using buildbot now..
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist2.pair.net/pipermail/scons-dev/attachments/20170727/9099023c/attachment-0001.html>