[Scons-dev] Adding SCons support for VCPkg

Sat Mar 30 02:30:58 EDT 2019

> SOURCE:
> Am I safe in assuming the source would be a github repo?
> Or could it also be a zip file you download?  Would you specify the version/branch/tag for such?
> Would you want newer versions to be retrieved as they are published?

It could be a GitHub repo, or a plain archive, but you don’t reference it by URL…just by name. The way VCPkg is designed, the VCPkg repo includes a build script (including the download URL) for each package, and then you just say (e.g.) “vcpkg install boost-static-assert” to download and build the version of the package that your copy of VCPkg currently knows about. Thus, the entire package ecosystem is effectively versioned in the VCPkg Git repo itself, and you upgrade to newer package versions by checking out a newer commit in your copy of the VCPkg repo, thereby advancing to a newer snapshot of the world.

Thus, the usual way to control versioning of the packages used by your project is to add the VCPkg Git repo as a submodule of your project. Then, the specific VCPkg commit that you select for your submodule determines the package versions that you get, and when you want to update to newer package versions, you update the commit that your submodule is tracking, and then you get newer versions of all your packages. I believe there are ways to force a different version of a specific package, but I’ve not experimented with these yet.

> TARGET:
> I think you'll have to list at least all the files which other parts of the build could depend on.
> So headers, sources, libraries, data files?

Yeah, I was afraid of this. It doesn’t look like VCPkg currently provides a way to compute these before doing the actual build. Once the build of a project is complete, you can find a listing of all files produced by building that package in <VCPkg-root>\installed\vcpkg\info\<proj>_<version>_<arch>.list.

Hopefully it isn’t too hard to generate this list without doing the build. I’ve started this thread<https://github.com/Microsoft/vcpkg/issues/5872> on the VCPkg issue tracker to get the discussion going regarding what would be involved to get the list of built files without building.

> Almost too bad vcpkg doesn't include a sconscript.. that'd make it simpler to bootstrap inside scons.

Hmmm, interesting suggestion. That did get me thinking…considering the possibility of adding the Scons integration in the VCPkg repo itself, rather than adding it into Scons. This would mean putting the VCPkg SCons tool in the scripts/buildsystems directory, alongside of the existing scripts for integrating with CMake and msbuild. Though, a downside of this approach is that Scons users would have to jump through some clunky hoops to enable VCPkg support. I think you’d have to say the following to enable VCPkg support, given a VCPkg repo cloned at submodules/VCPkg:

env = Environment(tools = [‘default’, ‘VCPkg’], toolpath = [ ‘#/submodules/VCPkg/scripts/buildsystems’ ])

Whereas, if Scons natively knows about VCPkg, then all a user would need to do is to set a construction variable pointing at the VCPkg root directory (which seems a lot tidier):
env = Environment()
env[‘VCPKG_ROOT’] = ‘#/submodules/VCPkg’

Thus, I’m still leaning towards wanting to add it to Scons itself. Does that seem reasonable?

Ryan

Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10
[HxS - 18332 - 16.0.11405.31450]

From: Bill Deegan<mailto:bill at baddogconsulting.com>
Sent: Saturday, March 16, 2019 8:13 PM
To: SCons developer list<mailto:scons-dev at scons.org>
Subject: Re: [Scons-dev] Adding SCons support for VCPkg

Ryan,

Interesting project.
Likely something more generic (with per platform logic) might be optimal.
That said we might not want to throw away the good waiting for the perfect..
I believe other build tools have similar existing functionality, so this could certainly be an interesting feature to add.

SOURCE:
Am I safe in assuming the source would be a github repo?
Or could it also be a zip file you download?  Would you specify the version/branch/tag for such?
Would you want newer versions to be retrieved as they are published?

TARGET:
I think you'll have to list at least all the files which other parts of the build could depend on.
So headers, sources, libraries, data files?

Re bootstrapping:
Can vcpkg be installed via chocolatey?
I'm thinking minimally the default should not be to download and bootstrap, but an optional flag could indicate to do so.
Thought it could get pretty messy.
Almost too bad vcpkg doesn't include a sconscript.. that'd make it simpler to bootstrap inside scons.

-Bill

On Sat, Mar 16, 2019 at 2:38 PM Ryan Saunders <jediry at outlook.com<mailto:jediry at outlook.com>> wrote:
Hi SCons developers!

I am working on a SCons tool integrating the new VCPkg package manager from Microsoft (https://github.com/Microsoft/vcpkg<https://nam03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMicrosoft%2Fvcpkg&data=02%7C01%7C%7C6f00fb627efc4247fc3308d6aa867cb7%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636883891919792641&sdata=fmge1E6EDxtP8q0wJeo2v39989Ir%2BfamSiwB4XLFtpY%3D&reserved=0>). VCPkg makes it pretty straightforward to acquire 3rd party libraries and header files…”vcpkg install boost” is basically equivalent to “apt-get install boost-devel” on Ubuntu. Personally, I’m excited about this, as acquiring and configuring development libraries on Windows has been painful, historically. VCPkg works on Windows, Linux and MacOS (though I've only tried it on Windows so far.)

(Note: I work for Microsoft, but not on anything related to VCPkg or Visual Studio…this is strictly a side project for me.)

I envision it working something like this:

env = Environment(tools = [‘default’, ‘vcpkg’])
env.Replace( VCPKGROOT = ‘#/some/path/vcpkg’ )  # Override default root directory of #/vcpkg
…
env.VCPkg( ‘boost-smart-ptr’ )
env.VCPkg( ‘FreeImagePlus’ )

…possibly with some as-yet-unspecified integration with the Configure context, so that packages can be auto-installed if not found.

Basically, my goal is to be able to “git clone” a project onto a machine with just Git, VS and Python installed, and then to build successfully (including acquiring all necessary libraries) simply by running “scons”.

Some questions I have:

  *   Is this something that would be acceptable/desirable as part of SCons core, or should I be thinking of this as a tool that lives in its own repo?
  *   It occurs to me that, while this does work on Linux/MacOS, these systems have their own native package management solutions for acquiring development libs. So, perhaps this would be better as a sort of “meta-package-manager” tool that uses VCPkg on Windows, and something native-ish on other systems. On the other hand, those tools usually require root access to use, so maybe that’s not a good idea. Thoughts?
  *   In my current prototype, if VCPkg is not found, then I’m downloading it and building it via its “bootstrap” script. However, I’m not sure where the best place to do this is…I could do it in my tool’s generate() function, which seems logical, but this appears to be called immediately upon loading the tool, before the SConscript has an opportunity to override VCPKGROOT. What do you recommend?
  *   I’m having difficulty deciding what “source” and “target” should be for my Builder.

     *   Based on scouring the ‘net, it seems that I want to use a Value() as my “source”…probably just the string name of the package to be installed. Does that sound right?
     *   For “target”, would it be better to exhaustively enumerate all the outputs of “vcpkg install <pkg>” and list these as “targets”, or should I find some single file that gets produced by installing a package (e.g., the package manifest), and use this as a “sentinel” for the whole package? The latter is clearly simpler, but could this result in an incomplete incremental rebuild after upgrading to a newer version of the package?

Any advice you can provide would be appreciated!
Ryan

_______________________________________________
Scons-dev mailing list
Scons-dev at scons.org<mailto:Scons-dev at scons.org>
https://pairlist2.pair.net/mailman/listinfo/scons-dev<https://nam03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpairlist2.pair.net%2Fmailman%2Flistinfo%2Fscons-dev&data=02%7C01%7C%7C6f00fb627efc4247fc3308d6aa867cb7%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636883891919802649&sdata=znrlQZFsMjrkFTRzwI4lvsR%2F4oRSwodyt%2BSYwWSjbz4%3D&reserved=0>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist2.pair.net/pipermail/scons-dev/attachments/20190330/4747413e/attachment-0001.html>