Building On Swift
March 06, 2018

The Swift Package Manager is a package manager and build system, which comes bundled along with Swift itself.

If you’re developing on a Mac, the obvious route to go down is to use Xcode, which has its own fairly capable build system.

Xcode only runs on the Mac though, so clearly you can’t rely on that if you want to be able to build on Linux. The fact that SwiftPM is present wherever Swift is - on Linux builds as well as Mac builds - is a distinct advantage, and makes it an attractive alternative for cross-platform work, minimising the amount of faffing around required when setting up on a new machine.

Unfortunately, whilst SwiftPM is a decent package manager, at the moment it is quite basic as a build system.

It has some nice clean aspects to it, relying on conventions to infer which code to build, which makes it very easy to get going with. However, it lacks some facilities from Xcode that are fairly essential for substantial products:

The SwiftPM team have stated their intention to address many of these limitations, but right now the details of what, how or when are thin on the ground.

As I’ve been experimenting with Swift on Linux, and looking into Xcode-alternatives, I didn’t want to wait for jam tomorrow, so I started wondering what it would take to add the missing capabilities, and how best to attempt it.

The result of this is a tool I’ve imaginatively called builder.

Builder

Builder is an experimental overlay on SwiftPM.

In thinking about its design I considered integrating it into SwiftPM (which is open-source, after all), but that would have meant asking people to custom-build SwiftPM in order to use it, which seemed like a high barrier. Instead, it is built as a Swift package itself. You can add it as a dependency to your own product, and bootstrap your project build by first building & running builder using SwiftPM1.

The basic capabilities provided by Builder are:

Whilst achieving this, the guiding principle has been that at all times we should use SwiftPM itself to fetch, build, and run any external code involved in your build.

If your tool infrastructure itself is all written in cross-platform Swift2, it should be possible to bootstrap your project build on any platform that has the swift command installed, just by:

Trying It Out

You can find Builder on github here.

As well as the code for builder, the repo includes an example project that you can try it out on.

To give it a go:

git clone https://github.com/elegantchaos/Builder.git
cd Builder/Example
swift run --package-path ../. --static-swift-stdlib -Xswiftc "-target" -Xswiftc "x86_64-apple-macosx10.12" Builder run

What this does is to:

How It Works

Builder works by looking for a special target called Configure in the package that it’s trying to build.

This target is defined in package’s Package.swift file, along with all its other targets and dependencies.

If it finds this target, it uses swift build to do the following:

It then looks in the configuration for a scheme3 matching the name that was supplied on the command line (or build if none was supplied). If present, this scheme describes a set of commands to execute in order to perform the build for that scheme.

These commands can consist of:

The real flexibility comes from that final option.

If Builder encounters a command name that it doesn’t recognise, it assumes that it’s an external tool that it needs to build. It looks for it in the dependencies, and then uses swift run to fetch the code, build and run it. In this way, not only can you run arbitrary tool code, but if it’s written in Swift, you don’t have to ask your users to first install it4.

Pros And Cons

This is an experiment, it’s at a very early stage, and it definitely has some pros and cons.

On the plus side, it allows you to layer a more complex build system onto SwiftPM. The SwiftPM package format already has the capability to include arbitrary Swift code in it, but that code can’t depend on external dependencies. By moving the configuration code into its own target, builder gets round this. It also has the potential to eliminate the need for non-deterministic swift from the Package.swift file, which may be a bonus for tools that want to manipulate the file programmatically.

On the negative side, having to build another target is a performance hit. Worse, since Builder is a layer on top of it, SwiftPM has no knowledge of what Builder is doing. It can’t reason about the actions that any external tools are performing - for example in order to prevent them from running when nothing has changed. This is a pragmatic trade-off right now. If SwiftPM had hooks into its underlying dependency system5, in a way that allowed external tools to describe the dependency relationships they introduced, then Builder would be able to integrate with SwiftPM more cleanly (or wouldn’t be needed at all). Maybe one day…

For More Information

This has deliberately been a brief tour.

There is more (and more rambling!) information to be found on the github repo, including a list of flaws and ideas.

I’m still very much tinkering with this system, and lots of things are in flux - including a lot of the terminology which is quite confusing since it overloads terms that we already know from SwiftPM or Xcode, in ways that may not quite match expectations.

There are also plenty of rough edges. A system like this would really come into its own with a rich set of tools already built around it, minimising or even eliminating the need to write any tool or configuration code yourself. Widespread adoption would encourage these tools, but at this stage you largely have to use your imagination - which can make the whole concept appear a lot less user-friendly than it actually has the potential to be.

If you have any feedback, I’d love to hear it.

I’m tracking issues and ideas myself on Github, or there is a thread on the Swift forums.


Footnotes

  1. Alternatively, if you prefer the convenience of having it available everywhere, you can clone it as a standalone project, build it, and install it into your path somewhere. 

  2. Clearly, many large projects are going to rely on tools that aren’t going to be written in Swift, or that have to be manually installed. That’s a problem that is beyond the scope of any build system to solve, but it’s a scenario that builder can cope with, since you can easily write a build phase in Swift which calls out to external tools. My aim though was to make it at least possible to evolve a rich ecosystem of Swift-only build tools. 

  3. This is a bit like an Xcode scheme, but not quite the same. I should probably find a different name for it. 

  4. with brew, apt-get, npm, or whatever. Obviously if you need to install and run something from one of those other ecosystems, you can. You just write a Swift script which does it. When you do that, you potentially open yourself up to platform variations which make your code less portable - but that’s your choice. 

  5. it uses llbuild under the hood, I believe. 

« Decoding Dictionaries in Swift More Build Adventures »
Got a comment on this post? Let us know at @elegantchaoscom.