elegant chaos

The Swift Package Manager is a package manager and build system, which comes bundled along with Swift itself.

If you’re developing on a Mac, the obvious route to go down is to use Xcode, which has its own fairly capable build system.

Xcode only runs on the Mac though, so clearly you can’t rely on that if you want to be able to build on Linux. The fact that SwiftPM is present wherever Swift is - on Linux builds as well as Mac builds - is a distinct advantage, and makes it an attractive alternative for cross-platform work, minimising the amount of faffing around required when setting up on a new machine.

Unfortunately, whilst SwiftPM is a decent package manager, at the moment it is quite basic as a build system.

It has some nice clean aspects to it, relying on conventions to infer which code to build, which makes it very easy to get going with. However, it lacks some facilities from Xcode that are fairly essential for substantial products:

• specifying standard build settings in external files
• being able to hook in code generators or other pre-build steps
• being able to package up the built binary with other resources
• being able to execute other arbitrary code as build phases
• being able to code sign, archive, or post-process the final product
• being able to install the final product locally, upload, or submit it somewhere

The SwiftPM team have stated their intention to address many of these limitations, but right now the details of what, how or when are thin on the ground.

As I’ve been experimenting with Swift on Linux, and looking into Xcode-alternatives, I didn’t want to wait for jam tomorrow, so I started wondering what it would take to add the missing capabilities, and how best to attempt it.

The result of this is a tool I’ve imaginatively called builder.

Builder

Builder is an experimental overlay on SwiftPM.

In thinking about its design I considered integrating it into SwiftPM (which is open-source, after all), but that would have meant asking people to custom-build SwiftPM in order to use it, which seemed like a high barrier. Instead, it is built as a Swift package itself. You can add it as a dependency to your own product, and bootstrap your project build by first building & running builder using SwiftPM¹.

The basic capabilities provided by Builder are:

• running code at configuration time
• specifying build settings
• complex builds which chain together multiple build steps
• calling out to external tools during the build

Whilst achieving this, the guiding principle has been that at all times we should use SwiftPM itself to fetch, build, and run any external code involved in your build.

If your tool infrastructure itself is all written in cross-platform Swift², it should be possible to bootstrap your project build on any platform that has the swift command installed, just by:

• cloning your project
• using swift run to fetch and run builder
• using builder to build your project

Trying It Out

You can find Builder on github here.

As well as the code for builder, the repo includes an example project that you can try it out on.

To give it a go:

git clone https://github.com/elegantchaos/Builder.git
cd Builder/Example
swift run --package-path ../. --static-swift-stdlib -Xswiftc "-target" -Xswiftc "x86_64-apple-macosx10.12" Builder run

What this does is to:

• clone builder
• build and run the tool itself, which…
• …builds then runs an example product

How It Works

Builder works by looking for a special target called Configure in the package that it’s trying to build.

This target is defined in package’s Package.swift file, along with all its other targets and dependencies.

If it finds this target, it uses swift build to do the following:

• build the Configure target
• run the resulting executable and capture its output
• parse this output to obtain a dictionary describing how to build the actual package
• extract compiler and linker settings from the configuration dictionary

It then looks in the configuration for a scheme³ matching the name that was supplied on the command line (or build if none was supplied). If present, this scheme describes a set of commands to execute in order to perform the build for that scheme.

These commands can consist of:

• build: build a product using swift build, applying the build settings from the configuration
• test: test a product using swift test, applying the build settings from the configuration
• run: run a product using swift run, applying the build settings from the configuration
• scheme: perform another nested scheme
• anything else: treat the named command as a dependency; build and run it with swift run, passing the arguments specified in the configuration

The real flexibility comes from that final option.

If Builder encounters a command name that it doesn’t recognise, it assumes that it’s an external tool that it needs to build. It looks for it in the dependencies, and then uses swift run to fetch the code, build and run it. In this way, not only can you run arbitrary tool code, but if it’s written in Swift, you don’t have to ask your users to first install it⁴.

Pros And Cons

This is an experiment, it’s at a very early stage, and it definitely has some pros and cons.

On the plus side, it allows you to layer a more complex build system onto SwiftPM. The SwiftPM package format already has the capability to include arbitrary Swift code in it, but that code can’t depend on external dependencies. By moving the configuration code into its own target, builder gets round this. It also has the potential to eliminate the need for non-deterministic swift from the Package.swift file, which may be a bonus for tools that want to manipulate the file programmatically.

On the negative side, having to build another target is a performance hit. Worse, since Builder is a layer on top of it, SwiftPM has no knowledge of what Builder is doing. It can’t reason about the actions that any external tools are performing - for example in order to prevent them from running when nothing has changed. This is a pragmatic trade-off right now. If SwiftPM had hooks into its underlying dependency system⁵, in a way that allowed external tools to describe the dependency relationships they introduced, then Builder would be able to integrate with SwiftPM more cleanly (or wouldn’t be needed at all). Maybe one day…

For More Information

This has deliberately been a brief tour.

There is more (and more rambling!) information to be found on the github repo, including a list of flaws and ideas.

I’m still very much tinkering with this system, and lots of things are in flux - including a lot of the terminology which is quite confusing since it overloads terms that we already know from SwiftPM or Xcode, in ways that may not quite match expectations.

There are also plenty of rough edges. A system like this would really come into its own with a rich set of tools already built around it, minimising or even eliminating the need to write any tool or configuration code yourself. Widespread adoption would encourage these tools, but at this stage you largely have to use your imagination - which can make the whole concept appear a lot less user-friendly than it actually has the potential to be.

If you have any feedback, I’d love to hear it.

I’m tracking issues and ideas myself on Github, or there is a thread on the Swift forums.

Footnotes