Elegant Chaos

Agents, Idealism, And Capitalism

2026-04-30T00:00:00+00:00

My timeline is full of discussion about AI, and agentic coding.

It makes me sad how culture-wars a lot of it is. Very strident. Very “you’re either with us, or you’re against us!”.

For what it’s worth, I’ve been playing with agents, and I think that there are some great things about them. As a solo developer, having an agent pair-programmer has revolutionised my work.

That doesn’t mean I’m blind to the economic, social, ethical and environmental concerns. I think they are real, but I also think that they are nuanced and complicated.

In a lot of the shouting, the AI or agent seems to be a proxy for something else.

Cynical Idealism

I generally describe myself as a cynical idealist. In perhaps a trite way, I characterise this as “I believe there are better ways of doing things. I remain to be convinced that humanity is capable of finding them.”¹.

AI feels like a perfect example of this. In an ideal world, it seems to offer so much hope. In the real world, it seems to be a tool for exploitation and division.

Information Should Be Free

I have always been uneasy about copyright, patents, and the idea of intellectual property.

I want information to be free - literally and metaphorically.

In this context, I’m comfortable with the idea of an AI system reading all the things, learning from them, and using that learning to benefit humanity².

That is, after all, not a million miles away from what I do when I read, absorb, remix, half-remember, and apply ideas from other people. If I had the time, memory, and talent to read everything and synthesize it into something useful, I would.

I do not think the mere act of learning from public culture is inherently wrong, whether done by me, a machine, or a mega-brained alien who landed on earth and somehow absorbed all human knowledge.

Everything Else Should Be Free, But It Isn’t

In an ideal world, if I made something, discovered something, added to the sum of human knowledge or beauty, I think perhaps that being credited with the act should be enough.

One could argue that even wanting credit is a bit egotistical, but well, yeah - I have an ego. I’d like people know it was me. I’d like some peer recognition. I’m only human - but it would be lovely if that was all I needed, and I could trust that my contribution would be duly noted, enjoyed, built upon, or whatever.

Pragmatically, we do not live in that world.

Credit is not enough when we need money to feed, house, and clothe ourselves and those that we care for.

Trust is not enough in a world where there are bad actors who don’t subscribe to my hippy-ish vision of utopia.

When I was starting out in my career I was deeply paranoid about someone stealing my ideas, or my source code³. I was poor, and I needed the income that I thought I’d lose. I was also deeply worried about the injustice of someone else profiting from what I’d done, instead of me.

So I do not pretend this is simple. I am not trying to argue that creators should just accept being strip-mined in the name of progress. But nor am I convinced that all machine learning is morally equivalent to theft.

What I want, mostly, is for the conversation to stay nuanced enough to contain both the idealism and the pragmatism.

Is The Problem AI?

It feels to me that a lot of the anti-AI posts I see are actually people raging against capitalism.

What they are worried about is that a handful of large corporations are taking knowledge and art from those who made it, locking it away behind a paywall, and burning a whole ton of resources doing it.

They are worried also that in doing this, the corporations are also providing other corporations the perfect excuse to replace humans with robots.

In this outcome, the creators don’t get paid for creating, and the workers don’t get paid for working.

These are not properties of the technology in isolation. They are properties of the technology being deployed inside a system that disregards humans, and optimizes for profit. The ultimate result, I suspect, will be a machine that ends up eating itself. In the meantime, the question arises:

Is AI the issue? Or is it money?

The Revolution Will Not Be Hand-Coded

I’m sure you get the gist of where I’m coming from, and there’s no need for me to belabour the money thing much further. The massive resource and financial drains are real, but humanity could solve them if we worked together for the common good.

If we could fast-forward to some sort of utopian post-scarcity culture (maybe The Culture?), I think AI itself would be fine.

The fact that we can’t means that we’re in for a messy ride.

One where AI becomes very much the agent of that change, but probably one where a lot of people get hurt along the way.

We’ll either end up with a profoundly different economic model, or we’ll die trying. Literally.

This may be an argument for applying the brakes a little. It’s not an argument for stopping, or turning round - neither are an option. The genie is out of the bottle. Pandora’s box is open. Delete-cliched-metaphor-as-appropriate.

AI Is The Mother Of All Disruptors

The best we can do I think is to keep talking - sanely, rationally, calmly - about it. That, and try to find some positives. Try to evolve AI, address the problems, move forwards.

On the practical side, agentic coding tools truly are an enabler for someone like me.

I love being able to ask a coding agent a question like:

I am planning a refactor of property x in protocol Y. I want to turn it into a function instead and pass in z for context. Will this cause any problems in the current codebase? Is there anywhere we do not have an instance of Y to pass in?

That is the sort of change I might previously have just waded into, despite theoretically knowing better, only to come unstuck when it was too late to back out cleanly.

I love getting the agent to explain things to me, and suggest improvements.

I love (and am not alone in discovering) the phenomenon of dusting off previously discarded projects, because suddenly they are feasible, and actually doing them.

I love that when I am frustrated by a hole in my toolchain (as I often am), and decide to go ahead and fill it (as I often have, in the past), I actually have a hope of succeeding now, rather than falling down a deep rabbit hole.

When you scale this phenomenon across the whole of society, we may make some surprising advances, by just making a whole ton of software.

Some of it will be “bug-ridden AI-slop”. Maybe even most. This is not that different from hand-written software, most of which is also imperfect at best.

Some though will be better than what was there before, produced by really great people with a clear vision, tons of experience.

People who are independent, not subject to group-think, and now have the means to see their idea through to release.

What was it that guy said? “Here’s to the crazy ones. The misfits. The rebels. The troublemakers”. You know the guy - I forget his name.

There are so many sterile markets where a small handful of very imperfect solutions have become the standard, and have frozen out the competition. Economies of scale, monopolistic behaviour, the inherent conservatism of people - all have allowed this to become the norm in many industries.

I think we’re going to see that breaking down, gradually.

I already feel it within the software development world, and the macOS and iOS ecosystem more generally. The amount of interesting tools and apps being released seems to have kicked upwards markedly, and these are good, solid well-engineered products, addressing genuine needs.

Anyhoo

My robot master tells me that I’ve been waffling on too long.

Apparently it’s time to get back to toiling in the lithium mines, digging out the resources for making the next robot that will replace it…

Incidentally, this a recipe for permanent misery. ↩
For what little it is worth, I do put my money where my mouth is here. The vast majority of the code I’ve written for myself in the last decade or more is open source. ↩
I came to understand that this was mostly hubris, as most wasn’t worth stealing. However, I think the fear did also hold me back, and prevent me from making connections that could have been positive. ↩

Fixing The ActionBuilder Plugin

2026-02-27T00:00:00+00:00

In a previous post, I mentioned a command line tool that I built which you can point at a Swift package, and which will generate a GitHub Actions workflow file for it.

I also built a SwiftPM plugin which invokes the tool, but at some point it stopped working. Instead of running, it would just hang, for reasons unknown.

The Deadlock

The root cause turned out to be nested SwiftPM invocations.

The plugin launches ActionBuilderTool, and the tool calls swift package dump-package to inspect metadata and infer sensible workflow defaults.

I think that did work, at some point in the past, but changes in Swift Package Manager appear to have broken it. When the swift command line tool runs the plugin, which runs the tool, which runs swift, you end up with lock contention around index.lock, and everything grinds to a halt.

The fix I found was to make the tool’s swift package dump-package use an isolated scratch path inside a subfolder of the .build folder. This means that the inner invocation doesn’t fight with the outer one, and it’s still in a place that we can write to.

The workaround isn’t ideal, since building to a different scratch path means doing a lot of work again, but this isn’t a command that you run that often.

I’d like to find a better fix, but for now, it works.

To make the behavior explicit, I added a --called-from-plugin flag. The plugin uses this, but if you just want to use the tool manually, you don’t need to.

Unexpected Lack Of Items In The Bagging Area…

I thought I was done at this point, and made a new release, only to discover that it was broken and not doing anything at all.

🤔

I eventually tracked that down to an issue with nested sandboxes. The tool invocation was silently failing when the plugin called it, but the plugin wasn’t reporting anything. The failure was because it was trying to sandbox inside a sandbox. Invoking the inner swift package dump-package without sandboxing seems to sort that out - and hopefully it’s still safe since the plugin itself is running sandboxed.

Smoke Em If You Got Em

I fixed the problem, eventually, after a lot of head scratching.

In doing so I also improved the plugin diagnostics so it now reports a proper error if the tool exits non-zero, and explicitly checks that the expected workflow file was actually produced.

I then added smoke tests for:

direct tool workflow generation
tool generation in plugin-mode
actual swift package plugin invocation

Obviously I then used the tool on the plugin source code itself, to generate a workflow file which runs these tests in an action.

Hopefully that will stop me from releasing a broken plugin next time…

Other Fixes In Passing

While I was in there, I also tidied up a few other things in ActionBuilderCore, which is the underlying library that the tool uses to do the generation:

bumped the supported Swift versions and moved the minimum up to 5.10
updated the GH images that the workflow uses; some of them had become obsolete
reduced generated workflow noise and improved log handling
removed some stale dependencies and legacy code paths

There’s a bunch more polishing I’d like to do, but at least the “plugin hangs forever” issue is no longer one of the mysteries of the universe.

Part of the reason I started doing all of this in the first place is that I decided to revive and update Action Status, and I needed some actual running workflows to point it at. Which led me to my discovery of the broken plugin. And thus another rabbit hole was entered.

More about the Action Status refresh anon…

Oh, if you’re interested, you can find the plugin here.

Localizable String Catalogues in Swift Packages

2026-02-12T00:00:00+00:00

I’ve been slowly splitting The Stack up into small internal Swift packages.

Unfortunately, if you use localizable string catalogues in Xcode, there are a few wrinkles.

Lots Of Small Targets

I’ve been splitting Xcode app projects into multiple packages for a few years now.

Mihaela Mihaljevic wrote about something similar a while ago. Some of the details in that post are different, but the basic idea is similar to what I do.

I find it a good way to minimize entangled dependencies, and it has other advantages.

It gives you an obvious place to put a small targeted set of unit tests, it can help with SwiftUI preview performance, and it lets you build and test a lot of code without the need for Xcode.

Using Catalogues In Packages

If you just want localised strings in a package, the basics are straightforward:

set defaultLocalization in Package.swift
put your catalogue in Sources/MyTarget/Resources/Localizable.xcstrings
tell Package.swift to process the resource file

Something like this:

let package = Package(
  name: "StackCore",
  defaultLocalization: "en",
  // ...
  targets: [
    .target(
      name: "StackCommands",
      resources: [.process("Resources")]
    )
  ]
)

If you’re only using Xcode, this mostly works.

Which Catalogue Was It Again?

By default, localization lookup assumes that the strings are in your main bundle.

That is fine for apps, but not so fine for strings that actually live inside a package target.

You can force this to work in a library target by being explicit:

LocalizedStringResource("show.active", bundle: .module)

That works, but it is noisy, and is particularly clunky in SwiftUI views where the visual clutter detracts from otherwise clean code. It is also easy to forget to add the extra bundle parameter.

A nicer option is to get Swift to generate symbols for your localization keys, and to use these directly from code:

public let name = String(localized: .actionNew)
Text(.settingsHotkeyHelp(appName))

One benefit of this is that if the key changes, the generated symbol name will change, and your code won’t build. No more mismatched keys.

However, another benefit specifically useful for libraries is that those generated constants carry the right context. If you use the generated symbol in a place that expects a localizable string, it will automatically pull it from the correct bundle.

This is nice. Less boilerplate, fewer mistakes with keys, and localizations close to the code that uses them.

Building Outside Xcode

There is another wrinkle if you build packages from the command line.

If you build within Xcode, or with the xcodebuild command line tool, symbol generation for .xcstrings happens automatically, even for SwiftPM library targets.

If you build with swift build or swift test however, it doesn’t. This is because Swift Package Manager does not support string catalogs out of the box.

Luckily, there is a command line tool to do the required work:

xcrun xcstringstool generate-symbols ...

It’s easy enough to write a small SwiftPM plugin to call that tool for each of your targets.

That solves command-line builds, but introduces a new problem: when Xcode does its own generation, it will run your plugin too, so you end up with duplicate symbols.

The way around this is to only apply the plugin when the build is not being driven by Xcode.

In your Package.swift you can work around this with a small environment check¹:

#if canImport(Darwin)
import Darwin
let buildingInXcode = Context.environment["__CFBundleIdentifier"]?.contains("Xcode") ?? false
#else
let buildingInXcode = false
#endif

let localizationPlugins: [Target.PluginUsage] = buildingInXcode ? [] : [.plugin(name: "StackStringCatalogSymbols")]

Then targets that own catalogues just use plugins: localizationPlugins.

It’s a little bit of plumbing, but it keeps both worlds happy: Xcode builds and command-line SwiftPM builds.

Other Possible Patterns

I did wonder whether having all the localizations split up like this is actually a good idea in the first place? You do end up with a lot of different catalogues, which could be hard to maintain.

Maybe, instead, localization should be a layer that you only add at the application level? That way they’re all in one place, and different apps could even use the same library but overlay different localizations.

Unfortunately, if you want to use the generated string symbols in your code, your code needs to be able to see them. If the code is in a library, the symbol has to also be generated by that library, or visible to it.

Following this line of reasoning a bit further, could I make a library target with all the localizations in it, and then import that from the libraries that want to use the symbols?

This would still tie a library to a specific set of localizations for its keys, but at least all the strings would be in a single catalogue, which might be easier to manage.

Sadly this doesn’t work either, because the code generator, that creates the symbols, declares them as internal. You don’t get a free, public, cross-package symbol API that every other target can just import and use 😢.

For now, I’ve stuck with per-target catalogues in the Stack, each local to the package that owns the UI or command surface.

This feels consistent with the rest of the architecture. Generally the Stack code is factored into services that are responsible for one aspect of the application. Each service target owns its commands, views, tests, logic and resources - including strings.

Currently, I like this setup. Your Mileage May Vary™.

Thanks to Gwynne for pointing me in the right direction with this check. I originally tried importing Foundation and looking at the environment with ProcessInfo, but I couldn’t find something that was reliably set by Xcode and not by SwiftPM. Using Context works, and also removes the Foundation import, which is apparently best avoided. ↩

SwiftUI Navigation Pain

2025-12-12T00:00:00+00:00

One of the things I’ve wanted to do with The Stack is to handle all of the user’s interactions with some sort of action abstraction.

It turns out that this is harder than it ought to be with the modern SwiftUI navigation mechanisms.

SwiftUI’s modern navigation uses NavigationStack instead of the older NavigationView, and it’s definitely an improvement on what came before.

The basic idea is:

you bind the NavigationStack view to some sort of model representing the route/path that is currently visible
ideally you use the type-erased NavigationPath for this purpose
whenever possible, you let SwiftUI manipulate the route for you using NavigationLink, the default back button, and/or @Environment(\.dismiss)
this pushes items onto the route and pops them off (you can also do this explicitly yourself)
you use .navigationDestination to teach SwiftUI which view to use to back each kind of item

Doing this has some nice benefits.

The view and models stay loosely coupled, and there is no one place in the code where you have to tell the NavigationStack about everything that might get pushed onto it.

The places where you do the pushing/popping can work in terms of the model or some other abstract notion of what is being presented, without needing to know anything about that actual views that will be used.

You can also put your NavigationPath instance into a router class and inject that into the environment. This lets you fish it out from anywhere and manipulate the route explicitly. For example you can implement deep links by splitting them up into items and pushing those items into the route. SwiftUI will see that the route has changed, and sort out the details of animating from where you were to where you’re going.

Actions

For The Stack (and my other apps), I want to define all the user’s interactions as abstract actions. This includes navigation actions.

I want each thing that the user can do to be represented by something that implements a Command protocol. The user will tap or click something, which will cause a Command to be performed, and the perform method of the command will do the actual mutations of the model or view-model. This will probably result in one or more updates to the UI, which the user will then interact with, and the cycle will repeat.

One reason for doing things this way is to allow the same actions to be triggered from multiple places without duplicating code. I can define a single Command for something like making a new note, and have a menu item, a button, a toolbar item, an app intent, or a siri command all invoke it.

This is especially helpful when targetting an application on multiple platforms that support different interaction mechanisms.

Another reason is that I can write common code once to take any Command and have it build the UI for me. If my command abstraction has enough information (for example a label and icon to use to represent the command, maybe an optional shortcut or tooltip), then some generic code can create a button for the command, implement the app intent for the command, insert the menu item for the command, and so on. This reduces boilerplate, and enhances consistency.

A third reason is that I can use this abstraction to support undo/redo at a level that makes sense to me. Foundation’s UndoManager has been around a long time, and is integrated into a lot of systems, including SwiftData, but it works at too granular a level for me. On this topic, I very much agree with what Drew McCormack says in this comment. I would prefer to define undo at a high level.

Finally, funnelling everything that the user does through a single mechanism allows me to support analytics and auditing.

I don’t want to know what any specific user is doing, but I would like some general analytics on usage patterns. What features to people use? What do they miss? After using feature A, do they usually choose B, or C (or rage quit in disgust!)?

Also, there is some need to monetize the applications I make! I have always been a proponent of what I would call “true micropayments”, by which I mean really charging tiny amounts for small actions, such that the user genuinely pays nothing if they don’t launch your app for weeks or months.

I’d like to try to do monetization this way. In order to do this, I need a reliable mechanism to record not what actions my users are taking, but perhaps just how many actions they’ve taken. I can then come up with a formula which ties this to real money in some way.

Unfortunately, using NavigationLink, the default back button, and/or @Environment(\.dismiss) does not lend itself particularly well to the action-based approach I’ve outlined above.

If I use NavigationLink, there is no way to ask it to invoke one of my actions when it is selected. Similarly when the user taps the default back button, or a button that calls the dismiss() method supplied by the environment.

The view-model manipulation of the route is just done for you in these cases. That means that if you want all of your navigation to be done through actions, you’re out of luck unless you make your own UI elements. This isn’t hard to do, but it is a bit annoying, and you can lose behaviour that you’d otherwise get for free. Given the fragility of SwiftUI, I prefer to give it as much semantic information as I can. Using NavigationLink tells SwiftUI more about what’s really going on, compared to just using Button. I don’t like the fact that I might be losing some clever behaviour (possibly future behaviour) if I don’t do that.

Watching The Path

A possibly solution to this problem is to allow SwiftUI to manipulate the route, but bind to the path variable and watch it for changes - for example using .onChange in my root content view.

When a change to the path happens, we can work out what it is and issue a ‘fake’ command to our action system, so that the interaction is recorded and can be audited, undone, etc. This isn’t great, but it might be better than the alternative of abandoning the recommended UI mechanisms.

Unfortunately this is pretty much impossible to achieve if you use SwiftUI’s NavigationPath to represent your route!

NavigationPath is a type-erased list of the items in your stack. The type erasure is good in that it works regardless of what you throw into it, which in turn means that you can use .navigationDestination the way that it is intended to be used, which is to declare an individual destination for each type that you place on the stack. This is important for keeping things loosely coupled - and so using NavigationPath is recommended.

Unfortunately, NavigationPath only lets you manipulate the path by adding or removing items from it, and the only information you can obtain about the current state is how many items are on it. At the point that a user taps on a NavigationLink, SwiftUI still has enough information to supply the linked value to the closure that you gave to .navigationDestination as an actual typed instance.

Sadly, it doesn’t seem to allow you retrieve a value from the path later. You can’t say, for example, “what’s the value on the top of the stack?”.

The upshot of this is that if you watch for changes to NavigationPath instance, all you can tell is that its length has changed!

If the length reduced, you know that the user popped, but if the length increased, you know that the user pushed something, but not what it was. This is not much use for generating fake navigation commands!

Not Using `NavigationPath`

There is an alternative approach.

NavigationStack allows you to supply a binding to a collection of anything hashable, and so you don’t strictly have to use NavigationPath.

You could, instead, define an enum with all the various kinds of things you want to push, and bind to an array of your enum type.

This works, and when you watch your array, you know exactly what kinds of things it contains, and so you can issue the relevant fake commands.

The downside of this approach is that there’s more coupling. You have a single enum that has to know about all of the things that you are going to push, and you will end up with a single .navigationDestination call for the enum type.

Imperfect Solutions

Binding to a path which is an array of enum (or some other custom type) values isn’t wrong, per-se. It is probably what I’ll need to do if I want to fully support recording commands for all of the user’s navigations.

It seems to me that it defeats much of the point of the way NavigationStack has been designed though.

In any case, personally I really wish I didn’t have to take the approach of watching mutations to the path in the first place 😢.

It would be far better if I could register a single hook with the NavigationStack. This would be called with a path manipulation to perform (push/pop/reset), and allowed to do it however it wanted to. I could use this hook to dispatch commands through my action system, and they would change the path.

This would allow me use all of the other navigation mechanisms that SwiftUI has supplied me.

Currently I’ve adopted a hybrid approach:

I don’t watch the path.
I use custom buttons instead of NavigationLink, and they issue navigation commands to change the route
I use the default SwiftUI back button, and accept that I’m not currently capturing the user popping views off the stack

Tune In Next Time…

Another possible approach to this problem might be to use .onAppear/.onDisappear on the views themselves, and somehow have them generate the navigation events that way.

This would also solve another problem that I have, which is that I’ve some global UI components that I would like to always be present, but to react differently depending on what’s currently at the top of the navigation stack.

Unfortunately, relying on .onAppear/.onDisappear also has a number of problems!

Originally I’d intended to cover those in this post too, but it’s getting a bit wordy, so I think I will leave that one for another day…

Am I Holding It Wrong?

As always, this post reflects my imperfect understanding of SwiftUI. Have I missed something? If so, let me know. I’d be delighted!

All Quiet In The Western Isles

2025-12-05T00:00:00+00:00

I’ve been a bit quiet and not posted for the last few weeks.

This is down to a combination of holidays, minor sickness, distractions, and beavering away preparing an MVP of The Stack, which I’ve now released.

I’ve also got a bit of a backlog of other things I mean to post about…

… but they will have to wait for another day.

Potential topics include:

the solution I settled on for abstracting menus, buttons and toolbar items in SwiftUI
extending my action abstraction to support siri, shortcuts and intents
more refinements to my Release Tools
localized strings in packages
plenty more updates to The Stack
potential next projects

The Stack

2025-12-04T00:00:00+00:00

I’ve just quietly released the first version of The Stack.

It represents a slightly different and fairly opinionated take on solving the To Do list / note taking problem, and I made it mostly to scratch my own itches¹. I’m hoping that some other people think the way I do and will also find it useful.

You can download it here.

For some more detailed answers as to why I made it, read on…

Isn’t This A Solved Problem?

Well, yes, sort of.

To Do lists are paradoxical though. Much like Rimmer’s revision timetable in Red Dwarf, they can become a distraction and a source of anxiety.

The Stack Isn’t Reminders

There are some categories of thing-to-remember where an app like Reminders is perfect. It’s helpful to be able to set alerts, especially . It’s helpful to be able to have multiple lists, and to share them with other people.

I use Reminders to alert me to things that repeat periodically (“file your taxes”, “put the bins out”, etc). I also use it to share a shopping list with my partner.

The Stack definitely isn’t trying to be that kind of app.

The Stack Isn’t A Methodology

There are some mature methodologies in this problem space - one of the best known being Get Things Done.

There are some great apps out there to support these methodologies, and if you swear by a methodology then you probably want an app that supports you - lets you put tasks into buckets, or give them colors, or group them in a way that makes sense for the methodology.

The Stack isn’t trying to be that kind of app either. Or, at least, it’s not trying to support another methodology. Arguably, it’s trying to introduce a new one, albeit a very lightweight one.

The Stack Isn’t An Issue Tracker

When trying to keep track of tasks, at the heavier end of the spectrum are fully-fledged issue tracking system.

If you’re in a team, the ability to share tasks with everyone, prioritize them and plan them, set deadlines, add keywords, etc, etc, is essential.

Even if you’re working on your own, the features of issue trackers may be helpful for you.

There are plenty of issue tracker apps out there for you. The Stack isn’t one of them!

So What’s Left?

The full name of my app is actually “The Stack - Notes To Self”, and that should give you a clue as to where I think it fits.

When I’m in the middle of something, and an idea pops into my head, I just want to record it and move on. This is what I call a “note to self”.

What matters most to me in this situation is just to capture the thought.

I don’t want to have to think about which list it goes in. I don’t really want to have to think about dates or priorities or any of that stuff.

I find having to make those decisions is off-putting and can slow me down or even paralyze me into indecision.

Even as a solo software developer, I find that I don’t really need a proper issue tracker a lot of the time. I’ve used many different trackers, and have come to realize that the simpler they were, the fewer options and fields they had, the more I liked them.

The Stack is built for these situations.

It isn’t intended to replace your shopping list app, or your issue tracking database, although it can, if you want it to.

It can complement a methodology-based app (or perhaps, it can replace the methodology with a simpler one?).

Triage?

The Stack is essentially just a sequential list of notes, with the latest at the top.

You may be worried that this will rapidly get too big and unruly, and you’ll not be able to find anything.

I can understand this concern, and if you want, you can think of it as just being for initial capture of thoughts.

You can then triage them periodically and move into other more permanent places, that the the stack itsef stays quite light and empty.

I don’t use it that way, but you can if you want.

A Big Heap ‘O Stuff

My idea is that it will work even when there are lots of notes in the stack, and it feels a bit like a big heap of unsorted stuff.

Embrace the Chaos, I say!

It is perhaps counter-intuitive, but I actually suspect that many notes get added to many to-do lists in many apps, and then never actually actioned.

My thesis is that it’s more important to just capture quickly and move, on. Later, when you’re looking for something, The Stack can help you find it.

There are three main mechanisms for this:

Ordering is one. Another of my theories is that a kind of locality of reference applies here. The things near the top of the stack, generally the things you added most recently, are often the things care about most.

For the few cases where this isn’t the case, I’ve added a mechanism for pulling older items back to the top. Note that this isn’t full re-ordering. You can’t move notes around in the list, you can just pull them to the top. This is a deliberate choice!

Tagging is another way to organise things. You can add #hashtags into notes, just like you do on your favourite social media site. You can then filter the stack by a tag. I find that this is all I need most of the time.

When I’m focussing on a task, I filter by the tag for that task, and I can see what I need to do next, in a list that isn’t too large. Periodically I will scan this list, and move items to the top if I think they are a priority.

For everything else, free text search lets you find notes that have fallen through the cracks.

Give It A Go

Anyway… that’s the idea behind the stack. I’m finding it useful, and will keep maintaining it for as long as I actually do. Your mileage may vary.

For now it’s free. I am hoping to use it as a vehicle to test some monetisation ideas based on true micropayments, in which case you might one day have to pay something for it, but if that ever happens, it will be a small amount derived from the real usage you make. Even then, the changes are that there will be an honour-based opt out.

If you’re interested in trying The Stack, download it from the app store on your phone or mac.

If you like it and would be interested in helping me test it, there’s also a public Testflight link.

Also, of course, The Stack exists as a place for me to experiment with new Apple technologies, and generally keep my skills up to date. One day it might also be a source of income, but for now, it’s free. ↩

App Storage

2025-10-30T00:00:00+00:00

Whilst adding some settings to The Stack¹, I was reminded of something that has bugged me for a while about SwiftUI’s @AppStorage.

In a classic act of yak shaving, I decided to fix the annoyance instead of doing the task I actually set out to do…

Is That The DRY-Violation Klaxon I Hear?

SwiftUI’s @AppStorage macro is a neat way to access a UserDefaults value from within a SwiftUI view²

You use it like this:

struct ContentView: View {
    @AppStorage("username") var username: String = "Anonymous"

    var body: some View {
        VStack {
            Text("Welcome, \(username)!")

            Button("Log in") {
                username = "@samdeane"
            }
        }
    }
}

The ability to declare a default value is useful for the first time the user runs the app.

Unfortunately though, for most apps of a non-trivial size, you are likely to have some user-interface code in one place to allow the user to change the setting, and some other UI or application logic elsewhere that reads the value.

Which leaves you with something more like this:

struct SettingsView: View {
    @AppStorage("doTheThing") var doTheThing: String = true

    var body: some View {
        Toggle(isOn: $doTheThing) {
          Text("Show the button to do the thing?")
        }
    }
}

struct AppLogicView: View {
    @AppStorage("doTheThing") var doTheThing: String = false

    var body: some View {
      if doTheThing {
        Button("Do The Thing!) {
          performTheThing()
        }
      }
    }
}

Shome Mishtake Shurely?

Did you spot my deliberate mistake?

I set the default to true in one view, and false in the other.

My assumption is that what will actually happen here will depend on which view the user visits first, in the situation where they don’t already have a value set for the setting.

Which is… non-optimal.

It also bothers me that I have to type the "doTheThing" key in both places.

I might change it or get it wrong in one place and not the other.

I can even potentially use a different type for the same setting in different places.

A Better Way™

After thinking about this for a bit, I added the following extension³:

import SwiftUI

public struct AppStorageKey<Value> {
  let key: String
  let defaultValue: Value
  
  public init(_ key: StringLiteralType, defaultValue: Value) {
    self.key = key
    self.defaultValue = defaultValue
  }
  
}

public extension AppStorage {
  init(_ key: AppStorageKey<Value>, store: UserDefaults? = nil) where Value == Bool {
    self.init(wrappedValue: key.defaultValue, key.key, store: store)
  }
  
  init(_ key: AppStorageKey<Value>, store: UserDefaults? = nil) where Value == Int {
    self.init(wrappedValue: key.defaultValue, key.key, store: store)
  }
  
  init(_ key: AppStorageKey<Value>, store: UserDefaults? = nil) where Value == Double {
    self.init(wrappedValue: key.defaultValue, key.key, store: store)
  }
  
  init(_ key: AppStorageKey<Value>, store: UserDefaults? = nil) where Value == String {
    self.init(wrappedValue: key.defaultValue, key.key, store: store)
  }
  
  init(_ key: AppStorageKey<Value>, store: UserDefaults? = nil) where Value == URL {
    self.init(wrappedValue: key.defaultValue, key.key, store: store)
  }
  
  init(_ key: AppStorageKey<Value>, store: UserDefaults? = nil) where Value == Date {
    self.init(wrappedValue: key.defaultValue, key.key, store: store)
  }
  
  init(_ key: AppStorageKey<Value>, store: UserDefaults? = nil) where Value == Data {
    self.init(wrappedValue: key.defaultValue, key.key, store: store)
  }
}

I can use this in my application:

public extension AppStorageKey where Value == Bool {
  static let doTheThing = AppStorageKey("doTheThing", defaultValue: false)
}

struct SettingsView: View {
    @AppStorage(.doTheThing) var doTheThing

    var body: some View {
        Toggle(isOn: $doTheThing) {
          Text("Show the button to do the thing?")
        }
    }
}

struct AppLogicView: View {
    @AppStorage(.doTheThing) var doTheThing

    var body: some View {
      if doTheThing {
        Button("Do The Thing!) {
          performTheThing()
        }
      }
    }
}

These declarations can all live in different swift files (or even different packages).

I can define the settings key, type and default value once.

I get type inference everywhere that I use @AppStorage in this way, so I don’t have to declare a type.

As a bonus, when I type @AppStorage(. Xcode will auto-suggest any static AppStorageKey values that it knows about.

Which is nice…

I’m still looking for Test Flight testers for this. If you’d be up for giving it a go - even just to help me out by confirming that it doesn’t immediately explode, then please sign up. ↩
Or modifier, command, observable object, or various other SwiftUI structures. ↩
The repetition here with different types is ugly, but it mirrors the way that AppStorage defines its own API. It would be nice if there was a better way, but I’ve not found it yet. ↩

Caveat Emptor

2025-10-17T00:00:00+00:00

I was looking back on some of my old blog posts from 2021 earlier¹, and I came across something that I thought bears repeating.

It was from this post, and the section I though was worth re-visiting was called “Caveat Emptor”.

It was about how my open source projects are often unfinished, and unpolished, and very much work in progress.

What I said then still applies today, and I’ll repeat it below.

As mentioned above, the context for the following is an old blog post, and so when it says “this post”, or “this project”, it’s talking about the original post.

Caveat Emptor

Part of the barrier to telling people about things I’ve done is sheer time it takes to write even quite a simple post like this one.

So my first disclaimer is just to say that this post is mostly a re-hash of the README file from the Github Repository. Nothing wrong with that I think, but just to be clear…

My second disclaimer is that this is work-in-progress code from the real world.

I’ve encountered a few people who subscribe to a fundamentalist view of open source code: that it’s useless unless it is fully polished, fully tested, 100% supported and actively maintained.

I understand this point of view; we’ve all encountered code that makes great claims and turns out to be broken or mostly unfinished.

Respectfully though, those people are wrong.

Imperfect open-source code can be frustrating. However, it can also be a helpful foundation for someone else to build on, a good example of the pros and cons of particular technique, or a useful supplier of that one crucial line you have been searching the internet for.

Aiming for perfection is setting the barrier way too high. I am as insecure as the next person when it comes to showing my workings in public. I’ve been a professional programmer for more than three decades, but I still suffer from impostor syndrome.

It’s tempting to hide away, but I’m trying to fight the urge, and I’d like to contribute in some small way to an environment where we aren’t scared to risk being wrong.

I offer up all of my open-source code in this spirit. It’s not perfect, because I am busy, and because I am still writing it. I find this code useful, and I hope someone else might. If you do find that it is fundamentally broken, please tell me why. That way I learn something.

Which sounds a little bit narcissistic now that I say it out loud. Ah well - bite me. I fell into a rabbit hole ok, and it just so happens to be one of my own making. ↩

Release Tools Tagging

2025-10-14T00:00:00+00:00

After my last post, I thought a bit more about Release Tools, and decided that requiring a tag was definitely the right way to go, and over a couple of days last week I implemented it.

I also decided that it was a big enough change that I may as well call this Release Tools 4.0, and take the opportunity to clean up and remove some legacy code.

Tagging

Managing release tags in rt is now a separate step.

Running rt tag will examine the existing tags, and the HEAD commit.

If there’s a release tag already at HEAD, it will exit with an error.

If not, it will figure out the latest version number build number in use, by examining the existing tags.

It will then make a new tag using the same version, and with an incremented build.

This tag will be in the form: vX.Y.Z-NNNN where X.Y.Z is the semantic version, and NNNN is the build number.

Note that this does not include the -platform component that rt previously used.

From 4.0 forwards, we assume that there will be just one version tag for any given commit, and we will use that tag for all platforms.

We do however still support the old format when scanning previous tags – so rt will pick up any legacy tags and correctly calculate the new build number.

Archiving And Submitting

With 4.0, if you run rt archive or rt submit, the first thing that rt does is examine the HEAD commit, looking for a version tag in the format that rt tag creates.

If no tag is found, it will exit with an error. This ensures that any build you submit from a given commit will be tagged correctly, and that if you submit multiple platforms from the same commit, they are guaranteed to have the same build number.

Continuous Build Injection

If your project is set up to run the rt update-build every time you build, it would clearly be unworkable for it to require a version tag to exist at the HEAD commit.

If there is a tag at HEAD, rt update-build will use it.

Otherwise, it scans backwards to find the highest previous build tag, and then adds one to the build number.

In general I would discourage using rt for debug builds, since running a script for every build will slow Xcode down a bit, and having the build number available is of limited value.

However, the rt update-build command may still be useful if you have a custom build pipeline but want to use rt to calculate and/or inject build information.

Build Variables

Version 4.0 of rt also changes the names of the variables that are injected, and adds a new variable.

We now set the following variables in the .h, .xcconfig and .plist files:

RT_BUILD
RT_COMMIT
RT_VERSION

These three variables are also passed to xcodebuild on the command line.

If you were using rt already, you will need to adjust your project accordingly.

The rationale for the change was that it was probably better to use our own variables, rather than using CURRENT_PROJECT_VERSION which already has a meaning within Xcode.

That said, you probably will want to set CURRENT_PROJECT_VERSION to the value of RT_BUILD. You can do this in the build settings for the project, or each target, or by setting up an .xcconfig file which includes the line CURRENT_PROJECT_VERSION = $(RT_BUILD).

The semantic version that is injected is taken from the build tag that rt tag creates. By default it’s the same version that the previous tag had, but you can change it explicitly by doing rt tag --explicit-version <x.y.z>.

This allows you to make the git version tag the single-source-of-truth for all version information, if you so desire, and to inject the semantic version into all other settings and plists.

Legacy Cleanup

In rt 4.0, two commands have been removed.

The rt install command used to exist as a quick way of linking rt into your path. In general I think it’s better that you install rt with a tool such as Mint. However, some people will want to build from source, or do something else, and each installation method is likely to involve a different way to update $PATH. On the whole, this feels like it’s something that it out-of-scope for rt itself.

In the early days of rt, the design encouraged you to run it via some standard shell scripts, and also to include some standard .xcconfig files in every target. The rt bootstrap command existed to help you copy (or update) these scripts and config files into your project. This way of working with rt is obsolete, and so the command was a bit of an unnecessary legacy.

For the sake of simplicity, these two commands are no longer supported.

The fallback mechanism for calculating a build number by counting git commits has also been removed.

Development Notes

Doing these updates was another opportunity to experiment with AI code generation, and that’s what I did.

Rather than making the code changes myself, I largely tried to instruct Copilot to do them for me.

This was an interesting process.

The codebase is fairly mature, and a lot of what I was asking for involved refactoring existing code to add a little bit of new functionality. The results were mixed, and once again, it felt quite akin to working with an enthusiastic and willing colleague who had a bit less experience.

The first challenge was to express clearly what exactly it was I wanted. Much like test-driven development, I quite enjoy the fact that this forces you, early on, to work out what you actually do want. Also much like test-driven development, I didn’t always do a good job of working it out to begin with. Luckily, AI coding agents have infinite patience, and probably don’t bitch about you behind your back whilst having lunch with the other juniors¹.

In general the code that came back did work, and performed as intended. It was however often repetitious. The AI rarely showed insight about the codebase as a whole, and it had to be explicit prompted to clean up duplication and to create appropriate abstractions, methods, or constants in order to generalize. Again, this is quite like working with someone less experienced. They focus on the problem you’ve set them, and they stop when they think they’ve got it working. They don’t see the larger picture, and don’t consider consistency or maintainability of the codebase.

Part of this I think may be down to the fact that I had not set any general instructions in the project. I have now added a copilot-instructions.md file, and am experimenting with trying to set some standards in it.

One thing that I did like, and do think is useful, is that the AI generally created accompanying tests as a way of verifying each change I’d asked it to make. This is something that I aspire to, but don’t always manage, and having another “person” encourage me to work that way is excellent.

It also proved to be the right approach, in that it unearthed some problems; not just in the new code, but in some of the dependencies that I was using. These were subtle concurrency issues which mostly manifested when performing tests in parallel, and they may not have impacted actual normal usage of the tool², but that makes it all the more valuable that I was forced to confront them.

At the end of this process I did find myself wondering whether I could have achieved what I wanted faster if I’d just done it myself. I think that perhaps I could, and the code would have been tighter, for a narrow definition of “what I wanted”.

However, having an essentially tireless worker, who pro-actively volunteered to make tests, encouraged me overall to do a better job. Whilst getting Copilot to fix the duplication that it had created, I was also motivated to ask it to take a number of other refactoring steps aimed at improving consistency and making the code that I had previously written more idiomatic. Whilst tracking down the concurrency issues in the dependency I was using to run subprocesses (another of my projects, called Runner), I think I improved my general understanding of the surrounding issues, whilst also fixing some bugs I probably didn’t know I had in other projects using the same package.

Overall, I continue to be cautiously positive about using Copilot. I think that the problem domain of programming is a good one, especially when the output from the AI tool is small enough that it can be reviewed by a human expert who can provide feedback.

Asymmetry

I was talking to friend over the weekend about a situation where they were using AI to review large documents. This is a very different scenario, and the balance is completely reversed.

In my case, the AI tool is taking a vast amount of external knowledge and applying it to produce a small amount of new content, which is easily reviewed.

I don’t need to know where the ideas behind the solution came from, I just need to be able to read the new code and evaluate whether it works, and whether it fits in with the existing codebase.

This is quite asymmetrical, but in a direction that is positive.

In my friend’s case, the output from the AI was going to be a report, but the accuracy of that report could only be evaluated by essentially doing the same job that the AI had been asked to do.

From my own experience I’ve seen the AI make more than enough mistakes, and disappear down enough rabbit holes, that I would never trust its answers without being able to at least scan the output to verify it.

I wouldn’t want to use it to analyze or summarize a large body of information unless I had a pretty reliable way to validate the results.

Future

I continue to use rt for my own projects, and so I expect that it will evolve further over time.

One item on my to-do list is to try to add support for installing it via homebrew.

That’s not something I really need myself right now though, and so it may take me a while. Pull requests gratefully received…

This may be a wildly optimistic assumption. ↩
Most of the time. Maybe… ↩

Release Tools (Fastlane For Idiots)

2025-09-26T00:00:00+00:00

My latest detour was to add an extra option to ReleaseTools, a command-line tool, written in Swift, that I made a few years ago.

Actually I’ve just checked, and I started it in 2019 🤯.

ReleaseTools (or rt for short) basically exists to automate the process of uploading an app to the app store, or packaging it up for external distribution via Sparkle.

I guess you could think of it as a bit like Fastlane, but without any of the bells and whistles¹.

It is yet another thing that I’ve never really got around to talking about! Largely because I made it for myself, and didn’t really want to end up supporting it for other people.

That said, it’s fairly mature now and I’ve used it in at least one client project as well as for a number of my own. Whilst I can’t guarantee that it will work for you, and I’m sure the documentation needs some improvement, I would certainly be happy to try to help someone else give it a go.

Scripting Like An Animal

Under the hood, rt uses xcodebuild, altool and git.

Prior to it existing, I was doing most of the same things that it does, using hand-rolled shell scripts for each project.

Or, in the case of my time working on Sketch, a big hairy load of Python I wrote²…

At some point I got annoyed with the fact that I was writing shell script, came to my senses, and decided that using Swift would be much more ~~efficient~~ fun.

Basic Design

If there was one thing I learned whilst hand-rolling all the previous bespoke solutions, it was that debugging build scripts is a massive pain in the arse.

Partly because debugging scripts is just generally painful, but also specifically because the build process tends to be slow, and it’s really annoying to have to keep waiting for a long task like archiving to run, only to have the subsequent exporting fail because you messed something up.

So when I made rt, I decided to try to factor it out into a series of small commands that were intended to be run in sequence.

Thus the basic design is that you can individually run commands to do things like:

archive
export from the archive
upload the export for distribution
upload the export for notarisation instead
wait for notarisation to finish
compress the notarised app
rebuild a sparkle feed to include the compressed app
publish the sparkle feed to your website

This lets you write a simple shell script³ to run the commands in sequence during normal operation, but when something goes wrong you can just re-run that command, and/or the subsequent ones.

Anyway, I won’t go into a lot more detail about the specifics - if you want those you can check out the README.

Suffice to say that it basically works these days, and I use it daily.

In fact I’ve added a fast path where you just run rt submit and it performs the archive, export and upload commands in sequence.

Build Numbers

One of the nice things rt does is calculate and inject a build number.

I prefer to do this, rather than relying on the app store portal to automatically assign one, as it lets me tag the exact commit that I built from in git. Once it knows that the upload has succeeded, rt makes a tag in git including the exact build number, version, and platform.

Originally, I calculated the number just using the count-the-git-commits trick.

This works surprisingly well most of the time, but it is a little fragile. In particular, if you switch branches, the count can change, and consequently the build number can go down. This is Not A Good Thing™.

In practice, I rarely encountered this problem. I commit little-and-often, and for projects I’m working on myself, I would probably only submit from a single branch.

However, when I working on Club, we did have this problem. That project didn’t use rt at all - it was cross-platform, and written in Dart/Flutter, with an entirely different toolchain. It started out doing the commit-counting, but we ended up switching to a solution that explicitly read the existing git tags and used them to work out the next build number.

When I subsequently came back to my own projects, I was inspired by the tag-based approach and decided that rt should use it too.

Version Tags

So I added an option to rt which calculated the new build number by:

scanning for tags in the form vX.Y.Z-platform-NNN, eg v1.0.2-iOS-234.
treating the highest existing NNN for our platform as the current number
adding 1 to it for the new number

That works really well, and because rt also creates and pushes a tag in the same format, it’s pretty reliable.

The only wrinkle with it is that when you are supporting two or more platforms, you have to submit each one with rt separately, e.g:

rt submit --platform=iOS
rt submit --platform=macOS

Sometimes a problem will occur, or something will get broken, such that one upload succeeds and the other fails.

This becomes a problem subsequently because you’ve now got one platform with a tag that worked, and which the app store knows to have that build number, but another platform that only has an older tag.

So when you subsequently upload the next build, you end up with different build numbers for each platform, even though they are built from the same commit. Which is confusing and a bit annoying.

Why Are You Telling Me All Of This Again?

Good question.

Come to think of it, the main point of this whole post - other than just answering, in a rambling way, the “how was your day dear?” that you didn’t ask - was to mention the existence of rt.

Which I’ve done.

You can stop reading now!

What follows is more detail than you probably want to know about why I started actually working on rt recently, what I did to try to fix the tagging problem, and why I’ve realised that I need to go back to my fix!

Inconsistent Tags

To fix the inconsistent tags manually, the easiest thing to do has been to use git on the command line to make a tag with the right build number, for the platform that had fallen behind, on the commit that it should have been uploaded from.

That way, rt will find the tag next time, add 1 to it, and come up with the right value.

This is a bit clunky. Also, occasionally, I only want to release one platform, as I may have changed code that just affects it. Next time I submit though, I want to release both platforms using the highest build number for any platform.

So I decided to try to add some new capabilities to support this objective.

First, I’ve added an --explicit-build option to allow you to specify the build number to use. That is a cleaner way to reset everything when doing a new upload, as you can just force the number for both platforms to get them back in sync.

I’ve also added an --existing-tag option. This looks for the highest tag for any platform, as well as for the platform you’re uploading, and compares the two. If there’s a higher number for another platform, it will use that. If not, it will do what it used to do and increment the previous build number by one.

This works nicely for the situation where a build or upload failed for one platform in a transient way, you’ve fixed it with some minor commit, and you just want to upload something with the same number as the other platform(s).

I actually thought that I was done, but whilst writing this blog post, I’ve realised that I’m not.

If the commit has changed, you really should use a higher build number - but it needs to be higher than any platform has used, and not just higher than the platform you’re uploading.

I think posibly what rt needs to do is to figure out whether the tag from the other platform is on the HEAD commit. If it is, then what you’re doing is “catching up” and it should just use the same build number - it’s the same commit, so using the same number makes sense.

If there’s no version tag on HEAD though, then we probably want to increment the highest number by one more, and use that. This will create a new baseline with the first platform that you upload, and subsequent platforms will then catch up to it.

So I guess that’s the next job then…

Or…

… although, maybe the problem here is that the design is wrong.

The basic problem is that when making the tag, rt doesn’t know which platforms you’re uploading, as it does them one at a time.

I can see a couple of ways to fix this.

One would be a way to pass a list of platforms in, and have them all upload at once.

Another… which I am coming round to instead… is making the tag creation an explicit and separate step.

This feels more in keeping with the small-individual-commands design.

Maybe version tags should not be platform-specific, and just in the form vX.Y.Z-NNN.

If you run rt submit or rt archive and there isn’t a version tag on HEAD, it should just refuse to do anything.

Then we can have a separate rt tag command which makes a new version tag. So you run that first, then run the archive or submit commands for each platform in turn.

We can still have a small shell script that you can run for the normal situation where you want to upload everything at once:

rt tag
rt submit --platform=iOS
rt submit --platform=macOS

You can still run the individual bits though, and they will do the right thing. If you’re on a tagged commit they will work. If not, they won’t. If you run rt tag on a commit that’s already tagged, it can spot that and complain.

This feels a lot cleaner actually.

I’m glad we had this talk.

Thanks for listening!

No offence to Fastlane, but I’ve always found it to be complicated and confusing, and the dependency on Ruby annoys me. ↩
To be fair, these scripts did a whole load of other stuff too. They were a bit hairy though… ↩
Yes we’re back to shell scripts. Teeny tiny ones though. Honest. ↩

Dipping A Toe Into The AI Waters

2025-09-23T00:00:00+00:00

Ok, I admit it, I’ve been tinkering with AI again.

Not using it, this time, but trying to add it to my app The Stack.

Yet another person shamelessly trying to shoe-horn some AI into their app, so that they can jump on the bandwagon?

Well yeah, guilty as charged, I guess. Anyhoo…

Summarising Notes

Individual notes in the stack are generally quite short. Things like “add a blog post talking about adding AI to The Stack”. There’s probably not a lot of mileage in summarising them.

Given a long list of them though, it’s easy to get swamped.

Could we use some sort of AI to generate a summary of all items, or of all items with a given tag? Maybe it could pull out one or two emerging themes? Perhaps also suggest the item to tackle next?

On-Device

For iOS and macOS 26.0, Apple offer a new on-device Foundation Model.

This seems like it’s tailor made for my use case.

The query doesn’t leave your device, so there are no privacy issues to contend with.

So I decided to give it a go.

Adding The Code

On the plus side, adding the code for this was ridiculously easy. About three lines of actual Swift:

  model = LanguageModelSession(instructions: summarizerInstructions)
  let response = try await model.respond(to: prompt)
  return response.content.trimmingCharacters(in: .whitespacesAndNewlines)

It’s no exageration to say that the instructions and prompt that I feed to the AI is a lot longer than the code I use to invoke it.

Therein lies one problem, however. The prompt. Getting a prompt that actually produces useful output is difficult.

I’m initialising the model with these instructions:

    let summarizerInstructions = """
      You summarize a list of to‑do notes for planning.

      Rules:
      - Output: one concise paragraph, no more than 50 words
      - Use plain text, no headings or lists.
      - Weigh recently updated items slightly more, but consider all items.
      - Use tags only as background context; never mention tag names in the output.
      - Prefer synthesis over enumeration; use actionable, neutral language.
      - Summarise for the current user only. Do not refer to teams or other users.
      - Write in the imperative mood where possible. Do not address the user directly.
      - Do not invent notes or details.
      - Write in the same language as the notes.
      """

Then giving it the following prompt:

  var prompt = """
      Summarize the following to‑do items into a single paragraph that captures the main themes and priorities.

      Each item is in the form <date>: <body>, where: 
      - <date> is the date that the note was last modified (more recent = higher weight)
      - <body> is the note body, including any hashtags

      Items:
      \(summarisedItems.joined(separator: "\n\n"))
      """

This took quite a lot of tweaking. Inevitably, I asked Copilot to help 😆.

I am sure it could be greatly improved.

So Far, So Meh…

The current prototype sometimes does a decent job, but the output has a tendency to end up a bit wordy.

I haven’t had it completely hallucinate, but it does sometimes completely ignores parts of its instructions!

I understand enough about the process to not be surprised by this, but the level of unpredicability makes it quite difficult to rely on.

Worse, I’m not entirely sure that the output is particularly helpful!

More testing required I think, particular with large lists of items.

There’s a bigger problem with that, however, when using the on-device model.

Unfortunately the context window is small. It only supports 4096 tokens, where each token is a few characters of text.

Even with a modest collection of a couple of hundred notes, it’s easy to exceed that window, resulting in an error.

I suspect that this means that the on-device approach is a non starter, and it will be necessary to send the entire item list to an online model. Which opens up a whole other can of worms…

Oh well.

Conclusion

Interesting experiment.

I’m not yet convinced, but will keep playing…

Dual Universe

2025-09-16T00:00:00+00:00

Meanwhile, back at Chaos Towers: work continues on The Stack, but I’ve also got a bit distracted by returning to Dual Universe (sort of)…

Dual Universe was an interesting game.

I say was, because the original MMO shut down recently.

I signed up to it as an alpha backer a few years ago, just before they went into beta, and I stuck with it until well past the point where they officially released and were charging a monthly fee for it.

Calling it a game might be a bit of a stretch actually - it is really just a big sandbox, and almost all of the content is user generated, including ships, buildings, and most of what you might call narrative.

That’s ok though.

You can create functional spaceships, buildings and factories on land, and spacestations, using a range of elements that you can trade or create with an industry system. You can mine for resources, or trade for them. You can model skins for these constructions using some voxel-based tools which can produce amazing results (if you have the talent to use them properly, which I manifestly don’t). Crucially for me, you can also script the dynamic aspects of your constructions, using Lua.

I liked the setting (space / procedural planets), and I like the building and scripting, which were right up my strasse. There was a bit of a grind occassionally, but it’s an MMO so that comes with the territory.

All in all it was a fun way to “unwind” - although a bit of a busmans holiday for me as did a lot of building and tinkering with scripts.

I ended up putting in a lot of effort on the Lua, and even started writing a blog about the scripting process, in the guise of my in-game alter ego.

Sadly, over time I found that the game’s scripting limitations frustrated me. The development pace was glacial. The designers were concerned about game balance, and imposed all sorts of annoying limitations on the scripts, which crippled them. Their concerns were understandable, up to a point, but they didn’t have much of a “game” in the first place, and I felt that they had taken absolutely the wrong path. I think that the amazing things we could have created would have attracted more players and created new reasons for playing. The game designers didn’t agree.

I’ll never know whether I was right, but ultimately the game designers were definitely wrong; the game didn’t attract enough users and was not enough of a commercial success to sustain the cost of the servers; hence the recent shutdown.

So why am I talking about all of this now?

Well, about a year before they shut down the MMO, Novaquark, the company behind it, released a standalone version of the server code, and a version of the client that could connect to third party servers. Which was… unexpectedly sensible.

As a consequence, some community servers sprung up, and a couple are still going with sustainable populations. When I saw that the MMO was shutting down, I became aware that there was now a way to set up your own server, or to play on someone else’s, without having to pay a monthly subscription. I tried to resist, but ultimately dived in again.

The server I picked is The Third Verse. I doesn’t appear have quite as many users as the other main one, but so far I find it to be reliable and a nice crowd.

I don’t know how long I’ll stick with it, but for now I’m having fun again with the slow bootstrap. You start with very little, and if like me you enjoy trying to get to the stage where you have mega factories, a fleet of ships, and a space station or two - well, it’s a long old haul.

I’m also back to developing my scripts and toolkits, which are all open-source.

If any of this sounds like fun, why don’t you pick up a copy of the client (which requires a one off payment, but not a ruinous one), sign up to The Third Verse, and come and say hello.

You’ll find me in a lovely little bay, just northwest of District 3 on Alioth!

SwiftUI Action Abstraction

2025-09-09T00:00:00+00:00

Since my last post I’ve mostly gone back to working on The Stack.

It’s reached that fiddly phase where the basic MVP functionality is probably in, but there are lots of little features and tweaks that are vying for inclusion.

Meanwhile the UI is a bit clunky and definitely needs some work, but how much is enough for now? Perfect is the enemy of good and all that…

Luckily I’ve been getting some great feedback from some of my testers, and so that is generating a nice series of small tweaks to aim at.

I spent quite a bit of time towards the end of last week cleaning up labels. There’s more to do, but they’re a lot cleaner than they were.

Toolbars

I’ve also been doing a lot of wrestling with toolbars and toolbar buttons in SwiftUI.

I guess perhaps I just haven’t internalised the knowledge yet, but I find it rather difficult to predict exacly which sequence of ToolbarItem, Menu, Button, Picker and so on I require in order to achieve my desired effect. I’m trying to follow the guidelines, and the common practice (especially that set by Apple’s own apps, which don’t always follow the guidelines 🤔), but it’s not always obvious what to do.

This is complicated by the fact that I’m supported macOS and iOS. I probably do want a different layout and slightly different behaviour for iOS and macOS, but I haven’t entirely worked out where the differences lie. I may build a standalone macOS target eventually, but for now I am using Catalyst. In some ways I think this is a further complication, because there are things that you can do in a macOS target in SwiftUI that you can’t do in a Catalyst target, and the limitations often feel quite arbitrary.

Given the way macOS and the iPad are converging, doing everything in the same target seems to be the right way, which pushes me towards Catalyst. I’m not sure though, and I’m often sad (frustrated?!) when I discover that there’s something I want to do that is macOS-target-only in SwiftUI.

WOT NO ACTION ABSTRACTION?

For the macOS app, I definitely also want to support a menubar. With iOS 26 coming, I probably also want to support one on iPad.

Sidenote: Is it going to be easier or harder if I use Catalyst? I currently have no idea, so am going for Catalyst – the solution which requires supporting one target rather that two in Xcode.

I find it a little surprising that menus seem to require a different approach to toolbars - using Commands, CommandGroup and so on, rather than Toolbar, ToolbarItem etc.

Whilst they aren’t identical, menubars and toolbars both seem to be doing roughly the same job of hosting a set of “actions” that can be applied, either globally or to the current selection.

In both cases these actions can get enabled and disabled, can change their name or behaviour based on context, or can be substituted for other actions. They both require a title, and probably also an icon. They can both be grouped, and sometimes the items in the group are mutually exclusive. They should probably both be able to have a keyboard shortcut assigned. They probably should both support undo.

I wonder if I’m missing something, but I’m surprised that SwiftUI doesn’t appear to support an nice abstraction that can be used for both.

Lead Us Not Into Temptation…

I like a good abstraction. Probably more than is strictly healthy.

I am wary of diving in to make my own action abstraction, but I do think that I will probably need to.

As well as the ability to make a menu item or a toolbar item from an action, I would like to eventually support undo.

I also want to support some form of opt-in anonymous analytics that I can use to work out what people are doing with the app. I don’t want to spy on people unnecessarily, but if I know which features people use, and which ones they miss, then I’ll have a better idea of where to focus my efforts.

Finally, I’m thinking of a payment model based on micro payments, and I like the idea of basing this not just on a crude measure of “did the user open the app today?”, but on something much finer-grained which understands how much someone is using the app.

All of this means that I think I probably should invest the time in some sort of simple abstraction, which allows me to package up operations and attach them to menus, toolbars, standalone buttons, or other events, and have them execute wherever they are invoked.

First though, I’m going to go back and watch a few WWDC videos… just to check that I haven’t missed something that’s already there!

PS

Don’t forget that there’s a Test Flight link available for the Stack. Give it a go!

Matching Strings In Unit Tests

2025-09-03T00:00:00+00:00

Whilst working on ActionBuilderTool, I was getting an annoying test failure.

A couple of my tests generate a workflow, and compare it against the expected text, which is 50+ lines of text.

Unfortunately the basic #expect(generatedYAML == expectedYAML) implementation doesn’t do a very good job of explaining where the mismatch is…

When the #expect macro gets a false result, it is clever enough to dump the source code of the comparison expression you give it, and also the values of both sides of the comparison.

That’s pretty useful for small values, but for large blocks of text, it’s not so helpful.

As it happens, I’ve been here before, with the XCTest framework, and I wrote myself a bunch of utilities, including some better code for matching two strings, lists or dictionaries against each other, and identifying the first difference between them.

The XCTestExtensions package contains more than just the comparison code, and at some stage I realised that the item matching part of it might have wider applications, so I factored it out into its own package.

I didn’t know at the time that the Swift Testing framework was on the horizon, but it turns out that it was, and that having Matchable as a standalone package is very helpful as it allows it to be used directly with Swift Testing based tests.

The README I wrote for Matchable is actually pretty comprehensive, and for full details, I suggest you go there.

I’ve copied & pasted some highlights below.

Matchable

The matchable protocol defines a way to compare two objects, structures or values for equality.

Unlike the Equatable protocol, Matchable works by throwing an error when it encounters a mismatch.

You can view this as an assertion of equality. For this reason, the primary method is named assertMatches.

This makes for compact code since you don’t need to write explicit return statements for every failed comparison.

It also allows the protocol to handle compound structures intelligently.

If a matching check of a structure fails on one of its members, the matchable code will wrap up the error thrown by the member, and throw another error from the structure.

Any catching code can dig down into these compound errors to cleanly report exactly where the mismatch occurred.

Usage

You can check that two values match with:

try x.assertMatches(y)

A sequence of checks can easily be performed – the first failure will throw, causing the remaining checks to be skipped:

try int1.assertMatches(int2)
try double1.assertMatches(double2)
try string1.assertMatches(string2)

A type can implement matching by conforming to the Matchable protocol, and defining the assertMatches method. Inside this method it can perform the necessary checks.

If it finds a failure, it can throw a MatchFailedError to report the mismatch.

Implementations of assertMatches are provided for most of the primitive types, and a few Foundation types (I’ve just done the ones I needed for now - pull requests gratefully received…).

Compound Types

Although you can match primitive value types, the protocol comes into its own when performing memberwise matching of compound types (structs, objects, etc).

In this case a type can conform to the MatchableCompound protocol, and defining the assertContentMatches method.

This works the same way as the basic assertMatches method, except that if a check throws an error inside this method, the error will be wrapped in an outer error reporting that the whole structure failed to match.

See the README for more details on this.

Swift Testing

With Swift Testing, you can just marks your test method as throws, and then place a call to try thing1.assertMatches(thing2) into it.

The nice thing about Matchable is that the error it throws contains enough information to pinpoint where the match failed.

If the assertion fails, you’ll see a relatively well formatted report showing where it went wrong.

For example:

------------------------------------------------------------
Matching failed at line 149 of ActionBuilderCoreTests.swift.
------------------------------------------------------------

strings different at line 13.

was:

"- name: Make Logs Directory"

expected:

"- name: Select Xcode Version"

There’s Gold In Them Thar Hills

That’s enough rambling for this diary entry. I have been doing more on ActionBuilder itself, but that will have to wait for another day.

One last note though. Like quite a lot of the code I’ve written over the last few years, I just sort of got on with Matchable.

I made it open source, but never really got around to telling people about it.

Partly I just didn’t find the time to write a post like this. Partly perhaps I didn’t want to commit to fully supporting a whole load of open source software. Possibly imposter syndrome kicked in and I didn’t think it was good enough.

Whatever!

The point is, there’s quite a lot of stuff like Matchable knocking around at https://github.com/elegantchaos.

I invite you to take a look at it, and especially, to get in touch if there’s anything that looks useful. It may be a bit out of date, but it’s likely to be easily fixable and I will probably be keen to help to bring it back to life.

Similarly, I can’t help feeling that a lot of it has been overtaken by advances in Swift, or other third party work. I’m equally interested to hear about that too.

More On Workflow Generation

2025-09-02T00:00:00+00:00

In my last post about automating Github actions workflow generation, I said I’d revived the old tool I wrote a while back, and got it working again (for some value of “working”).

I already figured that there was more to do, and then the post got an unexpected mention in iOS Dev weekly (thanks Dave!).

Once I’d got over my initial shock that anyone was paying attention (!), I decided that I definitely ought to spend a bit more time in the code… so that’s what I’ve been doing today.

Swift Versions

One thing that the tool lets you do if you want is to test your package against multiple versions of Swift.

If you’re just building macOS or Linux targets, it uses Swift Package Manager, and so can use Swiftly to choose the Swift version.

If you’re building for iOS/tvOS/visionOS/watchOS however, it has to use Xcode instead.

I had some code in the tool to support this; it works out which route to take, and then potentially chooses the right mac-os runner image, uses xcode-select to pick the right Xcode, and even if necessary downloads a toolchain installer and runs that to install a different toolchain into Xcode.

All of that was fine and dandy, but I realised that some of the macOS images I was trying to use on Github actions no longer exist, and some of the Xcode versions have been dropped. So quite a few of the old Swift versions I was supporting had no easy way to be tested.

There might be some way to work around this, but ultimately I came to the conclusion that just dropping support for these Swift versions was the pragmatic thing to do.

As a result, the oldest version of Swift that is now supported by the tool is 5.7. If you run the tool over a package using an earlier value for swift-tools-version, it will still generate a workflow, but it will default to testing with Swift 5.7 (and with the latest Swift).

Prettier Output

Something that always annoys me with CI is when the output is excessively wordy. Especially when it means that you have to wade through a ton of inconsequential logging to find the one piece of information that’s actually useful.

When I wrote the original tool, I was combating this for Xcode builds by piping the output through xcpretty.

Since then though I’ve discovered xcbeautify, which I prefer. It has the advantage of being written in Swift, and being easily installable using brew install xcbeautify.

So another thing I’ve done is switched over the tool to use it instead for any macOS jobs.

I pipe the output of xcodebuild into it, but also the output of swift test, since it understands that.

In addition to this, I’ve changed the swift build commands to take the --quiet flag.

All of this reflects my personal preference, which is to have very little output if everything is working, and just the errors if something is broken.

If anyone reading this is using the tool and wants a different approach, we could easily make it a config setting. Let me know (or open a PR!).

More Coming…

Other than that, I’ve tidied up the README a bit, and added some usage instructions if you just want to run the tool.

I did start taking a look at reviving my Swift tool plugin, but there’s more work to do there. I’m seeing some weird behaviour where the plugin hangs after it launches the tool, and as yet I’m not quite sure that I know why!

At some point I will do some more work on all of this…

Eh? Aye!

2025-08-29T00:00:00+00:00

I’ve been using Copilot for a couple of years, but mostly limited it to grunt work; auto-completing obvious code, filling out obvious patterns, and drafting comments (some of which I accept).

Recently I’ve been looking for an excuse to try out something more ambitious. More vibe-codey. Whatever the fuck that means.

Then it hit me. I built this website a few years ago with Jekyll and Bootstrap, which frankly are things I know little about. The world has moved on, but the website hasn’t.

This seemed like a perfect test case…

(sidebar: If you’re confused by the title of this post, try saying it with a thick Glasgow accent. You’ll get the idea.)

A Skeptic I Am Not

I think I’ve got a fairly good understanding of the current state of “AI”.

I get how LLMs work, what they’re doing, and what they’re not doing.

I think I understand why people in general (and creative people specifically) are scared of what AI is going to do to their careers (and whole industries).

Finally, I am also as cynical as they come, and so I’m certainly skeptical about the way that some form of AI-like functionality is being levered into every damn product around.

However…

Force Multiplier

I firmly believe that the coding tools that are beginning to emerge, in the right hands, are force multipliers.

Like any powerful tool, you can certainly make a monumental mess, and I’m sure that lots of people will.

In my professional life I think I’ve already seen examples of code that I suspect was generated by an AI tool, where the person using the tool was not sufficiently experienced to understand the implications of the code. Which bits to keep, which bits to change, which bits to discard because they duplicated functionality elsewhere or just didn’t fit the brief.

This, however, is very little different from the examples I’ve also seen where someone has copied & pasted some dodgy code from a Stack Overflow answer. Given that the AI tool was probably trained on those Stack Overflow answers, the only difference seems to be that the tool actually has a better chance of adapting the code to fit the user’s codebase than a naive copy-and-paster has.

So yes, vibe coding tools and other programming assistants are going to help some not very good programmers blag their way in the world for a bit longer and make some tangled code bases.

However, based on my experience today, if you already sort of know what you’re doing, oh boy do these things help.

Refreshing The Site

To be clear, I haven’t asked Copilot to change a single line of actual content on this site.

However, for a laugh, I did ask it to make a first draft of this post.

In it, it described the reason for refreshing the site.

This site has been running on the same basic design for years - a fairly standard Jekyll setup with Bootstrap 3, some custom CSS, and a layout that, while functional, was starting to feel quite dated. The typography was uninspiring, the color scheme was bland, and the overall feel lacked personality.

Brutal Copilot. Brutal - you bastard!

However, it may have had a point. The site has always been a bit quirky, and intentionally minimalist, but I wanted something that felt more modern. I suppose what I’m aiming for is more “I like it simple”, and less “I can only do simple”…

I most definitely have a sense of an aesthetic that I’m after, but I do most definitely not know much about modern web standards, and I’ve forgotten most of what little I did know about the technology I used to build the previous version.

I did think about switching to a whole new engine, but I really wanted to preserve all the previous urls and just clean up the appearance.

So what I asked it to do first was to “modernise the look and feel of this website”.

From this simple prompt, it basically did the following*:

Migrated from Bootstrap 3 to Bootstrap 5
Replaced the old CSS with a modern SCSS system using CSS custom properties
Switched from local Bootstrap files to CDN delivery
Replaced Font Awesome with Bootstrap Icons (also from CDN)
Added proper dark mode support via prefers-color-scheme
Implemented a clean, responsive grid system
Introduced subtle gradients throughout the design for depth without being overwhelming
Created better visual hierarchy and spacing throughout

(* again these are basically it’s own words, although I’ve edited them a bit. This time I mostly agree though!)

That was a pretty major upgrade, it would have taken me a lot of time, and I would have broken things. It did it in a bunch of logical steps, which it explained, and it didn’t just to technical things, it also applied some taste in the way it adopted gradients, adjusted spacing, and so on.

How easy this step was rather blew me away, so I got a bit more ambitious.

I’d been experimenting with a slightly friendlier hand-drawn version of my logo, and I wondered if I could change the main title fonts to match it. I’d also been meaning to add an optional subtitle, and there were some little layout details that had been annoying me forever, so I dived in a bit deeper.

Fiddling With Fonts

I had a vague idea of a font I liked, but it wasn’t quite right (a bit too close to comic sans for my liking!).

So I asked it to change the site to use it, but then if there was anything else out there that was similar. I tried to describe what it was I liked and didn’t like about the font I sort-of liked.

It basically suggested a few more fonts, and made a style sheet where it was easy to switch between them. We then had a bit of a dialogue where we homed in on my eventual final choice.

This all felt very natural. It was as if I was sitting alongside someone who knew a lot more about all of this stuff than I did, and who was helping me. I said what I thought I wanted. It tried its best. We iterated.

I have to say that the process was more than “ok”, it was actually pleasurable. It had all the good bits of what I would have experienced if I’d been tinkering on my own, but it expedited the discovery part, and when it applied suggestions, it didn’t make the dumb mistakes that I would have made, like forgetting semicolons or missing a place where I needed to change something.

My other changes went a similar way. We didn’t get there right away, but we honed in pretty fast with a little bit of iteration.

Honestly, I couldn’t have done what I did today in anything like the time it’s taken, without using Copilot. I think it’s probably taken about four hours maximum, and at least an hour of that was writing this post.

Conclusions

I’m not starry eyed about this technology, but I don’t think we can ignore it.

If you’re an experienced developer, I think it can really help, but probably only if you know what you want and can clearly articulate it.

You need to know good code from bad, and need to be able to direct the tools accordingly.

It probably also helps if you can start off from a template or existing project that is already well structured and consistent.

In my career I’ve come across a range of developers with different skills and different approaches.

The ones I tend to gravitate towards are the people who don’t stop when the code works, they keep going until they understand why the code works. They tend to have opinions about coding standards and techniques, and methodologies. They like tools, and building tools. They probably have a favourite programming language (or ten). They like talking about this stuff, and reflecting on it, and generally are engaged in the process of software engineering.

Honestly I think vibe coding tools, AI coding assistants, whatever the hell we call these things, are absolutely built for such people - and for me, since I put myself in this category.

Change is scary, and we are most of us insecure about getting left behind. We’re also rightly suspicious of the Emperor’s new clothes. I don’t intend to dive in head first and forget everything I already know.

I do intend to attempt to embrace the potential though - which seems to me to be substantial.

Automating Github Action Workflows For Swift

2025-08-28T00:00:00+00:00

Today I’ve been looking at reviving a tool that I built a while ago, which automatically builds a Github Actions workflow file for a Swift package.

The functionality for this was originally part of my Action Status utility, which is a small macOS app for monitoring the status of Github Actions. That app also needs a bit of TLC, but that’s for another day.

Action Status was originally just for monitoring CI jobs (like the old app CCMenu), but at some stage I realised that all the workflow files I was monitoring were pretty repetitive and fiddly to build. I figured that a tool ought to be able to do a better job of writing them, and so a little bit of feature-creep ensued.

Some time later, Swift plugins became a thing, and I realised that having a plugin to build the workflow might be good. Potentially you could even have a Swift Package which automatically built and updated its tests.yml file whenever it was compiled.

As a result of this, I factored out the workflow generating code from Action Status and moved it into a new ActionBuilderCore package, and then built an ActionBuilderPlugin with it.

The plugin worked, but I hit a few snags along the way, mostly relating to the relative immaturity of the Swift plugin support, plus the fact that adding plugins to a package means adding dependencies, which isn’t always what you want.

I also got distracted by other projects, and so both Action Status and the workflow building plugin have languished a bit. For a while now though I’ve been meaning to revive the basic concept.

I’m not sure about the plugin, but just having a tool that you can point at a Swift package source folder, and which will write a working Github Actions workflow.yml file into it, still seems like a pretty useful thing. Maybe you just run it manually, or from an Xcode build phase?

One refinement I do want to make is to get as close to a zero-configuration tool as I can.

The original implementation offered quite a lot of choice, and the UI was full of switches and selectors allowing you to turn things on and off. You could pick which versions of Swift to build against, which configurations to try, which platforms, whether to run tests, etc, etc.

This doesn’t suit a plugin so well, or even a tool that you invoke manually. I added a config file that you can drop into your source directory, but it would be nice if you rarely needed to do so. Really you just want to be able to point the tool at a package and have it work out that stuff and do The Right Thing™.

Luckily I realised that most of this information can be extracted from the package manifest itself.

Another problems that the tool implementation suffered from is that it needs to be taught which versions of Swift exist, and how to install/select the right one in the Github runner, and which runner environment to use. When testing Apple platforms, it also needs to know whether to use SPM or Xcode to perform the build and testing, since annoyingly SPM cannot easily be persuaded to build for iOS/tvOS/watchOS/visionOS.

Luckily this picture is slowly improving. Swiftly now exists, and can solve some of these issues for me. I can definitely use it to install Swift on the runner instances, but I think with a bit of luck I can also use it for Swift version discovery, which will allow me to simplify my code and remove the need for the tool to be updated when new Swift versions are released.

The tool still needs to do some shenanigans in order to support building and testing on iOS/tvOS/watchOS/visionOS. Apple currently say that they have no plans to support a unified build interface for all platforms, which is sad. That said, a lot of the underpinnings required to support it now seem to be in place, including the fact that Xcode and SPM are moving towards a shared build system, and under the hood most of the tools appear to support specifying a toolchain and/or triple to indicate which platform you’re building for, as opposed to which one you’re building on. For now I think I still need to manually teach the tool about the Xcode versions and their corresponding Swift versions, but I’m hoping that this, too, may eventually be easier (or unnecessary).

Anyway, all of the above are things that I’d quite like to tidy up and roll into a working plugin.

The first step today was to revive the command line tool, which is part of the ActionBuilderCore package.

I’ve kicked the tyres on this, got it working with swiftly, and taught it about Swift 6.0, 6.1 and 6.2-snapshot. There’s more to do, but hopefully I will slowly get it polished up again.

This Is Not GTD

2025-08-06T00:00:00+00:00

I’ve been working on a simple To-Do list manager, called The Stack.

In a way you could say that simplicity is the point of The Stack, but actually I think the point is “reducing anxiety”.

It’s not based on a particular methodology, though it may share that traits with Get Things Done. I’m not a GTD person, so I have no real idea!

Recording Is Key

The Stack is inspired by my own tendencies. Often, whilst I’m working on something, I have a thought: “I must remember to…”. Or something, “It occurs to me that I could…”.

Usually the thought is related to what I’m currently doing, but not always. Generally I feel the need to record the thought; this is mostly for reassurance.

I regard my memory as unreliable, and so I worry that I’ll forget whatever it is that I just thought of.

Actually, a lot of the time, I would remember it, or it would occur to me again later, but I can’t be sure.

So the primary motivation is just to make sure that I’ve noted something down, and can move on.

Big Bag ‘O Stuff

I could spend a lot of time trying to organise all these thoughts into categories and separate lists.

More than just could… would… if allowed. I have a programmer-ish brain.

My brain likes (kidding itself that it is?) imposing order on the chaos.

This is a seductive trap.

A lot of the time, the thought I had ultimately doesn’t matter as I don’t get around to it.

Sometimes this is because it was a bad thought.

Sometimes it’s that the context becomes obsolete before I do anything about it.

Mostly it’s just that something more important gets in the way.

Time spent organising these items into a complex set of categories would mostly be time wasted.

The Stack tries to help me to avoid the trap, through simplicity.

I can’t waste time putting my items into neat categories if there is just one big heap of items.

I can’t waste time figuring out what order to do things in if there’s no way to reorder the things.

That’s the theory anyway!

But What If…

Ok, the picture painted so far is a little extreme, and so I make a couple of concessions.

My first concession relates to ordering items.

If I think about an item again, or decide it’s high priority, I can pull it to the top of the stack. In effect, it’s as if I just made the item, and so it becomes the “newest”.

This could be used as a crude way of re-ordering the entire list - by pulling items in the right order - but you’d have to be insane to use it that way.

It’s largely there for reassurance, and I find that I rarely use it.

Context and Filtering

My second concession relates to categorising items.

Yes there is deliberately just one big stack of stuff, because when I’m recording an item, I don’t want to get hung up on which items go where.

Sometimes I might want to look at a subset of that stuff that relates to a particular context though, and so it’s helpful to be able to associate items with context in some simple way.

I’ve chosen to take a leaf out of social media’s book and to do this with #hashtags.

Hashtags are as low impact as I can make them. All you have to do to tag an item is include a hashtag in its text description.

The tags used in an item will appear next to its entry in the index.

Tapping on that tag will focus the index on a single tag, by temporarily hiding all the other items that don’t have the tag.

I use this to focus in on the items relating to the project I’m currently working on.

Order / Chaos

I deliberately don’t want you to feel like you have to use tags.

I especially don’t want you to feel pressure to curate a list of tags up-front.

Usually you can just invent a tag as you’re typing your item. Your brain will probably pick the same tag for the same context.

You can look at a list of all the tags if you need to. If you do get a tag name wrong, there is a way to rename a tag, which will fix up the #hashtag in the text of all items that referenced it. Hopefully though this won’t be needed too often, and won’t turn into a dangerous rabbit hole that you fall down.

I also don’t want you to worry about having a big stack of stuff.

Or about losing track of everything that’s not near the top.

That’s fine, in my book. It’s there if you need it, and there are ways to find it, by filtering or searching. You probably won’t though.

It’ll Never Work

Probably.

Who knows?

If you’d like to try it though, I’ve got a Test Flight link.

You need macOS 15.5 or iOS 18.5. I might make it require iOS/macOS 26.0 later - so that I can use some new features of SwiftUI - but not yet.

There are known issues. No doubt there are also unknown issues.

If you do fancy giving it a go, let me know how you get on.

To Do...

2025-07-24T00:00:00+00:00

There are certain personal projects that feel like cliches, due to the tendency for any and every developer to want to embark on them.

A To Do list manager is a classic case.

Nevertheless… I find myself here.

Motivation

I do have some professional motivation - it’s an excuse to catch up with some recent Swift/SwiftUI developments.

It’s the first project where I’ve used SwiftData, rather than Core Data, for example.

It’s also nice to have a fresh project where I can set the minimum platform high.

It’s maybe even an excuse to mess around with AI/vibe/whatever-we-call-them tools.

My main motivation though is just that I’ve never really found an app that works just right.

Just Right

As far as I know, my version of “just right” doesn’t correspond to a specific methodology. It might be similar to GTD or something like that, but I don’t think it’s exactly the same. Although, honestly, I haven’t done any research around this topic, I’m just following my gut.

At the moment, I’m calling my approach “The Stack”.

If I was high-falutin’ enough to claim a methodology, this would be it:

capturing ideas fast when they occur is key
there is only one list (“the stack”)
it is ordered, but above the top few items, the order probably doesn’t matter much
new stuff goes on the top
you can pull an item to the top
being able to categorise items can be helpful
categorisation should be easy and unstructured (I use #hashtags)
being able to focus on a hashtag is helpful
items can be in more states than “to do” or “done”; “on hold”, or “rejected” for example¹

That’s pretty much it.

If I had to put the approach into words more generally, I’d say that it’s all about staying focussed and removing choice.

Having more than one list would force me to choose where to put something.

Having the ability to drag items into any order or arrange them in a hierarchy would seduce me into tinkering with the order instead of tackling the items².

It’s helpful to be able to work on items from one project at a time, and so I do allow you to focus the stack down to items with a given tag. That’s pretty much all the focus I feel the need for though.

Wanna Help?

If any of this sounds vaguely interesting, I’ve got both macOS and iOS TestFlight builds that I’m pushing to regularly.

They are early versions, and the UI/UX is basic. I’d love some help with that.

Editing item text is a bit clunky right now. I wish SwiftUI had more support for auto-completion in the text editor. I will eventually polish this to try to make it as easy as possible to tag things, and I might add support for styled text (probably via Markdown).

Items are persistent, even when deleted. They just stay in the database, with their status set to deleted (which is different from the completed status).

Syncing between devices is automatic, and iCloud based. No plans to change that right now, but in principle any sync mechanism would be ok.

The project is closed source, but it’s not rocket science and I may well open it up.

It shouldn’t be regarded as a safe place for the only copy of your items right now. I know that makes it hard to really use, and I’m keen to get it solid. I think it is pretty solid, but I don’t want to make promises I can’t keep. Don’t blame me if it eats your items…

If you’re at all interested in giving it a try via Testflight, or just giving me some feedback on the idea, give me a shout on my personal mastodon account.

Footnotes:

This feels like a complication, but I think it’s useful. It will be interesting to see how much I use it. ↩
I suspect though that this is feeding mild latent OCD tendencies, like arranging my pencils in neat rows. It’s vaguely satisfying, but is it really doing anything useful? ↩

Installing Swift on Windows 11

2024-11-08T00:00:00+00:00

TL;DR – use the winget method to install everything

I recently set up a virtual machine running Windows 11, on an M1 Studio Mac.

I wanted to test Windows builds of various Swift packages on it, so the first order of the day was to install Swift.

I went to the Windows installation instructions on the Swift website and started downloading and running installers, including the Visual Studio one.

Lots of downloads later, I tried to build a swift package, and hit an error could not build C module 'SwiftShims' which was a clear indication that I’d somehow cocked up the installation.

I tried to reinstall AllTheThings, but couldn’t fix the issue.

Eventually I noticed that there was an alternative set of instructions on that page, for doing the install on the command line using WinGet.

Long story short: use this method!

After uninstalling everything I’d manually installed, and following the winget instructions, everything is up and running fine.

I’m not sure why this method isn’t given more prominence, as it’s clearly more deterministic and harder for a mere human to mess up!

Xcodebuild, Plugins and Platforms

2024-10-11T00:00:00+00:00

For quite a while, I’ve experienced what I thought was a bug when using xcodebuild to build projects for iOS (or any non-macOS platform), if the project or one of its dependencies used a Swift Package Manager plugin.

I typically invoked the build with something like this:

xcodebuild archive -scheme MyProject -sdk iphonesimulator -destination 'generic/platform=iOS Simulator' -archivePath "archives/MyProject-iOS_Simulator"

The build would seem to be working, but would fail because the plugin was being built against the iphonesimulator SDK, and not against the macOS one.

This puzzled me, and for a long time I thought that it was a bug, and so eventually I got round to reporting it.

It turns out that this is expected behaviour: applying -sdk iphonesimulator will force that SDK for all targets, including plugins.

The trick here is just to omit that argument. The -destination argument is sufficient to tell Xcode which sdk to use for the targets that will actually end up as part of the built products, but allows it to also choose the correct sdk (macOS) for any plugins.

I’m not certain, but I also suspect that this mistake of mine stemmed from the fact that I’ve been using xcodebuild since long before the -destination argument existed, and certainly since the generic/platform= pattern existed, and so I hadn’t really considered that it was sufficient.

What Next?

2024-09-03T00:00:00+00:00

I recently finished a fairly long-running contract with Formation Games, working on Club.

It was a fun project and a really great team who I loved working with. The one thing I perhaps didn’t love was that I was mostly using Flutter & Dart, with some Go thrown in. I’m not a Flutter hater, but I couldn’t honestly say that I loved it. Nor am I am Go hater, but I really did not get on well with the idioms that seem to be prevalent. Maybe I’m just too old and set in my ways :)

At the moment I’m taking a bit of time to recharge my batteries, and to think about what I want to do next.

Part of that has been trying to figure out where exactly my perfect project is located. Not geographically, but in terms of tools used, industry served, and so on.

I haven’t reached any solid conclusions, but I drew a big diagram on my whiteboard, with some headings circled.

Swift inevitably ended up floating in the middle, with the other headings orbiting.

Then I positioned some possible projects or products between the headings. As an overview, it all looks quite plausible!

These were the main headings:

Swift: I prefer it to Dart, and definitely to Go. Because I’ve been off in Flutter-land, I’ve not had the chance to really work with the new concurrency support – so would welcome the chance to catch up there.
Games: Exacty what “working in games” means is a bit nebulous, but I really enjoyed being back in the industry, and would like to stay there, or somewhere games-adjacent. Unfortunately the kind of games I like are big sprawling RPGs like Skyrim and Fallout. I think I may be too old to contemplate becoming a small cog in one of those massive machines, and I doubt if they’d have me. It’s also hard to set out to write one of those on one’s own. I suspect that’s what I’d actually like to do if I had the chutzpah (and near-infinite time). As a potentially never-ending side project, developing a Skyrim-like engine is quite tempting…
Desktop: Typically I end up working mostly on the client-side of things, often as desktop macOS apps. That’s still fine with me.
Mobile: I also work on mobile a lot. That’s also fine. Most of my experience is iOS. I’m open to Android (but see above about Flutter!).
Server: That said, I have a web-related side project that uses Swift Vapor, and I’d quite like to get back to at least a basic familiarity with that side of things.
UI: I do a lot of UI work, and generally if I’m starting something I use SwiftUI. I’m still ok with that, but as part of my recent work I’ve been building more of a bespoke UI toolkit (as I did once many moons ago for Football Manager). Whatever I do next, I quite like the idea of breaking out of the SwiftUI straight jacket, and going for a more game-like custom UI.
Engine: I’d quite like the excuse to get familiar with a 3D games engine / ecosystem. Unity would probably have been the choice once upon a time, but they blew it I think. Right now I’m pretty attracted to Godot. This is greatly helped by the existence of SwiftGodot. It’s possible that I can have my cake and eat it too.
Mods: The other way to get a bit of a gaming fix is working on mods. I’ve done a bit of this for Skyrim/Fallout, and am thinking that it might be fun to do something in that area, maybe for Starfield. Mutagen looks interesting, though sadly it means not working in Swift. That’s pretty inevitable for mods though.

This is all just food for thought at the moment, but I can see a hazy picture coming in to focus which makes some sort of sense… maybe…

Copilot Thoughts

2021-07-05T15:22:55+00:00

The furore surrounding Github Copilot interesting.

I’m no lawyer (nor do I play one on TV), but my feeling is that it may expose a flaw in the FLOSS community’s ideas about ownership of code.

If so, this is a good thing. The flaw (if it exists) has not been created by Copilot. It was already there, it just hadn’t come to light.

Is This OK?

Anyone who’s been coding for a while will have come across the situation where you’ve found some code with a license you can’t use, you’ve used the act of reading (or maybe even debugging) the code to teach yourself the solution to the underlying problem, and then you’ve written new code.

Then maybe you’ve felt uneasy and wondered if you’ve broken the rules.

Maybe all you did was cynically copy & paste and change a few variable names - in which case you probably did break the rules. Maybe though you genuinely rewrote it all from scratch. Maybe after rewriting you pretty much ended up with the same code because - well - that’s the best expression of the underlying solution to the problem you’re trying to solve.

Who Owns What, Exactly?

For any such situation, there’s going to be a blurry line. What did I copy here, and what did I create myself? The implementation? The algorithm? The expression of the algorithm in the context of the particular languague I’m using? The implementation in the context of the problem I’m applying it to?

Furthermore, how is this process essentially different from the one undertaken by the author of the GPL’d code?

Can I be sure in any way that they themselves weren’t just re-expressing something that has prior art?

Granularity

To look at it another way:

For any sufficiently small fragment of code, there’s likely to be a canonical way to express it. Taken to an extreme, a single line may well be infintely rewriteable, but one formulation is probably clearer, more compact, or better meets your particular criteria than any other.

In most cases it would be self-evidently ridiculous to assert that the GPL license applied to a body of code actually applies to each line in isolation.

If the line includes variable names, function names, comments, or other incidental metadata, it could be argued that they are not directly related to the pure meaning of that line. They do have meaning and value, but probably only in the context in which the line exists.

These names can be replaced, rewritten, or even randomised; this may obfuscate the meaning of the code, but it doesn’t stop it working.

Once you get to a small enough granularity, the same line of code almost definitely exists in countless other programs, both open and closed source, GPL’d or liberally licensed. The names might be different, but the meaning of the code is the same. The machine code instructions emitted by the compiler will probably be the same.

What Is Knowledge Anyway?

So what exactly are we arguing about here?

If something like Copilot is taking chunks of GPL’d source and pasting them into someone else’s program, how many contiguous lines does it have to paste before there’s a problem? Is there an arbitrary N number of lines that’s ok, where N + 1 is not ok?

If Copilot applied some natural language processing to infer the context that the lines are pasted into, and then automatically renamed the variables (or even rewrote comments) to use words appropriate to the new context, would that now be ok? Same code - different names?

If it randomised the names and stripped all comments, would that be ok?

Maybe we arrive at some formulation that states that a line is ok, but a whole function is not. Are we then allowed to apply the copying process to each line in turn? Refactor the function into multiple smaller functions and use them?

Known Unknowns

This all feels very wooly to me. The code represents a muddle of knowledge, experience, style, and algorithm.

Any assertion of the right to control how each of these things are used in isolation, or even recombined into a larger whole, feels like over-reach.

Worse, it may well be politically dangerous. If an entity can assert their right to apply copyleft to small fragments of code, doesn’t that logically mean that they are claiming ownership of the underlying meaning of those fragments?

Doesn’t that put us into territory where another entity can assert ownership of the underlying meaning of other fragments and choose to patent them or in other ways suppress their use by others? Isn’t that sort of what the Free/Libre side of the community is trying to avoid in the first place?

Nothing Is Simple

I’m not claiming any great insights here, and certainly not offering any solutions.

It just seems to me that the problem is a lot knottier than some people are making out.

It’s not clear to me that what Github Copilot is doing is breaking the rules, any more than I am when I read someone’s GPL’d code and learn something from it…

The Matchable Protocol

2021-04-30T12:15:00+00:00

For the last few years, the default setting for all of the Swift code I write has been open source.

As a result, I’ve accumulated a vast number of Github repositories and Swift Package Manager packages.

However, I’ve been really bad at telling people that they exist!

This post is an attempt to start to fix that, by talking about one small package I’ve recently created: Matchable.

First though, some disclaimers:

Caveat Emptor

Part of the barrier to telling people about things I’ve done is sheer time it takes to write even quite a simple post like this one.

So my first disclaimer is just to say that this post is mostly a re-hash of the README file from the Github Repository. Nothing wrong with that I think, but just to be clear…

My second disclaimer is that this is work-in-progress code from the real world.

I’ve encountered a few people who subscribe to a fundamentalist view of open source code: that it’s useless unless it is fully polished, fully tested, 100% supported and actively maintained.

I understand this point of view; we’ve all encountered code that makes great claims and turns out to be broken or mostly unfinished.

Respectfully though, those people are wrong.

Aiming for perfection is setting the barrier way too high. I am as insecure as the next person when it comes to showing my workings in public. I’ve been a professional programmer for more than three decades, but I still suffer from impostor syndrome.

It’s tempting to hide away, but I’m trying to fight the urge, and I’d like to contribute in some small way to an environment where we aren’t scared to risk being wrong.

I offer up all of my open-source code in this spirit. It’s not perfect, because I am busy, and because I am still writing it. I find this code useful, and I hope someone else might. If you do find that it is fundamentally broken, please tell me why. That way I learn something.

That said…

Matchable

The Matchable protocol defines a way to compare two objects, structures or values for equality.

Unlike the Equatable protocol, Matchable works by throwing an error when it encounters a mismatch.

You can view this as an assertion of equality. For this reason, the primary method is named assertMatches.

This makes for compact code since you don’t need to write explicit return statements for every failed comparison.

It also allows the protocol to handle compound structures intelligently.

If a matching check of a structure fails on one of its members, the matchable code will wrap up the error thrown by the member, and throw another error from the structure.

Any catching code can dig down into these compound errors to cleanly report exactly where the mismatch occurred.

Usage

You can check that two values match with:

try x.assertMatches(y)

A sequence of checks can easily be performed – the first failure will throw, causing the remaining checks to be skipped:

try int1.assertMatches(int2)
try double1.assertMatches(double2)
try string1.assertMatches(string2)

A type can implement matching by conforming to the Matchable protocol, and defining the assertMatches method. Inside this method it can perform the necessary checks.

If it finds a failure, it can throw a MatchFailedError to report the mismatch.

Implementations of assertMatches are provided for most of the primitive types, and a few Foundation types (I’ve just done the ones I needed for now - pull requests gratefully received…).

Compound Types

Although you can match primitive value types, the protocol comes into its own when performing memberwise matching of compound types (structs, objects, etc).

In this case a type can conform to the MatchableCompound protocol, and defining the assertContentMatches method.

Keypaths

As a convenience, we also define a form of assertMatches which takes a key path or list of key paths, and calls assertMatches on each path of two objects in turn.

This helps to keep down the amount of boiler-plate code to a minimum.

Here’s an example combining keypaths and the MatchableCompound. This tests the matchability of a structure that has 13 properties, and manages to do it with a minimum of boilerplate.

extension Task: MatchableCompound {
    public func assertContentMatches(_ other: Task, in context: MatchableContext) throws {
        try assertMatches([\.state], of: other)
        try assertMatches([\.name, \.icon, \.details], of: other)
        try assertMatches([\.started], of: other)
        try assertMatches([\.hasDescription, \.hasDuration, \.isScheduled], of: other)
        try assertMatches([\.duration], of: other)
        try assertMatches([\.scheduledHour, \.scheduledMinute], of: other)
        try assertMatches([\.streaks], of: other)
        try assertMatches([\.restDays], of: other)
    }
}

Note that currently if you pass a list of keys, they all have to resolve to members of the same type. Unfortunately this somewhat reduces the helpfulness of this method.

Unit Testing

The original motivating use-case for this protocol was unit testing, where it’s often necessary to compare two instances of something, and useful to be able to identify the exact point of divergence.

Whilst I still see this as the primary use for the protocol, I have split it out into a standalone package as it may be helpful in other places.

The fact that Matchable is different from Equatable is an advantage for unit testing, as it allows both to co-exist.

In your code, you might define Equatable to only check part of a structure (a unique identifier, for example).

This is good for efficiency in production code, but no use for test code where you really do want to know if all members are equal.

In this situation you can define a thorough check with Matchable, and use that for unit testing, without interfering with the efficient implemention of Equatable.

Initially this protocol was defined as part of my XCTestExtensions package.

That package includes some additions to XCTAssert which use Matchable to let you perform matching checks:

XCTAssert(savedModel, matches: reloadedModel)

This assert method catches any errors and presents them in a nice way by calling XCTFail, identifying the exact point of failure.

Because of the way the match-failure errors are wrapped for compound structures, the method can call XCTFail at all levels of the failure, which results in Xcode showing an error marker at all levels.

This can be helpful when tracking down a mismatch in a deeply nested structure.

Future

This is an early implementation, based on code pulled from elsewhere.

The API probably needs tweaking, and the methods definitely need documenting.

I also intend to explore the idea of using Swift’s introspection to automatically generate assertMatches for structures/classes.

In theory this should work well, but it’s possible that it will hit wrinkles.

All feedback, suggestions, pull requests and bug reports gratefully received!

Vapor 4 and Session Authentication

2020-05-01T16:08:00+00:00

I can only explain it as lock-down madness, but a couple of weeks ago I decided to have a little play around with Vapor.

What I wanted to do, initially, was just make a simple website that did user authentication. You could register, login, and logout. If you were logged in, it knew who you were. If you were logged out, there were things you couldn’t see.

Now I’m no web developer. Admittedly I did write a WYSIWYG html editor in Hypercard, in about 1994, but I’m no web developer.

Ok, I might have also written a complete CMS using Hypercard as a CGI engine for MacHTTP, also around that kind of time, but honestly, I’m no web developer.

If really pressed, I might admit to having had a job creating the first interactive shopping basket for Robert Fripp’s DGM record label’s website in about 1998¹ - a job which I had to learn Perl for² - but if that goes to prove anything, it is that I really am not a web developer.

Still, how hard could it be, right?

Spoiler Alert

So actually, it’s kind of not that hard, but…

It turns out that the latest version of Vapor - Vapor 4.0 - is quite new. New enough that the documentation isn’t finished, and there aren’t many tutorials or examples out there.

It also turns out that the authentication examples which do exist for Vapor 4.0, and indeed most of the examples for Vapor 3.0, focus on making API back-ends. Which is understandable, but not that helpful to me.

Add to this mix the fact that Vapor 3.0 is different enough from Vapor 4.0 that it will confuse a noob like me who is just copying & pasting things to find out how they work.

Finally, add to this the fact that Vapor is based on SwiftNIO, which has a Combine/Reactive-like declarative syntax for its networking operations, and a certain amount of bafflement ensues.

I think I’ve puzzled my way through it by now. I cannot offer up any great insights, but I will offer up the results of my labours in the hope that they might be a useful starting point for someone else. To be clear: this is still very much work in progress, and in any case, I’m only tinkering.

From my brief time with the framework (less than two weeks, during which time I’ve also been doing other things), I have to say that my impressions of Vapor are very favourable.

It seems pretty well thought out, nicely structured, and not too complicated. It’s just a shame that there wasn’t a ready-made template that fit my needs.

I’m still slightly less convinced about the whole SwiftNIO/Combine/React thing, as this Twitter thread illustrates.

It’s not that I think the basic approach of declarative-ness, and futures and compile-time checking of the information flow is wrong.

Just that I’m not convinced about the API naming, and I’m really not convinced about the readability of the resulting code, or the ability it gives one to reason about it.

I’m not sure that using language drawn from functional programming for the API operations is helpful. It’s technically correct, but I find it confusing when the problem domain I’m thinking about is really a stream or pipeline of “events”, or maybe even “network operations” or “database objects” - something like that anyway. It does not feel natural to me to describe this as a sequence of map, flatMap etc operations (even though that’s what they may be).

I’m also not sure that the clever type inference that Swift can do in these situations is always that helpful.

I mean, clearly it’s useful in some ways. It’s great to be able to statically check that a pipeline of asynchronous operations are passing along the right type of data to each other, and that a known number of outputs or error conditions can result.

It bloody confusing though when Swift gets confused. Usually this is because you got confused, but the resulting error is often cryptic, and pops out at an unexpected part of the chain. This is not a new problem, as anyone who has worked with SwiftUI, or indeed has any kind of shady past involved template meta-programming in C++ can attest.

Even when things don’t go wrong, as a reader of the code you end up with, it’s often not at all apparent what the underlying type is that’s being transferred from one closure to the next.

It’s not even that apparent sometimes if something is a futures, or an actual known values, and quite what the sequence of execution will be. Maybe my brain is still too used to procedural code.

There also seems to be a common pattern where you end up creating a nested sequence of futures inside a closure, only to then transform the result of the last one back into the input you got at the beginning - in order to ignore a few intermediate results from the pipeline and preserve a value from earlier so that you can forward it to a later part of the pipeline. There may be a clean way to generalise this pattern, but in quite a few of the examples I came across it’s just written out in long-hand, which makes it pretty hard to parse what’s going on if you’re just reading someone else’s code.

All in all I’d say that I like some of it, need convincing about some of it.

I also find myself wondering if this is a job for some funky custom operators (oh god…), or even some sort of domain specific language (oh god oh god oh god).

Either of these could make the code a lot clearer to follow. Or make it even harder to reason about and fuck everything up terminally.

Luckily, and I should probably emphasise this, I’m not a web developer :)

I was working at Abbey Road at the time. Yes, that Abbey Road. ↩
I still feel dirty ↩

Random Acts of Pragmatism

2020-03-06T10:04:00+00:00

I have been accused (by myself, mostly), of being a bit too much of a purist sometimes. It’s true that I do like things to have an intellectual rigour to them, but it’s mostly about being honest and clear with ourselves about what we’re doing and why. I welcome the application of common sense, and I’m fine with taking shortcuts as long as they’re consciously chosen for a good reason.

I’d like to think that I’m a pragmatist…

As I was adding an item to a list in some code this morning, it occurred to me that a tiny feature of list literals in Swift (and many other languages) is a great example of pragmatism.

What I’m talking about is the ability to do this:

let list = [
    "foo",
    "bar",   // <--- look ma, there's an extra comma
]

A purist might say that this is not correct (for some value of “correct”). It could be argued that it’s confusing to allow that extra comma. It could be argued that it implies that the programmer forgot to add the final item.

An experienced programmer, however, will often have had to edit long lists of things by moving existing lines around. When the last line of these lists didn’t have a comma, and they moved it to the middle of the list, they probably forgot to add the comma, and it caused an error. They also then exposed a new last line, with a comma, which also caused an error. Easy to fix, but annoying.

At some point during programming language evolution, someone asked themselves the question “does that error about the extra comma actually do anything useful?”. Is it actually ambiguous? No. Can we just get rid of it? Yes. Would that bring us a little bit closer to Nirvana? Possibly.

To me, this is a great example of a little random act of pragmatism.

Github Actions and Swift

2020-03-05T14:29:00+00:00

Bookish Development Diary, episode 8.

As I mentioned last time, I’ve been playing around with Github Actions, using them to build and test my Swift packages on a number of platforms.

They’re fairly easy to set up - you make a yaml file called something like Tests.yml, add it to the .github/workflows/ directory at the root of your repository, and commit.

The yaml file can contain a vast range of things, but for testing Swift what it usually boils down to some fairly standard steps.

First you select which system and tool versions to build on. For the mac, the macOS-latest image gives you the latest releases of macOS and Xcode. For Linux, there are Docker images available for Swift 5.0 and 5.1, as well as nightly builds of the latest Swift.

Then you clone your package with git.

Next you perform a build, using either swift build or xcodebuild build, depending on the platform you’re on.

Next you run some tests with swift test or xcodebuild test.

There are plenty of other things you can also do (for example posting notifications, uploading files), but a simple file that just builds & tests on the Mac might look something like this:

name: tests
on: [push, pull_request]
jobs:
  macos:
    name: MacOS
    runs-on: macOS-latest
    steps:
    - name: Checkout
      uses: actions/checkout@v1
    - name: Build
      run: swift build -v
    - name: Test
      run: swift test -v -c release

So far so good…

It’s pretty simple, and it’s free for open source projects, and has a reasonable allowance of free time for private projects if you’re on a paid Github tier.

Previously I was using Travis-CI to do something similar, but couldn’t justify the fees for the paid tier, so I decided to standardise on GH Actions instead, which meant moving over a whole bunch of my Swift packages.

One thing started annoying me a bit though. For Travis I was using CCMenu to monitor the test status. It doesn’t work for GH Actions. I figured out a way to get the actions to post notifications to a channel on Slack, but still…

Wascally Wabbits

I decided that it couldn’t be that hard to create a tool to address these problems, and off down another rabbit-hole I went.

Initially it started as a simple app, which just shows a list of repos in a window, with a tick or a cross to indicate success or failure.

This seemed like a good excuse to use SwiftUI, and I also figured that it might be handy to have it work on a mobile device, or even a tv, so I used Catalyst to target all three platforms at once.

Cunningly, I realised that Github Actions already spits out a status SVG for each action. So if I limited myself to public repos (to avoid authentication), I didn’t even need to use proper Github API - I could just parse that to figure out the status.

Thus Action Status was born.

CC, Oui Oui, Ja Ja

I fairly quickly had the basic app up & running, but to really work like CCMenu does, the Mac version needed to have a funky status item in the menubar, warning me if something was failing.

Adding these items is pretty simple, but requires access to AppKit API that Catalyst can’t use.

To work round this I needed to get a bridging plugin working.

Didn’t take long, and looked really neat!

This was a bit of a detour admittedly, but loading plugins to bridge to AppKit was something I was going to need to do for Bookish anyway - because Catalyst’s scanner support doesn’t work on macOS :(

Now I know that I can do it, so it was definitely worth the bother.

In fact, figuring out the plugin thing proved doubly useful, as I soon realised that I needed to use the same approach if I wanted to integrate Sparkle (see my previous post…). This was another thing I needed for Bookish (which is also Catalyst-based), so all good…

Status: Repeating. Status: Repeating. Status: Repea…

Once I had all this working, I figured that I should get round to moving over all of my open source Swift Packages.

It turns out that I have rather a lot. Adding them all to Action Status one by one was tedious.

Worse still, some of them work on all platforms, some work on just the Apple platforms, some just work on the Mac. Some need to run tests, some just to check that a build works.

That’s a lot of YAML files. All follow the same basic pattern, all need some steps (like the Slack notification), but all need slightly different permutations of other standard steps in them.

Surely I could get Action Status to help with this?

Of course I could.

So I added a couple of abilities.

The first is the ability to scan a local directory (on the Mac), to look for repositories to add. Anything that looks like a git repository is checked to see if it is based in Github. If it is, and doesn’t already have an entry in Action Status’ list of repos, one is added.

The second is the ability to generate a Tests.yml file for a repository, from within Action Status. Selecting this option displays a list of options to choose from. You can select platforms, configurations, and which actions to perform.

This turns out to be really handy. Getting the exact syntax of the steps right is a bit fiddly, and having something to do it for you really helps. Action Status remembers the choices you made, too, so if you want to tweak them and regenerate the workflow file later, you can.

The Future

This was all a bit of a distraction, but it’s proven to be useful enough a tool that I’m planning to make it available in the App Store(s).

It’s still quite simple at the moment, but if you’d like to test it, that would be great. You can download the macOS version directly, or sign up for the iOS/tvOS beta.

I plan to expand it to work with private repos, and also have a bunch of other features that I’d like to add.

Once you have an app which has a list of your code repositories in the cloud, and also knows their locations on your local disk, all sorts of interesting opportunities present themselves.

Maybe Action Status can perform other regular tasks locally (such as fetching, committing backups, etc)? Maybe it can also generate other standard entries in your repos (.gitignore files, README headers, licenses, standard folder layouts). It could certainly support a much richer range of templates, for use with other languages or situations.

Also, whilst it’s focussed on Github Actions at the moment, there’s no reason in theory why Action Status couldn’t also support Travis, or Gitlab, or something else.

Please let me know if you thing it’s worth me bothering with any of this, or if I’ve lost the plot!

Meanwhile, I really will get back to Bookish soon. Honest.

Catalyst+SwiftUI+Sparkle=?

2020-02-28T17:09:00+00:00

Bookish Development Diary, episode 7.

Ok, I admit it, I fell down a bit of a rabbit hole a couple of weeks ago.

Quite a lot of my open source packages have some sort of unit tests, and I’d been using Travis-CI as a way of running them, with CCMenu to monitor them locally.

Recently though I’ve been experimenting with Github Actions instead of Travis. They work really well, and I think I’m going to switch over to using them for everything.

The one thing I miss though is CCMenu. So I decided to see how quickly I could hack together a replacement. It was also an excuse to play around with SwiftUI…

Fast forward a few days and I have something working. It’s pretty simple, but nice I think, and works on iOS/tvOS and macOS, thanks to Catalyst. It’s called Action Status, and I think I’ll actually put it into the app store.

A few developers might find the app useful, but it is also a good way for me to refresh my memory of the whole app store submission process, before doing it again with Bookish.

This post isn’t about Action Status though, it’s about something that I realised I needed to do for it, which turned out to be a bit tricky: adding Sparkle support.

I wanted to do this so that I could distribute the Mac version to friends for testing.

I initially thought that this would be easy, but then the penny dropped that since I’m using Catalyst, I can’t link directly to Sparkle, because it uses AppKit.

I found a way round this problem, but it’s a little invovled. Once I’d got it working (roughly) in Action Status, I thought I should go back and isolate just the Sparkle stuff, as it might make a good example for other people wanting to do the same thing.

The result of that process can be found here.

If you want to know the full details, check out the repo’s read me.

Here’s the executive summary though:

You need to make a plugin, and have the plugin link with Sparkle.

Unfortunately Sparkle uses WebView in its UI, which crashes if used from an AppKit plugin in a Catalyst app. So you need to replace the Sparkle UI with your own.

One way to do this (which I chose to do), is to bridge Sparkle’s UI handling over the plugin and back into the host application. This means that the host has to make a custom UI for showing read me notes, updates, etc, but on the plus side, this means you can it funky. You can also use SwiftUI for it, if you are so inclined.

This was all an interesting, if unexpected, detour. A detour on a detour, in fact. However, I’m happy with the results, and I would have needed to do most of it anyway for Bookish since I’m using Catalyst there too.

Fighting With UISplitViewController

2020-01-31T13:27:00+00:00

Bookish Development Diary, episode 6.

Call me an idiot (“Sam, you’re an idiot” - ed.), but whenever I try to use UISplitViewController, I seem to get myself into a tangle. It doesn’t work the way I expect it to.

What I generally want is an index view side-by-side with a stack of detail views.

If I’m on a phone, or a horizontally-compact environment, I want the index to collapse onto the stack, so that there’s just one view. Tapping an index item here should push the detail view onto the stack, replacing the index.

Assuming that I’m not restoring the previous state of the app, this is how things should start in a compact environment. I definitely don’t want the app to start showing an empty detail view, and hiding the index 🤦🏼‍.

If the view is collapsed and some detail is showing, I don’t want to be able to pop up the index view, or slide it over the detail.

I just want to be able to pop the navigation stack to get back to the index.

This is not rocket science!

I’m sure that you can get UISplitViewController to behave like this, but invariably I seem to end up having to jump through a lot of hoops to do it, and even then sometimes do it wrong.

I hit this situation yet again whilst working on an example viewer app for Datastore, so I started wondering how easy it would be to just throw it away and make something simpler…

IndexDetailViewController

If you keep the spec simple, it turns out not to be too difficult, and you can find my attempt here.

I called it IndexDetailViewController as my use case is specifically for indexes. I guess it could be called MasterDetailViewController instead.

The way I approached this problem is to use a UIStackView controller to manage the side-by-side behaviour.

This already conveniently supports animation if you hide one of the arranged views, which is handy.

Rather than expecting you to supply a navigation controller for the detail (as UISplitViewController does), my IndexDetailViewController makes one for you and manages it internally.

When the controller is collapsed, it removes the index from its UIStackView, and adds it to the relevant place in the detail navigation stack.

That’s pretty much all there is to it. Not bad for a few hours of hacking.

If you want to see it in action, the repo includes a simple example.

No doubt there’s lots of subtle extra behaviour that UISplitViewController gives you, and which I’m missing.

It doesn’t add its own chrome, and I haven’t added the bits & bobs necessary to allow you to configure it completely in Interface Builder.

I could add all of this relatively easily, but I like the simplicity of what I have, and for now, IndexDetailViewController serves my purposes.

Maybe I’ll slowly add the missing stuff.

Maybe one day I’ll end up re-implementing it using UISplitViewController under the hood! Time will tell… but either way, IndexDetailViewController is a good abstraction for the behaviour I actually want.

XPkg - A Package Manager For Machine Configuration

2020-01-23T15:39:00+00:00

Erica Sadun blogged recently about the trials and tribulations of upgrading a system with a long history of tooling on it.

(the link to that blog post seems to have vanished; I’m not sure if it’s been taken down or just moved, but it doesn’t really matter, as the post was just a prompt to me to write what follows)

This is a problem I could relate to, having been working with Macs since 1988, and over many years developed a large number of scripts, utilities and system hacks to make my life easier and more productive.

Fortunately (sort of), it’s a problem I’ve had to try to solve (or at least make easier), since I’ve regularly had to move between systems, and work on multiple systems at once.

At one point, whilst working on Football Manager, I was writing the low level cross platform libraries which had to build on the Mac, Windows, Linux, XBox 360, PSP and PS3! I had about three monitors on my desk, four or five machines under it, and was regularly booting into development environments for all of them. Fun times.

What I used to do for many years was have a single monolithic directory under source control, which contained all of my helpful stuff. This had a bootstrap script in it so in theory I could just fetch the repo to a new machine, and run the bootstrap script.

It was never quite that smooth, but worked, up to a point.

However, it got a little crufty. Over time, as I moved on, most of what was in it became irrelevant, but was still hanging around. Some bits of what was in it actually logically lived elsewhere, which once I’d moved the thing into git meant some dalliances with submodules.

Eventually I realised that what I needed to do was to modularise this big lump of stuff, so that I could still share common things between systems, and still set up a new system easily, but I could also just install the bits I needed on any given system.

This sounded awfully like a package manager! Unfortunately, the ones I was familiar with tended to be tied to the Mac.

“How hard could it be?”, I thought - foolishly - to make a really simple one myself. All it needs is the ability to download packages, and to run a little script inside a package to install/uninstall it.

And thus, XPkg was born.

It is very much a work in progress, and I wouldn’t necessarily say that it’s ready for other people, but Erica’s blog post reminded me that I had been meaning to tell other people that it existed.

Rather than repeat myself, I’ll let the Read Me file tell you more.

You may also be interested to know that not only is it written in Swift, it actually uses the Swift Package Manager as the transport mechanism for fetching the packages that it manages, and resolving dependencies between them.

Datastore Views, Swift Packages, UI In Code

2020-01-16T13:23:00+00:00

Bookish Development Diary, episode 5.

It’s been quiet around here over Christmas, and I cut myself a bit of slack and didn’t really do a lot last week, but I’m back to it now.

Pretty much the last decision I made before the holidays was that I needed to develop some basic view classes for use with Datastore, and that they should live in a companion module (called DatastoreKit), as another published product of the Datastore package along with the Datastore module itself.

I also decided that this should go hand-in-hand with a test application, which is a separate project and a client of both Datastore and DatastoreKit.

This serves the dual purpose of allowing me to work through the basic user interface problems in a relatively uncomplicated environment, and also giving me a tool that can open any arbitrary store file and display its contents.

Having such a tool, which gives a raw view of the data, with minimal interpretation, is really useful for diagnosing problems in higher level application logic. In extreme cases it potentially also allows manual rescue and/or rebuild of user’s data; something that hopefully won’t be needed often, but a useful tool to have in the toolbox.

I suspect that for Bookish I’ll still need to customise the view classes that DatastoreKit provides, but hopefully they’ll provide a solid foundation.

YAGNI

This is one of those decisions that could be described as over-complication. I can hear the YAGNI brigade screaming that I should just make some views in Bookish, and be done with it. Perhaps I’m just destined to repeat the same mistakes forever, but I don’t buy that argument (in this case, at least).

I think it would be valid if Bookish was the only client, but past experience has shown me the value of a database viewer tool, and having two clients gives me a reason to move the views into a separate module.

It also gives me a fighting chance of keeping the view classes genuinely decoupled from either client. Or not… we’ll see.

Swift, Modules, Xcode

The choice of where to put a UI library, and indeed how to implement it, has become a bit more complicated in world where Xcode and Swift Package Manager have collided.

I was never a Cocoapods or Carthage fan, but I really like the SPM model, and I adopted it as soon as I could. It was a pain to integrate with Xcode in the beginning, but since XCode 11 it’s been pretty easy for code-only packages.

The problem is that SPM is platform neutral, and right now doesn’t support any of the non-code tasks that are required to build a fully fledged macOS/iOS application or a framework, such as compiling xibs, packaging resources and making bundles.

Ultimately this is a problem that needs solving, but the SPM team have been cautious about how they go about it, and frankly they probably have other priorities. A while back I messed around a bit with extending SPM to solve this problem myself, and that’s something I may go back to at some point.

Given the progress that has already been made with Xcode integration, I’m hopeful that the Xcode and/or SPM teams may get there before me.

In the meantime therefore, my general approach is to make anything UI-related as a normal Xcode app or framework, and just use SPM for code-only things.

This presents a bit of a problem for DatastoreKit, since I really wanted it to be part of the Datastore SPM module.

UI in Code

Historically, I’ve been very sceptical of anyone choosing to eschew Interface Builder (or ResEdit/Resourcerer/PowerPlant/MacApp equivalents!) and make their UI in code.

The motivation people had for doing this often seemed quiet dubious to me (more about a resistance to learning new things than a principalled stand).

From Hypercard onwards, I was an early convert to the value of being able to build a UI with direct manipulation, and being able to (in effect) data-drive the UI construction by describing it in something other than code (resource files, xib files, etc).

I’d like to think that I was also honest about the down sides - particularly the difficult of tracing all the connections in a complex UI at runtime - but on balance I always felt that the advantages outweighed the disadvantages.

It’s really easy for a reasoned argument like this to slide into dogma though, and I think that’s what I may have done. A lot of water has passed under the bridge since I first thought about it.

The sort of things that Xcode can do with playgrounds and live-previewing SwiftUI - in particular, the ability to round-trip between a textual and a visual representation, and edit either - are game changers.

They were always theoretically possible, but rarely (if ever) very well done. Having an implementation that works well offers a glimpse of a best-of-both-worlds solution, which moves the goalposts a bit.

Having worked in teams on large projects, I can also attest to the value of being able to diff changes, and the difficulties of doing so with an IB-based interface.

Nothing Ventured…

So although I’m not actually using SwiftUI in this case, I’ve decided to be pragmatic and see how far I can get with building the UI components of DatastoreKit in code.

Doing so allows me to publish the module with SPM, and it should be an interesting experiment.

We’re not talking about a vast number of components here either, so it’s a relatively contained problem.

The test view app, on the other hand, remains a fully-fledged Xcode project, with an interface built in IB.

It (and Bookish) are both clients of the Datastore SPM package, and are free to import both the code and the UI components from it.

Further Evolution

2019-12-17T13:24:00+00:00

Bookish Development Diary, episode 4.

A tale of how the last version of the Datastore was just right, but it turns out that the latest version is even righter.

As I mentioned in a previous post, I really enjoy the way an API evolves as one slowly figures out what it is supposed to do.

This process is continuing with Datastore, the database backend that I’m using for Bookish.

It’s perpetually humbling to realise just how unfinished that thing that you thought was finished actually is. For instance: it turns out you actually need API to delete records - who knew? ¹

Something that I particularly enjoy is the moment where two concepts/classes/algorithms that you thought were separate reveal themselves to be two aspects of the same thing.

This seems to be happening with Datastore…

The Story So Far…

To recap, Datastore is designed to be asynchronous, so rather than fetching entities and then manipulating them directly, the way you work with Datastore is more like:

write a description of the entities you want
write a description of the properties you want to fetch/change
submit a request, and receive a response callback
(optionally) interrogate the results you were given in the callback

In early versions of the API, this lead to two fundamental concepts:

an EntityReference which lets you describe the entities to work with
a PropertyDictionary which describes the properties you asked for, or wanted to update

You always pass in some references, to describe what you want. If you’re performing an update, you also pass in a dictionary with the changes. If you’re looking up entities, you get back some references. If you’re looking up properties, you get back a dictionary.

The references started off as quite simple things that just specified the entity identifier in a neutral way.

Then they evolved a bit so that they’re more like database queries that can be used to locate the entities to work with. Rather than just matching identifiers, they can also match properties.

Then they evolved further and became a way that you could not only describe an existing object, but also how to make it if it didn’t already exist. Passing in a bunch of these references has a pleasingly declarative feel about it. It’s as if you’re a subset of the database as you need it to be, and leaving the storage system to work out whether it needs to fetch existing entities or make new ones.

Meanwhile, another interesting aspect of references is that they can be resolved. As I mentioned above, to a client of the Datastore, they can be thought of as an immutable description, which is a way of locating an entity. Internally though, there has to be a process of resolving this description, to turn it into an actual entity record². This means that there’s some potential value in passing back the resolved reference as well as whatever other results you’re supposed to return. A resolved reference is guaranteed to describe a record in the underlying database, and can hold on to some internal state allowing them to skip the lookup work.

As a result of this, the concept of a GuaranteedReference was born, and the API was modified so that whilst it took instances of EntityReference, it would also return instance of GuaranteedReference. These guaranteed references were interchangeable with normal ones, but the need to return them, potentially along with a property dictionary as well, made some of the result code a little messy³.

Evolving References

I digress slightly.

Getting back to the evolution of references into things that could describe how to make something: doing this required some types of references gaining a property dictionary, used to populate the entity if it had to be created.

Which was interesting, because it offered a tantalising glimpse of a possible world where, rather than passing in references and getting back properties associated with them, perhaps you passed in references-with-associated-properties, and also got back references-with-associated-properties.

Instead of having to sometimes pass in a list of EntityReference instances, and other times a list of EntityReference:PropertyDictionary pairs, and sometimes getting back a list of GuaranteedReference instances, and other times a list of GuaranteedReference:PropertyDictionary pairs, maybe the API could be arranged so that you always pass in a list of references, and get back a list of references.

If the references own property dictionaries, then on the way in, these can be used as a description of how to set up the entity, or how to modify the entity. On the way out, the dictionaries can contain any properties that were looked up, and the returned references themselves can be upgraded to be guaranteed.

This leads to an API where every operation feels like a transformation. You pass in a list of existing references, and you get back a list of transformed references. Which feels nice.

It also feels very much like what I will need for the user interface, where you’ll need to have some sort of placeholder objects, standing in for the entities, which the user interface code can extract properties from, and perhaps perform changes to.

The Future

This is where I’m heading with the next iteration of the Datastore API (which I should probably call 2.0, but may well just call 1.3 since things are moving so fast at the moment that I’m not be completely strict about semantic versioning).

There are still a few wrinkles to work out⁴⁵⁶.

I’m not yet sure what all the answers are, but I’m looking forward to finding out.

I might have fogotten to add one in earlier versions of the API. :facepalm:. ↩
The current implementation uses CoreData under the hood, so needs to work with NSManagedObject and NSManagedObjectContext. We don’t want to leak these implementation details though, which is where references come in. The resolution process takes a reference and a CoreData context, and gets back the NSManagedObject corresponding to that entity. ↩
For example, you’d ask for a list of properties, for a list of entities, and get back a dictionary where the keys were the corresponding guaranteed entities and the values were property dictionaries. The references you passed in and the ones you got back described the same entities, but were different objects, which complicated the process of extracting results from the dictionary. ↩
One thing I’m pondering is whether the references that describe a new entity need to make a distinction between properties to use to create it if it’s missing, and properties for the actual operation you requested. For example, if you want to look up a record by id, but also set its name if you create it. If a record with the id exists, the name might have a different value, so you don’t want to set it in that situation. Does that mean that references need two dictionaries - an initial values one and a values-to-use-for-the-operation one? Or are the only operations that take a dictionary always going to overwrite anything that was there before anyway? ↩
Another thing I’m pondering is whether these references really can now act as fully fledged stand-ins for the real entities in the user interface. In which case they probably need a new name. Can you fetch a few, display them in the UI, use them to collect property modifications from the UI, and then issue an update back to the store to commit them? Does this work cleanly in all situations? From the UI’s point of view, these references are stand-ins, but are not perfect copies of the real things, and may not contain all the properties that the real thing contains; they may also be out of date if a synchronisation from another device has changed something in the meantime. ↩
Should the references’ dictionaries be mutable, or is it better that they are immutable and always replaced by a new object when the callback from the previous operation is called to indicate that it has completed? Is there a danger that using an older copy of a reference in a new operation might accidentally overwrite new property values with stale ones? ↩

Constant Keys (Sometimes It's The Little Things)

2019-12-09T13:39:00+00:00

Bookish Development Diary, episode 3.

In which our intrepid developer disappears down a deep rabbit hole, in search of the cleanest and most idiomatic way to express string constants in Swift¹. Because, you know, stuff…

Sometimes it’s the little things that make me happiest.

In an open-ended system like Datastore, where any property can be assigned to any entity, I needed to be able to specify a property using some sort of key.

It turns out that there are a few ways of doing this…

A key for a Datastore property could theoretically be anything hashable. As is often the case with the standard Dictionary type though, the most natural fit for these keys is simply String.

Strings are easy to express (and read) in code, and map well to textual representations such as XML or JSON (which is important for Datastore’s interchange format).

You can just type these in your code where you need them.

store.add(properties: [person: ["name": "Fred"]]) { results in
    // some time later, use it...
    store.get(properties: ["name"], of: [person]) { properties in
        // do something with the properties
        if let name = properties.first?["name"] as? String {
            print(name)
        }
    }
}

Anyone with more than a passing exposure to good coding practice might be getting twitchy at the prospect of repeating those string literals. It violates the DRY principle, and opens us up to nasty like bugs like this…

store.add(properties: [record: ["complicatedKey": someValue]]) { results in
    // ...
    store.get(properties: ["complciatedKey"], of: [record]) { properties in
        // ...
        }
    }
}

Better to use a constant. In a local scope, you might just do this sort of thing

let nameKey = "name"

// make a record
store.add(properties: [person: [nameKey: "Fred"]]) { results in
    // some time later, use it...
    store.get(properties: [nameKey], of: [person]) { properties in
        // do something with the properties
        if let name = properties.first?[nameKey] as? String {
            print(name)
        }
    }
}

However, although Datastore is generic, your use case is specific. You will probably have a set of commonly used keys. So typically the solution is to define a load of string constants at a higher scope. You might do something like this:

struct Model {
    struct Keys {
        static let name = "name"
        static let address = "address"
        // ...etc
    }
}

then use it like this:

// make a record
store.add(properties: [person: [Model.Keys.name: "Fred"]]) { results in
    // some time later, use it...
    store.get(properties: [Model.Keys.name], of: [person]) { properties in
        // do something with the properties
        if let name = properties.first?[Model.Keys.name] as? String {
            print(name)
        }
    }
}

This is better, but there are some niggles. It feels a little bit repetitious having to say Model.Key every time.

What’s worse, the Swift compiler has no clue that Model.Key has anything to do with your API, so Xcode can’t auto-suggest it, nor can it auto-suggest the constants themselves until you’ve typed the Model.Key bit.

You could just do let name = "name" at the global scope, but then you’re polluting the global namespace which has its own set of issues (such as Xcode suggesting name for any API that takes a string, anywhere).

What would be nice is to be able to do this:

// make a record
store.add(properties: [person: [.name: "Fred"]]) { results in
    // some time later, use it...
    store.get(properties: [.name], of: [person]) { properties in
        // do something with the properties
        if let name = properties.first?[.name] as? String {
            print(name)
        }
    }

and be able to type ‘.’ and have Xcode suggest suitable constants, but also be able to just type a string literal instead.

So how to achieve that?

For Xcode to suggest one of our key constants, it needs to know that we are typing a key. For that to happen, the key needs to be a custom type, and the API has to be written to use that type instead of a string.

Those .name entries in the code sample above look awfully like cases in an enum. Maybe if we made the key a string-backed enum:

enum Key: String {
    case name = "name"
    case address = "address"
        // ... other keys...
}

That sounds like what we want, right? We can define the keys we know about. They are still actually strings, so converting to/from interchange should work ok.

Except that we’ve forgotten something. One of our requirements is that Datastore supports any key, not just the ones we happen to need for a particular scenario.

We can turn raw strings into our enum type with Key(rawValue:) initialiser, but it returns an optional, and that optional will be nil if we try to give it a key that we didn’t make a case for. Yet clearly we can’t make an infinite number of cases.

Back to square one.

So if we can’t use an enum, maybe we just need a custom type, which contains the string?

struct Key: Hashable {
    let value: String
}

This is a custom type, so the compiler can make sensible suggestions when it is expecting one.

It isn’t an enum with a finite set of cases, so we can define them as constants, but also make them from dynamic strings when we need to.

But what about that .name syntax? How do we get the compiler to support that, without it being an enum.

It turns out that what the compiler is doing in those enum cases isn’t actually because they are enums. It is an example of a feature of Swift called Implicit Member Expressions.

What this boils down to is that when the swift compiler knows that the type of the value it is expecting is Foo, and you supply it with .bar, it can infer that what you actually meant was Foo.bar.

In the case of our enum example above, this means that .name is treated as Key.name, which is the name case of the Key enum, and so works.

So for our Key struct, we just need to make sure that Key.name resolves to a Key where the value property is “name”.

Which is easily achieved by defining a static on Key. What’s really nice is that we can do this in an extension:

extension Key {
    static let name = Key(name: "name")
}

This is pretty close to where we wanted to be.

The .name constant now works wherever the compiler is expecting one of our keys.

Having the definition in an extension feels right too - it can live in a logical place which collects together all the keys for our use case.

There’s just one other thing that would be nice.

Sometimes in our code we will have places where we only need to use a key once. Maybe in a test, or maybe in a place where the value is set externally, and all our code ever does is read it.

Having to define a constant, or create a Key(value: "one-shot key") object, feels a little clumsy here. It would be nicer if we could just use a string literal, and have the compiler work it out.

Knowing what we now know about implict member expressions, we might be temnpted to do away with our custom class and go back to just using strings. We can get that .name behaviour that we wanted by defining our static on String itself.

extension String {
    static let name = "name"
}

Now we can use .name for a key, but also just supply a literal.

This works, but what we’ve just done is thrown away useful type information that we were giving the compiler by defining the Key type, and also spammed the String type with a constant that will be suggested anywhere that Xcode expects a String.

So let’s not do that.

What we need instead is to use another Swift feature: Initialisation With Literals. What this gives us is the ability to have the compiler automatically create a custom type from a string literal.

All our custom type needs to do is to adopt the ExpressibleByStringLiteral protocol, and implement init(stringLiteral: String).

Now, when we know that a key is going to be repeated, we can define a constant like name, and supply it to the API as .name, but when we have a value we’re only using once, or when we’re writing test code or just trying an experiment, we can just supply the string "one-shot key" instead, and everything will still work.

One little side-benefit of this is that we can rewrite our constant definitions to be a little less wordy:

extension Key {
    static let name: Key = "name"
    static let address: Key = "address"
}

So there we are. The route was perhaps a little circuitous, but what we’ve ended up with feels nice, and clean, and hopefully reads like good idiomatic Swift.

If you’re a sophisticated user of Swift, none of what I’m about to say will probably come as a surprise. You may be bored or underwhelmed by this tale. Sorry! In fact, maybe you know a better solution? If so, please let me know. The solution I describe is was one of those things that I had to puzzle out over time when I started writing Swift - mostly by seeing other code that looked cleaner than mine, and figuring out what the difference was. I’ve been meaning to write it down explicitly for a while. ↩

No API Survives Contact With The E̶n̶e̶m̶y̶ Client

2019-12-05T10:35:00+00:00

Bookish Development Diary, episode 2.

One of the devevelopment tasks I enjoy most is designing then implementing a new module. The smaller the module, the more self-contained the job it is tasked with, the cleaner and simpler the resulting API, the better.

Perhaps that’s the reason that I allowed myself to be ~~distracted by~~ convinced of the need to develop Datastore, my generic data storage layer for Bookish…

A large part of the fun is how totally wrong I get it to begin with. You create a new project, write a few tests and a bit of code, and think you’re done. Simples!

Then you go back to the client project to try to adopt the new API, and it all falls apart. You forgot X. And Y. Actually, you wrote it in the wrong language, missed out ɑ through to Ω, and may be holding the map upside down. So you return to the API, and fix your obvious mistakes, then try again. And come up short again. And so it goes… rinse and repeat.

I love this process. Not for the frequent reminders of what an idiot I am (healthy though that is), but for the sense of progress that you get as the real shape of the API slowly emerges out of the mist.

I won’t write in detail here about all of the inspirations behind the choices I made for the Datastore design (that’s for another post), but I’m a great believer in Occam’s Razor.

So I started with the simplest thing that I thought would meet the needs of Bookish: look up records with an identifier, fetch them in batches and get the results as dictionaries, and send batches of (identifier:dictionary) pairs back to the store to write or update records.

Bookish supports importing from other products (such as the Kindle app, or Tasty Archive¹), and they don’t always give you identifiers - sometimes you have to make do with names. So I realised that I need to be able to fetch a record by a name as well as an id. Back to the API drawing board.

Then I realised that entities have any number of properties, and name is just one of them. Being able to fetch by name was a special-case, and I don’t like special cases. So I reworked the API to allow looking up a record by any arbitrary property.

There are competing tensions at work here, influencing the API. The first change was driven by need. The second by a desire for flexibility. Behind it, always, is that Occam’s Razor desire to keep the API as small as possible: “the simplest thing that works”.

So once I’d made those changes, I realised that there are times when I need to look up an object if it exists, or create it if it doesn’t. This was particularly relevant in cases where I was adding a property to one object which added a relationship to another one.

Doing this with the asynchronous API felt clunky and resulted in a chain of calls that didn’t feel natural. What I needed was a way to specify an object, along with a minimal set of values to use, if necessary, to create it. Back to the API drawing board, and entity references were born; lightweight objects which combine both of these abilities.

Adapting the Bookish import to use these references, something else fell out of the design. I wanted the importer to be able to cope with being run multiple times with the same source. This can’t be perfect in all cases, but it is nice if it does something vaguely sensible rather than just duplicating all the records again (especially for imports from things like your Kindle library, which is going to keep on changing outside of Bookish).

In order to do this, I needed some of the identifiers that the importer creates to resolve to the same thing for each import. If it’s run a second time, it’ll pick up the same records rather than making new ones. However, if a record with that identifier doesn’t exist, but a record with the right name already does (eg the user already had an existing record for a given named author), I probably want the importer to pick this in preference.

So now my entity references needed to be able to specify a list of criteria to match against - identifier, then name - only falling back to making a new object if one of those failed. Back to the drawing board…

Finally, I got to the point where the existing importer code could be rewritten cleanly to use Datastore. Hurrah!

On to the next bit of conversion, at which point I realised that I had some debug code that was counting objects. It’d be really inefficient to have to do that by fetching all of actual entities. How could I have missed that use case? Back to the drawing board…

Then I realised that my existing Bookish implementation has the ability to start you off with a sample database (useful for real users), or to reset the database back to the sample at any point (useful for testers). Which relies on some core data API to safely duplicate/delete the backing files. Which needs hiding behind the Datastore API, since it’s not supposed to expose the technology it’s using for the store. Back to the drawing board…

And so it goes on.

If I was just adding new API each time, this could feel like code-rot was creeping in. Generally though, I’m not adding, I’m refactoring. When I do have to add new API, I can usually do it in such a way that it generalises some old functionality at the same time.

The overall size of the API grows as I add new things, then shrinks as I spot generalisations, or realise that something I’ve added has made something else obsolete.

Each time this happens, it feels like I’ve got a little bit closer to the true expression of the problem I’m trying to solve. Which feels satisfying.

I know I’m not there yet. I think I’ve caught a hazy glimpse of something even simpler - maybe involving actual objects which are references on the way in, but can also contain properties. These would be the current values on the way out, or new values on the way in. Or maybe that’s over-complicating things. Not quite sure, but I’m looking forward to finding out…

This is the app that I currently use to catalogue my books. Its limitations, and the trajectory it has followed over the years since it was first released, are a large part of my reason for making Bookish. I may have its name slightly wrong. ↩

MVP, YAGNI, DRY, acronym soup, pivot points, and justifying change

2019-12-02T13:39:00+00:00

Bookish Development Diary, episode 1.

So currently, Bookish is sort of on the floor in pieces.

I was getting fairly close to something I thought that I could start sharing with a few select friends, then I decided that a feature that I thought I could leave until version 2.0 really needed to be in version 1.0.

Or more precisely, the underlying technology for the feature needed to be in version 1.0, even if the feature itself didn’t get exposed until version 2.0.

The feature in question was the ability for the user to add custom fields to the database. Sounds innocent doesn’t it…

People use different classification systems for books, and have their own weird taxonomies, and maybe want to associate unexpected extra bits of data with the books. They want to track when they last read something, who they lent it to, where they were when they read it. What colour the cover is. What sort of binding it has. Having a single fixed schema for the database doesn’t really accomodate this usage pattern. Something flexible (more like the UI for Apple’s Contacts app, where you just add things when you need them) probably makes more sense.

The classic MVP¹ philosophy would say that this is the sort of feature that isn’t essential, and could be pushed back a version or two. Make the schema fixed for now, with a decent set of defaults, maybe add a single “custom” field that the user can use for their own purposes, and refine this into something more flexible later.

After a fair bit of agonising, I talked myself into this argument, and went for a fixed schema. I have a tendency to go too far with flexibility, so I thought that it would be good to resist that urge. Discipline, Deane, discipline!

So I started building out the UI with this fixed schema. Fairly soon, I realised that whatever the database end of things looked like, I was going to need a dynamic UI on top of it. Pretty much every field that existed in my schema could actually be absent in practice, and it made no sense to show a bunch of empty rows with labels that the user might never need or use.

I could have hard-coded each line of a table to a particular attribute in the database, and have them hidden or shown as necessary. It didn’t take long to realise though that I needed much the same code to show the entries for people and publishers and other entity types as I did to show the entries for books - but with different fields of course. Hard-wiring the UI in Interface Builder for one bunch of records was tedious enough, but doing it for four or five bunches… meh. MVP is all very well, but what about DRY²?

Then there was the cross-platform question. At this time I was building Bookish as two separate apps: macOS and iOS. I got a basic version of both up & running, but was concentrating mostly on working on the macOS one. Hard-coding the UI in Interface Builder, in this scenario, really felt like it was a bit of a foolish thing to do when I knew I’d have to repeat the process again later with UIKit rather than AppKit.

So after a bit more work I ended up with some relatively abstract code which could take one of my database records and display in a sensible way just the stuff that was filled in. All very standard - the kind of code most of us have probably done a ton of times. It used property names, and key paths, and an enum of possible fields and custom table row types, and a little hard-coded table which mapped them all together.

Much of the view-model of this code could be shared between both platforms, minimising the amount of repetition between macOS and iOS. Still a hard-coded mapping though, note. Not fully dynamic and generic. We’re going for MVP here remember. Generic would be cool, but YAGNI³ baby. Or, at least, YAADGTNIEBNYH⁴.

As an aside:, this was probably around the time when Catalyst finally went from “Apple are clear that this is not ready for prime time”, to “Apple say this is ready for prime time, which we of course know means that it’s not ready for prime time, but it’s probably going to be ready by the time Bookish ships, so maybe I should use it”. So that was nice. On the plus side, I only had one UI to maintain in IB. On the minus side, it was on the platform I’d not been concentrating on so much. On the plus side, the cross platform effort I’d put in meant that it was pretty easy to catch-up on the iOS side. On the minus side, maybe I now had more flexibility than I needed in a singe-platform-from-the-ui-point-of-view world? Ah well, no harm done.

So this is sort of where I was at, except a few things were bugging me.

I was really sure that I would want the flexibility. YAGNI be damned. How hard would it be to add in later to what I had? Might I be causing more problems for myself in the long run?

I was also finding that the UI was falling in to that area where you just know that it makes a lot of sense to write it generically, and you are in danger of endlessly turning objects into dictionaries, and dictionaries into objects. Was I making things unnecessarily complicated by defining rigid model classes and then having to define translations to/from something looser?

Unrelated to the model per-se, another of my early design decisions was to try to adopt a proper action abstraction. Rather than mutating model objects directly in UI code, I wanted the UI to issue actions which mutated the model; so that the same actions could be issued by scripts, or macros, etc (more about this in another post, at some point). But how did this square with having fixed model classes? Wasn’t I ending up having to define a lot of very similar actions (“make new book”, “makle new person”, etc) which did essentially the same thing to different entities?

Finally, what about things coming down the line like SwiftUI. Ok, I’m maybe not going to adopt that right now (or am I?), but…

All of this was pushing me towards a solution that was more asynchronous, more generic, more transactional, and a bit less tied to manipulating actual model objects. I probably wanted to be able to request information on one or more model objects and then build the UI when the information arrived. I probably wanted to be able to modify the properties and relationships in the UI without really changing the model, and then submit everything atomically when the edit was done, to update the database. If I did all this right, the database schema would be a much looser sort of thing, and I’d have my flexible user-defined fields almost by default.

I wouldn’t be being honest here if I didn’t also admit that at the time, I was getting a bit bored.

There’s a kind of anxiety and imposter-syndrome that sets in for me when you’re building something new that you’re not sure anyone else will care about or want. Whilst I’m in the early stages of development, I can rationalise that away because I’m getting a ton of other benefits out of the process: learning new things, and catching up with technologies that had been on my to-do list for years. Once all the big decisions have been made and you’re just trying to version 1.0 out of the door, it’s harder.

Having the chance to play around with new technologies is great. When you’re working on a mature application like Sketch, it’s a lot harder to do. Once you have a lot of users, and a substantial codebase, and a ton of features on the list, your options are reduced and your horizon narrows.

It’s a problem most of us would kill to have, and I don’t for a minute imagine that Bookish would ever be a tenth of that size, but having customers and a released product isn’t without some loss of flexibility… it’s nice to be able to change whilst you still can, before any of that sets in.

So there we are. All of this stuff was bouncing around in my head, leading to a kind of pivot point.

Maybe I should chuck away my existing model and start again with something more flexible. How hard could it be?

Well… quite hard it turns out - but that’s ok.

Bookish is currently in bits on the floor. There’s a hole where the database used to be, and I’m in the process of plugging in something else instead.

It’s only when you do something like this that you realise how coupled you’ve accidentally made parts of your code to classes and concepts that perhaps they didn’t need to be coupled to. This in itself is a valuable lesson.

My decision to do this is probably crazy at this point.

If I’d been a manager, working to a deadline and spending someone else’s money, and one of my coders had come to me asking to make this change, I would have said “let’s ship what we’ve got first”.

But hey - if you can’t follow crazy ideas when you’re your own boss, and turn everything upside down because you’re getting bored - when exactly can you?

Maybe it’ll prove to be a big mistake and set me back by months, but… it’s fun. Which at the end of the day, is what I’m doing this for in the first place :)

Minimum Viable Product. ↩
Don’t Repeat Yourself. ↩
You Ain’t Gonna Need It. ↩
You Are Almost Definitely Going To Need It Eventually But Not Yet, Honest ↩

Development Diary (Uncloaking)

2019-11-29T12:48:00+00:00

Since finishing work on Sketch, I’ve been a bit quiet on this blog.

Initially I took quite a bit of time off because I needed a rest, and was battling various demons.

The largest of these was burn-out, which for me presented as a major loss of confidence and enthusiasm, brought about by long periods of professional conflict. This left me wondering whether I was any good at coding, and whether it was even what I wanted to do any more. I realised that if it was, I needed to find a healthier way of doing it. There are certain negative patterns that seem to have repeated a number of times at various stages in my working life; generally when I’m working full time on a project, have a sense of shared ownership of it, and feel that I want my future to be tied up with it. I needed to take some time to reflect on why these patterns happen, and to work out how to square a certain need for control¹ with my genuine desire to work with other people as part of a team.

All of which is a big topic, worthy of a blog post (or more) in its own right, but this is not that post. Suffice to say that those battles are still part of my life, I’m still not sure I have all the answers, but on the whole I think I’m winning.

In the meantime, this post is about how (somewhat predictably, and very reassuringly), I couldn’t stay away from coding for long, and what I’ve been up to in the couple of years it’s been since I wrote my last line of Objective-C for Sketch.

I’ve actually being doing a lot of development (almost all in Swift) :)

Quite a bit of it has been for pet projects which aren’t particularly notable, or ready for prime-time, and I’ve blogged about some of that in the past. I might pick specific things out to talk about in the future if I ever get round to it, but pretty much everything I’ve done in that category is on github if you’re interested.

Along the way I’ve also done a few bits of contracting, which I’ve really enjoyed. I find that I have no control issues at all in these situations; I find it’s very liberating to take a brief from a client, give suggestions on a design or implementation strategy if asked to, but then just go in the direction that the client chooses, doing what they ask to the best of my abilities. It’s a trade-off between control and responsibility, which seems to work well for me if the job is fairly short and self-contained.

However, I haven’t been actively seeking out the contract work. Pretty early on I decided that the main thing I wanted (needed?) to do was to concentrate on my own products, and that requires focus. I struggle with that at the best of times, but it’s especially hard if you’re splitting your time with working on something for someone else².

So… my own product. I had a ton of possible things I wanted to do, but for various reasons the thing I settled on was some book cataloguing software. It’s something I use myself, and there are products out there already (some quite well known), but I wasn’t satisfied with the state of any of them, and felt that I could do better. I also happen to have an expert in the family, which helps (even if she stole my company name to use for her twitter handle!).

This is not a product announcement, and my product is not ready for prime time yet, but to some small extent this is me coming out of stealth mode and admitting that it’s a thing. The working name is Bookish. I’d like that to be the final product name, but I think there may be a conflict with another old product, so I may have to think again.

It’s macOS and iOS, and aims to sync your collection seamlessly between your devices, and it has been through quite a few iterations already (Catalyst anyone?).

The main reason for this blog post is to follow up on something a friend said to me privately right back at the beginning of this process: “I hope you’re going to do a development diary”.

I’ve always enjoyed blogging, but I hadn’t really thought of doing that.

Partly because of the burnout and the fear that nobody would be interested anyway or that I’d say something stupid and reveal my ignorance.

Partly because it would take time away from the coding.

Partly also because it would mean having to admit what I was working on, and expose myself to all sorts of concerns about announcing it and then never finishing it, or someone else rushing something to market first, or… whatever else one worries about when one has a tendency to worry.

Anyway, at some point, I realised that my desire to re-engage with the development community had grown to the point where it overrides those concerns. So here we are.

One part of my way to convince myself to do it was the idea that I’d keep the posts conversational and unstructured. More of a stream of consciousness than a coherent story.

A key part of that in turn was to keep each individual post short. That doesn’t seem to be going too well does it? In that spirit though, I will shut up for now.

Currently Bookish is in small pieces on the floor, anyway, and I need to get back to it. I’ve taken it apart as I realised at some point that the database schema that I was using was not flexible enough for what I wanted to be able to do. Maybe I’ll talk more about why that happened in the next blog post. If I don’t get distracted…

If indeed control is even the right word. Possibly it’s more about trust and mutual respect than control. I’m still not sure. ↩
A certain amount of contracting keeps the wolf from the door. I’m lucky enough to have built up a bit of a buffer though, I have decided that now is the time to use it. I really need to give my own products most of my attention right now. ↩

Document Restoration And Xcode UI Testing

2018-08-31T13:32:00+00:00

I’ve been meaning to try out Xcode’s UI Testing support ever since they introduced the feature way back in Xcode 7 / 2015!

In the past I’ve tended to worry that UI testing was too brittle - it was too easy to change the appearance or layout of the UI and accidentally break tests, even though the functionality was still ok. It’s also a really major committment to retro-fit UI testing to an existing application of any significance, which has made it hard for me to adopt for existing projects.

However, I’m currently working on a new application with desktop and mobile aspects to it, so this seemed like the ideal time to give it a go.

So far the jury is out on how useful it’s going to be, but one thing was immediately irritating me.

By default, when the user starts up an application in macOS, it will restore any documents that they had open - even ones that haven’t yet been explicitly given a name or saved.

This is great for the user, but not so helpful for UI testing, where we really want to be starting from a known state - generally a blank slate.

It’s easy to end up crashing or halting the application whilst debugging it (and the tests themselves), often leaving it with half-finished or inconsistent documents which it will merrily attempt to restore.

Xcode To The Rescue… Or Not

For normal debugging, Xcode has a solution - a way to suppress this restoration when it runs the application, in the “Run” part of the scheme:

Annoyingly though, when the application is launched via UI testing, this setting does not get applied¹.

Doing It In Code

Luckily, there’s a way to achieve the same thing, in code, by passing the right arguments to the application when we launch it from our test.

I knew that there was a flag that would do this, but couldn’t remember what it was - and it took me a ridiculously long time to find the correct incantation:

let application = XCUIApplication()
application.launchArguments = [
    "-NSTreatUnknownArgumentsAsOpen", "NO",
    "-ApplePersistenceIgnoreState", "YES"
    ]
application.launch()

Taking It Further

This technique can be usefully extended, by passing your own custom arguments as well as system ones.

You can then pick these up in your application delegate (see here for an example), and use them to trigger extra setup code - for example to put the application into a known state or load a test document for your tests to play with. Setting up state like this can theoretically be achieved purely with simulated UI actions, but doing so can get complicated, is slow to execute, and if the main point of it is just to get the application into a known state before testing something, it arguably makes more sense to take that work out of the tests.

That said, you’ve obviously got to be careful not to take this idea too far. You want your UI tests to be actually testing the same thing that your users will be using, so if the first thing you do is tell your application to radically change its behaviour because you’re running tests, the effect could be counter-productive.

This is understandable, since Xcode actually just launches the tests, and it’s the tests themselves which launch the application by calling XCUIApplication().launch(). However, it would be nice if the implementation of XCUIApplication was smart enough to pick up the setting from Xcode. ↩

Building Swift 4.2 For Debian 9 (Stretch)

2018-07-27T13:51:00+00:00

I recently installed a new server, and decided to use Debian 9 (Stretch).

Apple distribute built versions of Swift for Ubuntu 14.04/16.04/16.10, but nothing specifically for Debian.

Since Ubuntu is downstream(-ish) from Debian, there’s a chance that the binaries might have just worked, but I decided that it would be worth the effort to try to build Swift from source on the new server, to see how hard it was.

The answer turned out to be: not very.

The first steps were to fetch the source, and install some dependencies. Apple’s full instructions can be found here, but I’ll summarise them below, along with the tweaks I needed:

Dependencies

Swift needs a few tools. These all installed cleanly on Stretch.

sudo apt-get install git cmake ninja-build clang python uuid-dev libicu-dev icu-devtools libbsd-dev libedit-dev libxml2-dev libsqlite3-dev swig libpython-dev libncurses5-dev pkg-config libblocksruntime-dev libcurl4-openssl-dev systemtap-sdt-dev tzdata rsync

Source

The basic Swift code lives in Github.

mkdir swift-source
cd swift-source
git clone https://github.com/apple/swift.git

Branch

I wanted to build the 4.2 branch, so before proceeding I switched to it:

pushd swift
git checkout swift-4.2-branch
popd

More Source

That done, I could fetch the rest of the source, using a script from the main Swift repo:

./swift/utils/update-checkout --clone

Build All The Things

The instructions I linked to above show you how to build, but ideally what I wanted was to package up everything in the same format as the downloadable distributions.

It turns out there’s a script for that too:

swift/utils/build-toolchain org.swift

(the second argument is a prefix, which I suspect may only be relevant for Mac builds; I wasn’t sure what the correct value to use was, so guessed)

It’s All Going Well… Oh, Damn

This kicked off the build, and things looked like they were going well, but eventually I hit an error.

Afer a bit of rummaging around through the logs, the problem seemed to be that Swift uses swig. By default Stretch has version 3.0.10 of swig installed, and Swift wanted 3.0.12.

As luck would have it, 3.0.12 has made it into testing for Debian, so was theoretically installable with:

sudo apt-get install swig/testing

Things turned out to be a little bit more complicated, since by default, my server setup was not configured to have access to the testing stream.

I didn’t want to upgrade everything to testing, just swig, and it took a little bit of a while to figure out what I needed to do, since I’m no expert with the way of the Debian.

After a bit of Google-fu, I came up with this answer, which pointed me in the right direction.

I didn’t follow their instructions precisely, but I did create the following files in /etc/apt/preferences.d:

security.pref:

Package: *
Pin: release l=Debian-Security
Pin-Priority: 1000

stable.pref:

Package: *
Pin: release a=stable
Pin-Priority: 900

testing.pref:

Package: *
Pin: release a=testing
Pin-Priority: 750

I then edited /etc/apt/sources.list to include the lines:

deb http://mirrors.linode.com/debian testing main
deb-src http://mirrors.linode.com/debian testing main

(I’m using Linode hosting, hence the use of their mirrors - but you could obviously use the official Debian urls there instead, or other mirrors provided by your hosting service)

The theory (according to that linked answer) is that by setting the testing priority lower than the stable one, I wouldn’t be offered the testing version in the normal course of things - but I could access one manually if I wanted.

This seemed to work, and allowed me to run:

sudo apt-get update
sudo apt-get install swig/testing

which got me to where I wanted to be, with swig -version reporting 3.0.12.

Build Take Two

This time, running

swift/utils/build-toolchain org.swift

chugged its way through without falling over, and left me with a tar archive:

swift-LOCAL-2018-07-26-a-osx.tar.gz

\o/

Installing

Expanding this archive leaves you with a usr/ folder containing a fairly normal looking set of sub-folders - bin/, lib/, and so on.

So the simplest “installation” approach is to just get the usr/bin folder into the PATH environment variable.

Exactly where to put the built files is really down to personal preference.

My current strategy is to put them into /usr/local/share/swift/<version>/, symbolically link the version I’m currently using as /usr/local/share/swift/current, and then add /usr/local/share/swift/current/usr/bin to my PATH.

Having the symbolic link in there is just an extra layer of indirection¹, allowing me to switch to switch quickly between alternate builds.

Your Mileage May Vary

So far, this seems to have achieved what I wanted. I’ve got Swift up & running, without (hopefully) accidentally upgrading to the complete testing version of Debian.

If this works for you, or doesn’t, or if you’ve got suggestions for a better approach, give me a shout on Micro.blog or Twitter.

well known in computing circles to be the solution to all problems ;) ↩

Situation Normal: All Inter-Linked

2018-07-03T17:16:00+00:00

In setting out to write up a little status report of what I’ve been up to recently, I found myself thinking of military acronyms.

Combat, not surprisingly, engenders a dark cynicism, leading to some amusing abbreviations for situation reports. A few examples:

SNAFU (Situation Normal - All Fucked Up)
TARFUN (Things Are Really Fucked Up Now)
FUBAR (Fucked Up Beyond All Recognition)¹

No doubt there are many more.

I suspect that programmers generally share the same attitude to planning - and the sense of how much any plan matches reality - that most troops on the ground in the thick of it do.

This may explain why, when setting out to describe my current situation, an acronym of my own came to mind: SNAIL.

The good news is that it’s is not (quite) so negative as the miliary ones.

It is though quite descriptive of the state I normally find myself in - and also the pace at which things tend to move as a result.

SNAIL: Situation Normal, All Inter-Linked.

Focus (Ooh Look, A Unicorn)

I’m kind of between projects at the moment, taking some time out and investigating new avenues. As a result, I’ve been tinkering with quite a few things. This could be regarded as a lack of focus (but is a quite deliberate choice on my part).

I’m interested in Swift - not because I think it’s the new shiny or is going to solve all the world’s problems - but because it’s evolving at a healthy pace and seems to have an interestingly open design process. We weren’t really in a position to use it on Sketch², so since moving on I’ve been eager to get some experience with it.

For largely political reasons, though also fuelled by some frustration with Apple’s desktop hardware, I’m also interested in Linux. Or, more accurately, in platforms-that-are-more-open-than-macOS-and-dont-totally-suck-by-not-being-macOS.

Meanwhile, in a very background sort of way, I’ve started tinkering with a game. The code name is “Problem”. Possibly as in, “Houston, we have a…”.

This could be regarded as: Project #1: Problem

I used some Apple-only technology to quickly bootstrap things - which was the right decision, but probably in the long term I want to replace them with something I have more control over. Especially given the Linux interest.

Linux

As I said above, I’m interested in Swift, and interested in Linux.

Working with Swift on multiple platforms brings a few challenges. The absence of Xcode on Linux being an obvious one.

For everything but Swift/Objective-C coding, my editor of choice has long been Atom, so I started wondering how feasible it would be to use it instead of Xcode for swift.

Xcode is more than just a text editor of course - it’s an IDE with a deep understanding of Swift, which allows it to do syntax highlighting, show warnings and errors in context, let the user set breakpoints and symbollically debug, and so on. So I got interested in working out how to integrate those features into Atom.

Leading to Project #2: atom-ide-swift.

This is very much a work in progress, and at a very basic stage right now.

It’s useable to edit and build code now, however. I know: I’m using it.

It could do with a lot more work however, and better integration with Atom’s new IDE features. There are some third party projects which would really help to flesh it out, and as if I didn’t have enough distractions of my own, I’ve been sorely tempted to get involved with them.

Build Actual Things

As well as it’s editing and interactive debugging features, Xcode is also a sophisticated build system, which lots of support for bundling non-code products and resources, running additional tools, and so on.

Because of the two fairly orthogonal interests of Swift and Linux, I’m also interested in the Swift Package Manager, which is a way to specify and build Swift modules, and has the advantage of being platform-neutral³.

Working with Swift on Linux has lead me to the realisation that SPM does a fraction of what Xcode does, and in particular it’s missing pretty much everything that you’d need to actually build a large product with bundled resources or a sophisticated build chain.

Trying to explore ways of fixing this has lead to another interesting diversion.

Hence Project #3: Builder.

I wanted to work on this on Linux as well as macOS, so, unsurprisingly, I needed the Atom Swift stuff. Thus: linkage!

I’ll probably need this, or something like it, if I ever want to really make my game, use Swift to make it, and have it build on Linux. More linkage!

Too Many Projects Already

Meanwhile, shifting around between machines and projects also reminded me of how painful it was to try to keep track of which projects were where, and make sure that the version on each machine was up to date.

In days of yore, my old habit - which predated the existence of things like git / github - was to keep absolutely everything important on every machine⁴. I’d arrange things in a nice reverse-domain folder hierarchy, so that as long as you knew the location of the root, everything else had a logical place.

Going through various machines trying to remove a lot of projects that were now obsolete - and should theoretically be backed up in git anyway - turned out to be a pain when I realised that I needed to do a lot of manual checking that things were in fact correctly pushed and didn’t have half-finished local changes outstanding.

I concluded that, at a time when anything I do pretty much must be in git as a matter of course, it makes no sense any more to keep a load of old or parked projects on my machines. And if I was going to have fewer projects on my machine, having a complicated folder structure made no sense either.

What would be a lot easier would be some sort of management tool - kind of like a cross-platform package manager for projects - which let me “install” the projects I needed into a known location (maybe ~/Projects/), when I needed them.

With a tool like this, I could add just the projects I was currently working on, keep them up to date across machines.

When it came to removing projects from my machine, a tool could automatically perform checks first to ensure that there were no local-only changes and that it was safe to just throw away the local copy.

I could also easily bootstrap a machine by just installing the tool, then giving it a list of projects to install.

Having the tool know which projects were “installed” would have other advantages. It opens up the possibility of automatically fetching updates for all projects, automatically pushing any changes, adding command-line support for quickly navigating to them, and so on.

An Aside: My Scripting Environment

Peripherally related to this, working across both platforms (and having to set up a new Linux box) brought home to me the state of the cobbled together set of scripts, aliases, symbolic links, and the like which I’ve been carrying around and evolving for the last twenty or more years - most of which live in a single (and now rather monolithic) git repo.

Many of these scripts and tools are quite minor, but add little quality-of-life things to my setup: easy ways of navigating to projects, a standard shell prompt, various bits of auto-completion support, aliases for connecting back to known machines in my setup, ways of sharing settings across machines for things like git, atom, and so on - often by symbollically linking files or folders from the place that an app expects them, to another place in my scripts repo.

Having all of this dumped into a monolithic repo was feeling unweildy, especially when a lot of the repo’s content was now obsolete, or only appropriate to one platform.

Some of these scripts also rely on hooking into bashrc, or placing symbolic links to places like ~/. or /usr/local/bin in a way that was also a bit inflexible and hard to manage. I had a “bootstrap” script that theoretically set everything up right, but it was a bit one-shot and hard to test.

It occurred to me that what I really needed was to be able to split these utilities into individual git repos, install/remove them easily, and have some hooks execute when they were installed/removed. The install hook could manage the creation of symbolic links, and generally hook into bash, git, or whatever. The remove hook could do the reverse, making sure that each individual bit of support could be uninstalled without leaving a mess behind.

This Is Sounding Familiar

At some point I realised that an easy way to add/remove projects (based in git repos), and an easy way to add/remove my personal tools (also based in git repos), was basically the same problem.

Being able to run hooks whilst doing so, and being able to choose where the local copy ended up (somewhere obvious like ~/Projects/ for the former, somewhere more hidden like /usr/local/share/ for the latter) was a feature that could be useful in both cases, and was in any case a per-package kind of setting.

So in summary, I needed a kind of package manager like Homebrew or npm, that wasn’t tied to a particular tools ecosystem or OS platform.

Thus was born Project #4: xpkg.

I wanted to work on this on both platforms, hence I needed the Atom Swift stuff for it. I also wanted to build some features into it that required a more complex build system than SPM is capable of. Thus I needed Builder.

Four Projects Doesn’t Sound To Bad

So all of this is really just a long winded way of talking what I’m currently working on / playing with.

The game is on hold for now (mostly because of the time I’m spending in Linux), but I plan to get back to it when I’ve built out some infrastructure. At the very least, I intend to modularise it in a way that allows most of the code to be built and tested across both platforms.

The Atom/Swift stuff needs work, but I’m kind of using it, so evolving it slowly.

I’m nibbling away at various aspects of Builder, mostly as a result of using it.

The thing I’m spending more time on right now is xpkg - which of course uses Builder for it’s build, and which (in Linux at least), I’m using Atom to work on.

These four projects are up at the top of a pyramid, but inevitably there’s more clutter underneath.

Builder itself isn’t really one project anyway, it’s a small cluster of projects. It’s deliberately modular, with the idea that you can pull in other tools that you need for your build process - tools which themselves are just Swift packages.

In addition, both Builder and xpkg are command line tools that share some common requirements. From my time working on tools for Bohemian (which were written in Python), I got familiar with Docopt. I like it as a solution for the command line, so wanted to use it in Swift - but it turned out to need a bit of work to build on Linux. I’m starting to see some other common command-line patterns appearing in both xpkg and Builder, which I think are going to get split out into a separate package.

My old legacy scripts and tools need splitting up to work with xpkg. Most of them are in private repos right now, but a few - such as a generic way of hooking into the bash startup process, may be of more general use.

I also have a common Logging pattern that I tend to use as one of the “Hello World”-style test projects whenever I change language. Pretty much all of the above are using that.

I have plans to add more tools to Builder, and/or more tools installable with xpkg, to simplify some other repeating patterns, such as setting up Travis to test Swift modules cross-platform.

So there’s plenty to keep me busy. And surprise, surprise, it’s a case of: situation normal - all inter-linked.

It’s the way I like it. From a casual perspective, things are moving forward so slowly as to almost appear static. I’m building an interconnected set of things though, and hopefully the eventual payoff will result in much faster progress for some future projects.

Also incidentally much loved by programmers. It took me a surprisingly long time to work out why people always picked the names foo and bar whenever they were writing out a code example… ↩
Although I was lucky enough to be at the WWDC where it was announced, and managed to pretty much inhale the first version of the reference book within a day or two, moving a codebase like Sketch to it just wasn’t practical at the time. We had too much legacy Obj-C code, and the tools have taken a long time to get to the point where you’d want to use them in anger on a large product. ↩
As platform-neutral as Swift is, at least. ↩
This also predated the (somewhat suprising, and perhaps temporary) move to smaller drives on laptops, which coincided with the SSD revolution, and has put more pressure on disk space on my laptop. ↩

Testing Swift With Travis CI

2018-06-01T13:07:00+00:00

Recently I’ve been working with some cross-platform Swift, that needs to build ok on both macOS and Linux.

Naturally I want to unit test it, and I’d like to have the tests hooked up to continuous integration.

Previously I was a big fan of Jenkins, and whilst working on Sketch I helped to build up a fairly complex testing setup with a bunch of Mac Minis all hooked up as remotes to a Jenkins server.

At the time, it gave us the flexibility we needed, and it still has a lot of things going for it as an approach, especially if you want it to perform a lot of complex operations, including code signing, and preparing and releasing final builds. It does mean housing (or co-locating), and maintaining a bunch of physical machines though, which is a pain in the arse.

These days, my needs are a little simpler, and I’m an even bigger fan of Travis.

The single biggest upside of Travis, for me, is that it’s all in the cloud, based off virtual machines with known snapshots. No more tedious updating of Xcode on every test machine! No more inconsistent results due to minor discrepancies between test machines! No more noisy Mac Minis sitting in my office burning my electricity!

The single biggest downside is that it’s costly, starting at about $69/month for the most basic setup if you want to use it for private projects. The cost is understandable - it’s burning a lot of resources in the cloud, after all - but it could still be substantial barrier to an indy developer (it’s enough to buy a new test machine every year, for example, which you can also use for other things).

The good news, however, is that it’s completely free for open source projects. This is fantastic, and the Travis guys are to be applauded for doing it this way. It is also a powerful incentive to open-source ) - which in itself is no bad thing at all.

As luck would have it, I’m currently trying hard to open-source as much of my work as I can, so Travis is just the ticket.

Swift In Travis

I’m not going to give a complete Travis tutorial here (their own docs are excellent), but essentially the way it works is that you specify an os image (eg “macOS 10.13 with Xcode 9.3”), then some commands to run to perform your tests.

You can also specify a matrix of alternative images and settings (for example, specifying both a macOS and a Linux image), and Travis will run your tests across each permutation.

If there’s a preconfigured image that has the right combination of stuff, this is an incredibly simple process.

However, when testing cross-platform Swift, there are a few wrinkles:

there’s no Linux image that includes Swift
the macOS images don’t always have the very latest Xcode
if you’re living on the bleeding edge, the very latest Xcode might not use the version of Swift you want to test

As a concrete example of this, I wanted to test my Builder project on both platforms.

It’s a plain-vanilla Swift module, with no dependency on Xcode, but it does rely on Swift 4.2.

So how would we go about doing this in Travis? It’s actually pretty easy…

Mac

On the Mac, the newest image contains Xcode 9.3, but Swift 4.2 is in 9.4.

You can, however, download it and install it into Xcode as a toolchain.

The .travis.yml file looks like this:

os: osx
osx_image: xcode9.3
language: swift
sudo: required
install:
- wget https://swift.org/builds/swift-4.2-branch/xcode/swift-4.2-DEVELOPMENT-SNAPSHOT-2018-05-30-a/swift-4.2-DEVELOPMENT-SNAPSHOT-2018-05-30-a-osx.pkg
- sudo installer -pkg swift-4.2-DEVELOPMENT-SNAPSHOT-2018-05-30-a-osx.pkg -target /
- export PATH="/Library/Developer/Toolchains/swift-4.2-DEVELOPMENT-SNAPSHOT-2018-05-30-a.xctoolchain/usr/bin:$PATH"
script:
- swift --version
- swift package update
- swift test

What does this do?

Basically the stuff in the install: section downloads the relevant Swift snapshot, runs the installer to install it, then sets up PATH so that the version of Swift contained in it is the one that will get picked up when we run swift on the command line.

Simples!

Linux

On Linux, the picture is similar, except that there’s no Xcode to worry about, and hence no toolchain installer. You just need to download and unpack the snapshot.

Here’s an example of setup where I’m testing on both platforms, using the matrix feature of Travis:

env:
  global:
    - SWIFT_BRANCH=swift-4.2-branch
    - SWIFT_VERSION=swift-4.2-DEVELOPMENT-SNAPSHOT-2018-05-30-a
matrix:
  include:
    - os: linux
      language: generic
      dist: trusty
      sudo: required
      install:
        - sudo apt-get install clang libicu-dev
        - mkdir swift
        - curl https://swift.org/builds/$SWIFT_BRANCH/ubuntu1404/$SWIFT_VERSION/$SWIFT_VERSION-ubuntu14.04.tar.gz -s | tar xz -C swift &> /dev/null
        - export PATH="$(pwd)/swift/$SWIFT_VERSION-ubuntu14.04/usr/bin:$PATH"
      script:
        - swift package update
        - swift test

    - os: osx
      osx_image: xcode9.3
      language: swift
      sudo: required
      install:
        - wget https://swift.org/builds/$SWIFT_BRANCH/xcode/$SWIFT_VERSION/$SWIFT_VERSION-osx.pkg
        - sudo installer -pkg $SWIFT_VERSION-osx.pkg -target /
        - export PATH="/Library/Developer/Toolchains/$SWIFT_VERSION.xctoolchain/usr/bin:$PATH"
      script:
        - swift package update
        - swift test

Notice that I’ve extracted out the Swift branch and snapshot version into variables. This makes it a doddle to update them when new versions are released on the Swift site.

Every time I push a commit to Github, Travis will run the tests on both macOS and Linux, and shout at me if I’ve done something stupid.

Conclusion

If you’re relying on a particular version of Swift, and there’s a snapshot of it, this is a really easy way to test with it.

If the snapshot that you need doesn’t exist on the Swift site, there’s also nothing to stop you from building it yourself locally, uploading it somewhere, and having your Travis script download it.

Living In Linux

2018-05-21T17:14:00+00:00

Back in February, I mentioned that I was pondering which platform(s) to work on.

Having been a Mac developer for most of the last 30 years, I was starting to feel that it might be time to at least experiment with other things, and also starting to feel a little frustrated with the hardware options available to me if I stuck with buying from Apple.

A few weeks ago I finally got round to doing something about this. I specced out a PC, bought the components, and put it all together.

As part of this process, I obviously had to decide what systems to install on it.

System Choices

The main choices being:

macOS (via the Hackintosh route)
Windows 10
Linux (or some other flavour of Unix)

When compiling the parts list, I made sure that I bought components known to work with Hackintoshing, so continuing to use macOS was definitely an option. It’s one that I’m still holding in reserve, but for now, I haven’t gone that way.

When it comes to Windows, I’m not a fan, but sadly the state of the gaming world is such that if you want to play certain games, you have to use Windows. So I did decide to install that.

I don’t really want to live in Windows though. I’ll reboot into it to play games, but that’s as far as I really want to go.

The main reasons that I’m considering moving away a bit from Apple are related to platform lock-in, and the general corporatisation (is that a word?) of computing. All the same problems apply to Windows too, and in addition to them, I find the usability level to be a lot lower than macOS (admittedly, that may be because of my overwhelming familiarity with the Mac).

So that leaves Linux. I decided to install Ubuntu 18.04, and see how long I could survive in it before I potentially gave up and went back to the Mac.

So far, it’s been a few weeks, and I’m still going!

There are undoubtedly some wrinkles, but mostly I’ve been pleasantly surprised at how easy it is to do all the same things.

Work Tools

On the Mac, I spend most of my time in Xcode, Safari, SourceTree, Atom, or the Terminal.

On Linux, Atom and the Terminal are no problem.

Instead of Safari, I initially switched to Firefox. I picked this in preference to Chrome for the same reasons that I’m using Linux rather than macOS… corporate stuff. Subsequently I’ve also discovered Epiphany (based on WebKit), so the jury is out a little as to which one I’ll settle on.

I haven’t really found a good substitute for SourceTree yet, but I’ve realised that the Atom git/git+/github packages, which I already had installed, do pretty much everything I need. For the rest, I’m also comfortable using git on the command line.

So really that just leaves Xcode. I say “just”, but that’s quite a big hole to fill.

Xcode Schmexcode

I’d already been using Atom for a lot of things elsewhere, including as my text-editor of choice. Two of the things I spent a lot of time working on for the last couple of years with Sketch were the build tools (written in Python), and the plugin system (based on Javascript).

I found Xcode not to be a great environment for this sort of work, so I used Atom.

Most of the coding I’m doing right now is in Swift, and not UI related, so I don’t need things like Interface Builder, Code Signing, or in fact a lot of what Xcode does.

What I do need from an Xcode substitute are:

editing text, obvs…
syntax colouring
decent auto-completion
the ability to build, test, and debug

There are lots of Atom packages out there, and some of them purport to add Swift support, so it seemed like it might be a viable route.

Sadly it turns out that many of these packages are out of date, or don’t work on Linux

However, I’d also already been getting interested in actually working on the Swift compiler / package manager a bit, or at least getting familiar with the tools at a pretty low level. Also, in writing some Atom plugins.

You can probably see where I’m going with this. Right now, my Xcode substitute is Atom, plus some Swift-support packages for Atom, some of which I’m writing myself :)

This is the build-it-all-from-scratch approach! I’d normally dismiss this approach as being bound to lead to endless Yak-shaving, a bit self-indulgent, and an enourmous waste of time! In fact, I still can’t really deny any of that, but this year for me is all about a combination of having fun, exploring new horizons, and generally just following my interests and seeing where they lead me. Right now, trying to turn Atom into a half-decent environment for working with Swift ticks these boxes.

I’m planning to write more about my Atom/Swift work in a later blog post. In the meantime, if you’re curious, have a rummage on github.

(Incidentally, I considered VisualStudio Code too, but their Swift support seems equally non-functional on Linux, and their model for building things like auto-completion appears to be more complex.)

Other Apps For Fun And Profit

Clearly I do need some other applications. I don’t know much about the Linux world. There’s a lot of stuff out there, and from my limited experience, a lot of it is crap!

One thing that has been invaluable is Epiphany’s ability to bundle up individual websites as application (like Fluid does on the Mac).

This has really dug me out of a hole.

Here are some of my other (current) solutions:

Music: my music lives on a Synology box on the local network. On the Mac I mount it as a local share and use Swinsian. On Linux I experimented with doing the same thing, using RhythmBox or Clementine. Both a bit meh, for different reasons. Right now I’m instead using the Audio Station web app on the box itself, bundled up via Epiphany. I also have Spotify and Amazon Music bundled in a similar way.
Social Media: Corebird (for Twitter), Slack is native, Facebook (in the browser), micro.blog via Epiphany.
Mail: I was attracted to Mailsmith, but it doesn’t do GPG signing. Right now, I’m using Evolution, mostly because it does do GPG signing.
Other Bits: Dropbox (works natively), as does Keybase. I used to use 1Password as a desktop app - this has convinced me to stump up the money for a subscription. I connect to my other machines when I need to, using VNC Viewer.

Is That All?

Ok, this is where I have to admit to cheating a little. Or maybe a lot.

I really only use productivity apps like Pages/Word, Numbers/Excel, etc for the occasional business-related thing. From time to time I use graphics editors like Acorn and Sketch, but again, not very often.

I could go down the route of installing Open Office, Gimp, Inkscape, etc, but… I still have a Mac laptop :)

The truth is that I need to work with this sort of document rarely enough that the laptop is perfectly adequate. If I want to do it sitting at my desk, I have VNC. Alternatively, it’s a perfect excuse to relocate to a cafe…

General Look & Feel

So what about the general look & feel?

So far, I’m actually fairly ok with it.

I’ve done a few tweaks and brought a few things like fonts across from the Mac, to make things a little more familiar. I’ve hacked the keyboard layout a bit so that I can use the expected shortcuts for some things, because: muscle-memory. I’ve tried a few Mac-OS Gnome themes, but currently I’m using something else. The one thing I miss is the Mac/Unity-style menubar, but I can live without it.

There are undoubltedly some rough edges in the Linux desktop experience. There was a time when I would have found them just too grating. I think what’s happened since then is a combination of some of those edges being smoothed, and a certain amount of pragmatism developing on my part.

I’ve spent quite a lot of my working life doing cross-platform development of one sort of another.

I’ve never been a big fan of cross-platform UIs. They tend to look ok on the platform that the developer knows, and shit everywhere else. They also tend not to be well integrated with the coolest/most-useful system features on any platform, since those are often the features that are unique to one platform and hard to generalise.

Hence I’ve never been a fan of Electron-based apps, for example.

The reason I’m starting to revise my opinion, however, is all tied up with the long-term direction of travel of the various OSes, and perhaps of my own experiences as a user.

In a world where I’m not sure I want to be forever tied to Apple’s platform (or Microsoft’s, or Google’s), the ability to use open source tools built on open source platforms is becoming a political choice that I’m willing to make sacrifices for.

The ability to integrate deeply into (say) Apple’s killer features also becomes less attractive when using them involves an inescapable tie-in to their ecosystem. Incidentally, it’s also a lot less attractive in a world where the software quality has diminished, and “it just works” is no longer true.

In a world where I’m also switching between platforms(Mac laptop, Linux desktop, Windows for games), consistency also takes on a different meaning. Instead of it being annoying that a Twitter app I use doesn’t support some standard feature of the Mac, and is therefore inconsistent on that platform, it now becomes annoying if the app I use for Twitter has to be different depending on what computer I’m on, and is therefore inconsistent across platforms.

If an Electron app can work across all of them, I’m increasingly ok with that.

It sucks that it doesn’t know about the clever keyboard expansions that I told macOS/iOS about - but maybe I should find another macro expansion system that works everywhere?

I also think that if the ecosystem of things like Electron itself continues to expand, it may start to become the platform. I don’t know if Electron yet supports the concept of add-ons or extensions that you install once and then show up on any Electron-based app, but if it doesn’t yet, it probably will soon (and probably should). Then I have the option of writing one of these which does what I want across all platforms, or writing one of them for each platform, to give myself a consistent experience whilst using different back-ends to achieve it.

How Long Will It Last?

It depends a bit on what I end up doing next, I suspect.

Whilst I’m working with Swift, and the package manager, and low-level things that don’t require Interface Builder, I’m pretty happy where I am. Knee-deep in Yak hair though that may be.

If I start to build a desktop application from scratch, I will have to seriously consider whether I want to try to do with something like Electron. On the other hand, I’m enjoying working with Swift, so I don’t know that I’d want to give that up.

If I get back to the game I’m working on (as a background task - it’s a slow-burner), I’ll have a similar decision to make. Right now it’s tied to a few Apple technologies, but that was mostly to get up and running quickly, and so that I could use Swift. I could disentagle it from those technologies without too much work. I could switch to Unity, if I’m prepared to ditch Swift. I could stick with Swift and just roll my own for the GameKit/SpriteKit stuff I’m using. Etc. Choices, choices.

Fun times! Watch this space…

More Build Adventures

2018-04-24T18:52:09+00:00

After my last blog post back in March about the Swift Package Manager (SPM) and some experiments I was layering on top of it, I actually continued with a bit more work, but didn’t get around to blogging about it.

Cleaner Example

One of the things I did was to split the example out into a separate repo.

Hopefully this makes it a little bit easier to see how, if you use Builder in a project, you are able to pull in all of your tool dependencies (including builder itself), just with a single line of bootstrapping using SPM.

Better Configuration

I also worked a bit on a cleaner way to specify settings and schemes/actions.

Using SPM (and Builder) encourages you to break things up into small modules, and so to do this I added a BuilderConfiguration package that you can pull in to your Configure target, and which lets you specify both the configuration and the build settings using a similar style of syntax to SPM’s own Packages.swift file.

To use this, you just include it in the dependencies section of your Packages.swift file:

    dependencies: [
      .package(url: "https://github.com/elegantchaos/Builder.git", from: "1.0.3"),
      .package(url: "https://github.com/elegantchaos/BuilderConfiguration.git", from: "1.1.2"),
    ],

and then list it as a dependency for your Configure target:

    .target(
        name: "Configure",
      dependencies: ["BuilderConfiguration"]),

This allows you to import BuilderConfiguration from within your Configure target, which lets you write a nice clean configuration like so:

import BuilderConfiguration

let settings = /* see below for more details... */

let configuration = Configuration(
    settings: settings,
    actions: [
        .action(name:"build", phases:[
            .buildPhase(name:"Building", target:"Example"),
            .toolPhase(name:"Packaging", tool: "Packatron", arguments:["application", "com.elegantchaos.example"]),
            ]),
        .action(name:"test", phases:[
            .testPhase(name:"Testing", target:"Example"),
            ]),
        .action(name:"run", phases:[
            .actionPhase(name:"Building", action: "build"),
            .toolPhase(name:"Running", tool: "run", arguments:["Example"]),
            ]),
    ]
)

For simplicity I’ve glossed over the settings part for now, but you can see how the configuration itself can be specified as some settings and some actions.

Actions represent discrete tasks you’ll want to perform. They will typically be things like build, run, test - but they aren’t limited to a particular set of names, you can add as many as you need.

Each action has a name, and some phases.

Each phase represents a “command” that builder knows how to perform. The current set is:

buildPhase: invoke swift build to build something
testPhase: invoke swift test to test something
runPhase: invoke swift run to run something
actionPhase: run another action
toolPhase: invoke an external tool to do something

The most interesting of these is the toolPhase, which you use to run other tools. This is what allows you to call out to perform custom build steps.

Where do these tools come from?

They’re SPM packages of course. You just add them in as dependencies, and they are fetched and built along with everything else. For example, above we’re calling out to an imaginary tool called “Packatron” to bundle up the application.

For this to work, we amend the dependencies like so:

    dependencies: [
      .package(url: "https://github.com/elegantchaos/Builder.git", from: "1.0.3"),
      .package(url: "https://github.com/elegantchaos/BuilderConfiguration.git", from: "1.1.2"),
      .package(url: "https://github.com/elegantchaos/Packatron.git", from: "1.0.6"),
    ],

and add the tool to the target dependencies:

    .target(
        name: "Configure",
      dependencies: ["BuilderConfiguration", "Packatron"]),

With these steps done, the Packatron tool will be fetched and built when our project is first built, and can then be used from toolPhase.

Settings

Earlier I glossed over how you specify settings.

This is done by defining “schemes”¹.

Schemes are hierarchical, in the sense that one scheme can inherit values from others.

This allows you to split up your settings in a logical way. For example if you have multiple products that share some settings, but have other settings that vary, you can make a scheme for each one and another for the common settings.

You can also make the scheme inheritance conditional, based on a filter. This allows you to effectively say “include this scheme if you’re on this platform”, or “include this scheme if you’re building this configuration”.

This results in quite a flexible system which hopefully allows settings to be specified in a clean, logical manner:

let settings = Settings(schemes: [
    .baseScheme(
        swift: ["Dexample"],
        inherits: [
            .scheme(name: "mac", filter: ["macOS"]),
            .scheme(name: "debug", filter: ["debug"])
        ]
    ),
    .scheme(
        name: "mac",
        swift: ["target", "x86_64-apple-macosx10.12"]
    ),
    .scheme(
        name: "debug",
        swift: ["Onone"]
    )
    ]
)

The settings for each scheme are specified as a list of strings.

There are currently four categories of setting that you can specify:

swift
cpp
c
linker

Behind the scenes these values get converted into flags which get passed to the relevant compiler when SPM is invoked.

So in the example above:

swift: ["target", "x86_64-apple-macosx10.12"]

will ultimately be passed to swift build as -Xswiftc "target" -Xswiftx "x86_64-apple-macosx10.12"

Future Enhancements

In the long run the Swift team are working on some proposals to enhance SPM, which will probably make Builder obsolete.

The approach that Builder is taking is currently not really focussed on performance, and is more about experimenting with finding a good syntax for some of this stuff, and just exploring the problem space.

Whilst the SPM solution hasn’t arrived, therefore, I will probably continue to tinker.

The next things I’m looking at are:

a bundler tool which allows you to bundle resources and build a full Mac executable
a better way to specify settings semantically

Building On Swift

2018-03-06T13:08:05+00:00

The Swift Package Manager is a package manager and build system, which comes bundled along with Swift itself.

If you’re developing on a Mac, the obvious route to go down is to use Xcode, which has its own fairly capable build system.

Xcode only runs on the Mac though, so clearly you can’t rely on that if you want to be able to build on Linux. The fact that SwiftPM is present wherever Swift is - on Linux builds as well as Mac builds - is a distinct advantage, and makes it an attractive alternative for cross-platform work, minimising the amount of faffing around required when setting up on a new machine.

Unfortunately, whilst SwiftPM is a decent package manager, at the moment it is quite basic as a build system.

It has some nice clean aspects to it, relying on conventions to infer which code to build, which makes it very easy to get going with. However, it lacks some facilities from Xcode that are fairly essential for substantial products:

specifying standard build settings in external files
being able to hook in code generators or other pre-build steps
being able to package up the built binary with other resources
being able to execute other arbitrary code as build phases
being able to code sign, archive, or post-process the final product
being able to install the final product locally, upload, or submit it somewhere

The SwiftPM team have stated their intention to address many of these limitations, but right now the details of what, how or when are thin on the ground.

As I’ve been experimenting with Swift on Linux, and looking into Xcode-alternatives, I didn’t want to wait for jam tomorrow, so I started wondering what it would take to add the missing capabilities, and how best to attempt it.

The result of this is a tool I’ve imaginatively called builder.

Builder

Builder is an experimental overlay on SwiftPM.

In thinking about its design I considered integrating it into SwiftPM (which is open-source, after all), but that would have meant asking people to custom-build SwiftPM in order to use it, which seemed like a high barrier. Instead, it is built as a Swift package itself. You can add it as a dependency to your own product, and bootstrap your project build by first building & running builder using SwiftPM¹.

The basic capabilities provided by Builder are:

running code at configuration time
specifying build settings
complex builds which chain together multiple build steps
calling out to external tools during the build

Whilst achieving this, the guiding principle has been that at all times we should use SwiftPM itself to fetch, build, and run any external code involved in your build.

If your tool infrastructure itself is all written in cross-platform Swift², it should be possible to bootstrap your project build on any platform that has the swift command installed, just by:

cloning your project
using swift run to fetch and run builder
using builder to build your project

Trying It Out

You can find Builder on github here.

As well as the code for builder, the repo includes an example project that you can try it out on.

To give it a go:

git clone https://github.com/elegantchaos/Builder.git
cd Builder/Example
swift run --package-path ../. --static-swift-stdlib -Xswiftc "-target" -Xswiftc "x86_64-apple-macosx10.12" Builder run

What this does is to:

clone builder
build and run the tool itself, which…
…builds then runs an example product

How It Works

Builder works by looking for a special target called Configure in the package that it’s trying to build.

This target is defined in package’s Package.swift file, along with all its other targets and dependencies.

If it finds this target, it uses swift build to do the following:

build the Configure target
run the resulting executable and capture its output
parse this output to obtain a dictionary describing how to build the actual package
extract compiler and linker settings from the configuration dictionary

It then looks in the configuration for a scheme³ matching the name that was supplied on the command line (or build if none was supplied). If present, this scheme describes a set of commands to execute in order to perform the build for that scheme.

These commands can consist of:

build: build a product using swift build, applying the build settings from the configuration
test: test a product using swift test, applying the build settings from the configuration
run: run a product using swift run, applying the build settings from the configuration
scheme: perform another nested scheme
anything else: treat the named command as a dependency; build and run it with swift run, passing the arguments specified in the configuration

The real flexibility comes from that final option.

If Builder encounters a command name that it doesn’t recognise, it assumes that it’s an external tool that it needs to build. It looks for it in the dependencies, and then uses swift run to fetch the code, build and run it. In this way, not only can you run arbitrary tool code, but if it’s written in Swift, you don’t have to ask your users to first install it⁴.

Pros And Cons

This is an experiment, it’s at a very early stage, and it definitely has some pros and cons.

On the plus side, it allows you to layer a more complex build system onto SwiftPM. The SwiftPM package format already has the capability to include arbitrary Swift code in it, but that code can’t depend on external dependencies. By moving the configuration code into its own target, builder gets round this. It also has the potential to eliminate the need for non-deterministic swift from the Package.swift file, which may be a bonus for tools that want to manipulate the file programmatically.

On the negative side, having to build another target is a performance hit. Worse, since Builder is a layer on top of it, SwiftPM has no knowledge of what Builder is doing. It can’t reason about the actions that any external tools are performing - for example in order to prevent them from running when nothing has changed. This is a pragmatic trade-off right now. If SwiftPM had hooks into its underlying dependency system⁵, in a way that allowed external tools to describe the dependency relationships they introduced, then Builder would be able to integrate with SwiftPM more cleanly (or wouldn’t be needed at all). Maybe one day…

For More Information

This has deliberately been a brief tour.

There is more (and more rambling!) information to be found on the github repo, including a list of flaws and ideas.

I’m still very much tinkering with this system, and lots of things are in flux - including a lot of the terminology which is quite confusing since it overloads terms that we already know from SwiftPM or Xcode, in ways that may not quite match expectations.

There are also plenty of rough edges. A system like this would really come into its own with a rich set of tools already built around it, minimising or even eliminating the need to write any tool or configuration code yourself. Widespread adoption would encourage these tools, but at this stage you largely have to use your imagination - which can make the whole concept appear a lot less user-friendly than it actually has the potential to be.

If you have any feedback, I’d love to hear it.

I’m tracking issues and ideas myself on Github, or there is a thread on the Swift forums.

Footnotes

Alternatively, if you prefer the convenience of having it available everywhere, you can clone it as a standalone project, build it, and install it into your path somewhere. ↩
Clearly, many large projects are going to rely on tools that aren’t going to be written in Swift, or that have to be manually installed. That’s a problem that is beyond the scope of any build system to solve, but it’s a scenario that builder can cope with, since you can easily write a build phase in Swift which calls out to external tools. My aim though was to make it at least possible to evolve a rich ecosystem of Swift-only build tools. ↩
This is a bit like an Xcode scheme, but not quite the same. I should probably find a different name for it. ↩
with brew, apt-get, npm, or whatever. Obviously if you need to install and run something from one of those other ecosystems, you can. You just write a Swift script which does it. When you do that, you potentially open yourself up to platform variations which make your code less portable - but that’s your choice. ↩
it uses llbuild under the hood, I believe. ↩

Decoding Dictionaries in Swift

2018-02-21T12:21:39+00:00

I seem to spend at least half of my programming life transferring values between an object or structure and some sort of dictionary format.

Swift’s Codable support is really great for doing this when you’ve got JSON or a Plist, but not all dictionary data ultimately lives in a file.

For example, I was recently doing some stuff with the Disk Arbitration framework. This has a DADiskCopyDescription call which gives you back a dictionary of known keys (many of which may be missing). This dictionary only ever exists in memory.

What I really want to do in this situation is extract some of these keys into a structure or object as a way of validating that I have what I need, and discarding anything I don’t need.

There will probably be some keys that are essential. If the dictionary doesn’t have them, I want to throw an error.

There may be other keys that are optional. In some cases I’m happy to mark these as optional in the structure, so that I can tell whether they were in the dictionary. In other cases I want a non-optional property in the structure, and to use a default value if the dictionary doesn’t have one. Boolean flags are a prime example here - if the flag is in the dictionary, I want the value, but if not, I want to assume that the flag value is false.

I want to do all this with the absolute minimum of boilerplate, yet I find myself having to write initialisers with this sort of stuff in them:

name = info.stringValue(key: kDADiskDescriptionVolumeNameKey)
device = info.stringValue(key: kDADiskDescriptionMediaBSDNameKey)
id = info.stringValue(key: kDADiskDescriptionMediaUUIDKey)
removeable = info.boolValue(key: kDADiskDescriptionMediaRemovableKey)

etc…

The Codable support would be ideal here, except that as it comes out-of-the-box, it appears that I’d have to convert the dictionary into JSON first in order to convert it back. This seems… non optimal… so I set out to make encoder / decoder classes which just work with dictionaries.

Here’s One We Made Earlier

After a bit of research I realised that in fact Apple have done almost all the work for me in the implementation of JSONEncoder.swift - they just haven’t exposed it.

It turns out that Foundation’s JSON codable support does in fact work using dictionaries as an intermediate form, but the code to do so is marked as fileprivate.

Since this is open-source code though, it’s a relatively straightforward task to extract the code, clean up a few things, and make it into some standalone classes, so that’s what I did.

Usage

The code includes some unit tests which illustrate the basic usage patterns.

In a nutshell, however, it goes like this:

struct Pet : Codable {
  let name : String
}

struct Person : Codable {
  let name : String
  let age : Int
  let pets : [Pet]
}

// to encode...
let test = Person(name: "Sam", age: 48, pets:[Pet(name: "Morven"), Pet(name: "Rebus")])
let encoder = DictionaryEncoder()
let encoded = try encoder.encode(test) as [String:Any]
XCTAssertEqual(encoded["name"] as? String, "Sam")
XCTAssertEqual(encoded["age"] as? Int, 48)
let pets = encoded["pets"] as! [NSDictionary]
XCTAssertEqual(pets[0]["name"] as? String, "Morven")
XCTAssertEqual(pets[1]["name"] as? String, "Rebus")

// to decode...
let dict : [String:Any] = [ "name" : "Sam", "age" : 48, "pets" : [ ["name" : "Morven"], ["name" : "Rebus"]]]
let decoder = DictionaryDecoder()
let decoded = try decoder.decode(Person.self, from: dict)

XCTAssertEqual(decoded.name, "Sam")
XCTAssertEqual(decoded.age, 48)
XCTAssertEqual(decoded.pets.count, 2)
XCTAssertEqual(decoded.pets[0].name, "Morven")
XCTAssertEqual(decoded.pets[1].name, "Rebus")

Supporting Default Values

Taking Apple’s code gives us the basics of what we need, but one place where it falls down is in its treatment of missing values when decoding.

The way that coding seems to work by default is that if you’re decoding something and a key might be missing, you need to mark the corresponding property in your structure as optional. Fail to do this and you’ll throw an error if you try to decode something that doesn’t have all the required keys.

That makes sense, but I like to avoid optional values in my structures whenever I can.

It would often be acceptable to substitute a default value for a missing one. The boolean example that I mentioned above is a prime example - using false for the value of a missing property will often make perfect sense. Similarly, using "" for a missing string property, or 0 for a missing numerical property may often be good enough. You lose the ability to tell that the value was actually unspecified, but if it leaves you with a completely optional-free structure, it’s often a tradeoff worth making.

It turns out to be relatively simple to add support for this to the code, in the form of a MissingValueDecodingStrategy.

By default, if a value is missing, the old code would throw an error. By setting the missing value strategy to useDefault instead, the code will now attempt to replace missing values with a sensible default. Right now this just works for the types mentioned above, and the default value is set

Possible Enhancements

For now, this is just an experiment.

I’m slightly surprised that Apple haven’t exposed this functionality by default - perhaps it was an oversight, or perhaps they felt that doing so would encourage a reliance on dictionaries that they’re trying to discourage.

Either way, the code currently feels potentially helpful to me, but time will tell if I actually use it widely or if it’s just a curiosity.

If I do use it, there are a few things that could be done to improve it.

Swift Dictionaries

Internally it’s (mostly) working with NS-types, since that’s what the JSON code needed. Native-swift dictionaries are bridged, of course, but not without some cost, so one obvious enhancement would be to add a parallel implementation which works natively with swift arrays and dictionaries.

More Generic Code

The JSONEncoder source itself contains a surprising amount of repeated boilerplate code. I’m pretty sure it could be tidied up and made more compact with some judicious use of generics.

This would probably also simplify the process of producing both NSDictionary and native Swift variants.

Flexible Defaults

One can imagine a few ways to be more sophisticated about supplying default values when keys are missing.

It would be possible to register prototype objects, for example, who’s properties could be copied to fill in missing values. These could be looked up by type, but also potentially by coding path, perhaps with some wildcard or partial matching.

JSON-specific Code

Because JSON supports a limited set of types, the original encoding/decoding code has some support in it for dealing with things like dates - converting them to/from strings so that they can live in a JSON file.

For plain dictionaries, maintaining these restrictions may make a lot of sense, depending on your use case. They aren’t strictly necessary though - it’s fine to leave some things as native objects in a dictionary.

It might make sense to revisit this code, and either remove it or make it more flexible.

Swift Without Xcode

2018-02-14T00:00:00+00:00

I spent a bit of time over the last couple of days looking into the options for Swift without Xcode, aka Swift on Linux.

What it seems to boil down to is: there’s not a lot out there for free.

I’m quite happy (for now) with the Swift Package Manager as the basic build tool, so all I really want is syntax colouring, auto-completion, and debugging support. I can live with lldb if I have to, but…

For the first two, there are some options, but if you want any kind of symbolic debugging then good luck with that.

The main choices seem to be:

Atom: this
Visual Studio Code: this or this
JetBrains CLion: this

The Atom option is very basic, but could be improved quite a bit I suspect, for not a vast amount of effort.

For VS Code, vscode-swift looked promising, but I don’t think it works on Linux and in any case it’s currently moribund as Mr Owens has been assimilated!

As yet I’ve not tried the CLion option. It’s JetBrain and commercial, so could well be good, but won’t be cheap.

I’m wondering if I’m missing something else here. Is non-Mac Swift support really so bad? Is Swift-on-other-platforms really just a fantasy?

I’m sorely tempted to start hacking on something of my own for Atom. Monumental Yak-shaving will ensue no doubt, and I’m not really sure I want to learn enough about making Atom plugins to be able to do a decent job. Something, however, needs to be done…

Playing with FIDO U2F

2018-02-12T00:00:00+00:00

I expect you’re wondering what I’ve been up to. I don’t blame you, I’ve been wondering too.

In order that we can all sleep easy at night, I think I’m going to blog a bit of a developer diary.

I’m not sure I’ll manage a post every day, but I’ll try to keep a regular commentary.

In all seriousness I don’t expect it to be of great interest to anyone else, but in the past I’ve found it a useful discipline to keep a log of what I’ve been doing each day, and I may as well do it here.

Since finishing with Sketch I’ve been playing with a few very small new projects, but I’ve also spent a lot of time catching up on a large backlog of boring admin tasks.

I’ve no doubt that I’ll talk about some of my own projects in the future, but actually the thing I’ve been working on most over the last couple of weeks is the open source project Safari-FIDO-U2F.

I got interested in the idea of using hardware keys for authentication a year or so ago, and bought a couple of cheap ones to play around with.

Having done so I discovered that there’s no native support for them in Safari! Luckily there was a plugin project on Github that looked like it might do the trick.

I gave it a go, and it didn’t really work, but I decided to rummage around a bit in the code, and came up with one or two minor suggestions which I submitted as PRs.

The original developer got in touch and said words to the effect of “thanks very much, but I’m afraid I’m not really maintaining it any more as I no longer use OS X” for work.

The project appealed to me, so I decided that I’d keep hacking away at it anyway, and after a bit of back-and-forth I sort of ended up inheriting it.

I think what attracted me most, other than the chance to get the keys working, was just that it was a really small codebase, and I figured that even with my very limited free time, I might be able to actually help.

So last year I tinkered with it every now and then, but that limited free time thing made it hard to do much.

It also turned out to be quite a specialised domain, and although there wasn’t much code, it was quite gnarly. FIDO U2F support in general is at a pretty early stage, and only really works out of the box on Chrome. This means that sites which support it at all sometimes assume that if you’re not using Chrome, there’s no point bothering. They also sometimes make assumptions based on the Google/Chrome implementation, which the Safari plugin has to try to play along with.

The architecture of Safari app extensions in general is a bit messy anyway, mostly because of security considerations. The extension is written in Swift, but also injects javascript into each page. It’s the javascript bit that client pages talk to, and it bounces messages over to the Swift bit to do the heavy lifting, which in turn has to ping the responses back to the javascript side. It’s actually worse than that as there are two javascript components, with one acting as a bridge, doing the initial injection and then passing messages to and fro. Safari talks to the native code via XPC, and the extension is actually embedded in its own application bundle (the application doesn’t have to be running, and doesn’t talk to the extension even when it is running). This is all a bit of debugging nightmare, as anyone who’s ever done any sort of cross-process or plugin development can probably imagine. It is possible to symbolically debug both sides, but it’s not easy. Especially when you have no unit tests on either side - which the code I inherited didn’t (that’s not a criticism, the original project was a quick hack and awesome in that it paved the way and showed how to do the actual hardware bit).

Because of all this, not a lot happened to it in 2017, but even with very limited input from me, a few people showed a bit of interest in using it, and I still quite wanted to be able to use it myself. So when I had some free time this year, I decided I’d take another look, and I’ve spent a few days over the last couple of weeks doing quite a bit of rewriting.

It’s still not perfect, but it is now starting to actually work. Most critically, I can use it log into Github with a hardware key, which was my original use case.

I’ve pretty much rewritten all the original code, on both the Swift and JS sides. I’ve also added unit tests to both sides (using xctest for the Swift, and Jest for the javascript). It’s given me the excuse to play with Travis-CI too, and I’ve got that set up now to run the tests and keep me honest.

There’s probably still quite a bit of cleanup I could do. I’m no Javascript expert, despite finding myself having to do quite a lot of it on Sketch, since our plugin architecture relied on it. Come to that, I’m no Swift expert either, since Sketch was a pretty mature codebase and basically still Objective-C.

I won’t go into any more detail here about the actual code. Take a look at the Github project if you’re interested, or ping me.

It’s been a fun little project, and I expect I’ll keep tinkering with it for a while, though I suspect it has a limited lifespan.

Ultimately I imagine that either U2F keys will take off, in which case support will get baked into Safari itself and this plugin will become obsolete, or someone will come up with a better mechanism for hardware two-factor authentication, and U2F will slide off into obscurity, taking the plugin with it :)

Pondering Platforms

2018-02-09T00:00:00+00:00

For the first time in a long while - possibly ever - I have the time, the space, and the funds to work exclusively on my own software.

It may not last forever, but whilst it does it is fantastic, and I’m really relishing the chance to start with as much of a blank slate as I can manage.

As part of this process, I’ve recently been examining some basic assumptions, and thinking the unthinkable: maybe I shouldn’t be working in the Apple ecosystem?

This is partly motivated by the desire to upgrade my hardware, and the reality that what’s on offer from Apple currently is under-powered, over-priced, virtually un-upgradeable, or all of the above.

It’s also party motivated by frustration with the quality of the tools at my disposal, and the sense that Apple’s focus has slipped badly in the last few years, and software quality has suffered as a result.

To be fair though, part of the motivation is just the desire for a change.

The main flaw in this plan, of course, is the lack of decent alternatives. I’ve got a Windows 10 license (which I use to play the occasional game), and despite wanting to be open minded, I truly feel that it sucks. This seems to pretty much leave only one or other flavour of Unix-derivative. I’m increasingly ok with that, in principle, but in practice I find it to still be a world of incredible pain where even the simplest details seem to have been overlooked. As Winston Churchill said, “Mac OS is is the worst operating system, except for all the others.”.

Nevertheless, I’m seriously contemplating speccing out a high-end PC and putting Linux on that, instead of splashing on an overpriced iMac Pro or waiting for the mythical Mac Pro / Mac Unicorn.

With this in mind, one thing I am experimenting with is Swift-on-other-platforms. In particular I’m interested in trying to use the Swift Package Manager as my basic tool for modularising builds, and on seeing how good an editing / debugging environment I could get set up without relying on Xcode. I’ve been down this sort of route before, and I can easily imagine that I’ll conclude that I need to stick with Xcode on the Mac for now, but it is at least an interesting area to play in.

More about Swift later, by the way. I kinda like it, but only (at this point) kinda.

Moving Things Around

2018-02-01T00:00:00+00:00

When I originally created this site, it had a “news” section, which over time mutated into a blog.

At a later date I realised that personal things and work things were getting a bit muddled up, so I split off the personal stuff to Born Sleepy. In doing so, I envisaged this blog going back to essentially just being a “news” service, and most of the posts were just announcements about software releases.

At the moment there aren’t really any software releases, however. In addition, Born Sleepy, which was supposed to be my personal blog, still ended up carrying a lot of technical posts, as well as completely non-technical ones.

So I’ve decided to re-organise things once again. It’ll keep you all on your toes, if nothing else.

As a result I’ve moved back most of the technical posts to this blog, and henceforth decree this as the place I’ll write about the technical things I’m doing. Born Sleepy will very much remain active, but will be restricted to non-technical things.

I expect I’ll end up changing it all again in a while, but let’s see how long this plan remains in force!

The Kraken Wakes!

2017-12-19T00:00:00+00:00

It’s been pretty quiet around here recently.

Much as I wish it was because I’d been lounging on a beach somewhere, sipping cocktails and generally chilling out, it’s actually because I’ve been spending most of my time contracting on Sketch, and it hasn’t really left any space for other projects.

That’s all coming to an end in a few days, as I’ve decided that it’s time to move on.

So with a bit of luck you will see the sleeping ~giant~, erm, minnow that is Elegant Chaos slowly come back to life.

I have a large number of potential apps in mind, but at this stage I’m in no particular hurry to settle on one of them, and in fact I may take a little time out before doing anything much. I might even lounge on a beach or two, but I live in the Outer Hebrides so I will be well wrapped up if I do.

That said, if you have a project that you think I’d be interested in, don’t hesitate to get in touch.

Some Content Removed

2016-05-30T00:00:00+00:00

I’ve finally got round to moving my blog Born Sleepy over to Jekyll as well.

As part of this, I’ve been rationalising which posts live where. Originally both this and Born Sleepy were part of the same site, and when they forked, I retained all the old posts in both places for the sake of any old links.

Now though I’ve decided to retrospectively tidy things up a bit. I’ll attempt to add enough redirections that old article links still work, but the indexes for both sites should now reflect just the content that is relevant for them.

Site Migration

2016-04-28T00:00:00+00:00

As you can probably see, this site has changed!

I’ve finally migrated it away from Drupal (I was still on 6!), and it’s now a static site built with Jekyll.

I’ve probably missed a few pages along the way, so please let me know if you spot a broken link or some missing content.

Thoughts on Apple and Panorama

2014-12-20T00:00:00+00:00

Having the BBC make its Panorama program all about Apple is tabloid sensationalism, and it sends out completely the wrong message.

It’s saying to companies who attempt, in however ineffectual a way, to do the right thing: “stick your head above the parapet, and we’ll monitor you twice as hard as all of your competitors, even though they are engaged in exactly the same practices”.

The exploitation of labour in poorer countries is a massive problem, but it’s also what the current free trade, free market economies of the richer countries are largely based on.

Our cheap consumer goods don’t appear like magic from nowhere. These things only cost what they do because the people who make them were paid a pittance to work too long in dangerous conditions, and the materials they were made of were sourced in a similarly dubious way.

I’m not saying that it’s somehow easy for us as individual consumers to combat this, but it’s completely hypocritical for anyone who owns a computer, tv, mobile phone etc to single out individual companies without examining the conditions in which their own possesions were produced.

The way to solve this problem is to regulate. We need to raise the burden of proof on manufacturers and sellers that the wares they are peddling have been produced ethically and sustainably. We then need to make it illegal to import and sell things that don’t meet the standard.

That is something that can only be done by governments, and it can only be done if the political will is there to do it.

Which comes back to voters.

Which means you.

Premature Pessimization and the like...

2014-10-27T00:00:00+00:00

A nice quote from the developers of SQLite, who have apparently got some serious performance improvements in their latest version:

The 50% faster number above is not about better query plans. This is 50% faster at the low-level grunt work of moving bits on and off disk and search b-trees. We have achieved this by incorporating hundreds of micro-optimizations. Each micro-optimization might improve the performance by as little as 0.05%. If we get one that improves performance by 0.25%, that is considered a huge win. Each of these optimizations is unmeasurable on a real-world system (we have to use cachegrind to get repeatable run-times) but if you do enough of them, they add up.

I have from time to time been accused of being too obsessed with seemingly trivial performance issues when writing everyday code.

There is an orthodoxy based around the concept of premature optimization which seems to encourage some people to believe that performance should be wilfully ignored when writing code, and only dealt with in an isolated step, using tools like Instruments, once all the dust has settled.

Even then, there is a tendency to focus on the low hanging fruit - the top few methods that show up in a profile - and to ignore the rest, or throw up one’s hands at the prospect of improving them. Small (or not so small) overheads that show up in all the code, such as those associated with message passing, memory allocation, and that sort of thing, can easily get overlooked. Similarly, the decision to aim for a particular programming style of idiom can sometimes overlook the fact that the choices they impose have consequences, like lots of dynamic allocation, or lots of memory copying, or lots of synchronisation, or hitting memory in the wrong order and screwing the cache.

I’m not saying that the basic premise of the premature optimization argument is wrong - far from it. It does make sense not to waste massive amounts of time doing insanely complex optimizations too early, on the wrong code. It does make sense to use tools to guide you, rather than guessing. It does make sense also to write clean code that makes your intent obvious.

Most of the time in any case the biggest improvements come from picking the correct algorithms, rather than in twiddling individual lines of your code.

What numbers like the ones quoted above show though, is that a large number of small improvements to performance can have a massive impact in aggregate. You shouldn’t obfuscate your code unnecessarily or obsessively, but if there are two ways to achieve the same aim, both of which are clean and easy to understand, and one of them is obviously more efficient in speed or space, then you’d be a fool not to choose it.

You can only make an informed decision about which implementation to choose if you have some basic awareness of performance and the implications of your choices. Aiming for a consistent style (functional, object-oriented, whatever) probably makes a lot of sense if it cleans up your code base and makes the whole thing easier to understand; but only if you acknowledge the impact is has.

It’s not wise to just defer even thinking about all this stuff until some mythical optimization phase later.

It definitely is wise to gain a basic understanding of how the building blocks of your language work, and roughly how the things on which you build are implemented, and to make decisions accordingly.

In case you’re wondering, the title of this post comes from the following quote by Herb Sutter:

Definition: Premature pessimization is when you write code that is slower than it needs to be, usually by asking for unnecessary extra work, when equivalently complex code would be faster and should just naturally flow out of your fingers.

Hubris?

2014-10-27T00:00:00+00:00

A cautionary tale from Christoffer Lernö: To Swift and back again

Doesn’t inspire a great deal of confidence.

The big thing that worries me about Swift is the way that Apple’s culture (cult?) of secrecy meant that it was developed in a virtual vacuum, and not even widely dog-fooded within Apple.

Writing a new programming language is a bit like doing a cover of Stairway to Heaven. It’s the kind of thing every aspiring programmer wants to do, and the results usually range from mildly embarrassing to excruciatingly bad. To pull it off, you have to be fucking good.

There’s no denying the pedigree of Swift’s authors, but I actually think that a better name for the language might have been Hubris.

Code signing is flaky and unreliable

2014-10-27T00:00:00+00:00

A bug report from Tom Harrington (via Michael Tsai): Code Signing Is Flaky and Unreliable

I’m not sure flaky and unreliable is quite how I’d describe it.

I’d describe it as: unnecessarily complex, overly bureaucratic, badly supported, incompletely documented and subject to random change at any point without notice.

Other than that, it’s really good.

The Point Of Optionals?

2014-10-26T00:00:00+00:00

Optionals are one of the more interesting things in Swift, if you’re coming from a C/C++/Objective-C background.

I think I grok them now, and this is my attempt to explain it - to myself and others. Not so much what they are, but why they are.

Put crudely, they’re a way of saying “my variable (or parameter) can either contain an object of type X, or nothing”.

Coming from pointer based languages, we’re used to doing representing this as just a pointer to an X, which the pointer being nil (or NULL, or zero) to indicate the “nothing” case.

Formalising this concept at first seems a little esoteric. It’s not though.

Coming from pointer based languages, we’re probably also used to the idea that pointers can be dangerous. Quite apart from the scenario where they end up accidentally pointing to random memory, the fact that they can be nil means we have to either:

Make sure that they aren't
Impose an entirely voluntary convention that says “yeah, well, I know that in theory I might pass you nil here, but I promise I never will”. We then have to hope that everyone gets the memo.

In the first case, things are potentially safer, but we can start eating into performance. The overhead is tiny in any one instance, but in a complex web of nested calls over a big object graph, it can conceivably add up. More importantly perhaps, we also add a bit of cognitive baggage that we have to mentally strip away when reading / modifying / reasoning about the code. We have to think about the case where we can’t do what we wanted to do because we were given a nil. We get used to doing this and it becomes second nature, but it’s still there.

In the second situation, things get a bit more hairy. We may think we’re in total control - we might even be right - but we don’t really have any way to verify this. Nor does the compiler in all but the simplest of cases. We can use assertions to try to check our convention is being adhered to, but then we’re back to adding some mental baggage.

Optionals don’t entirely solve any of this, but they seem to me to do two important things.

They make it easier to represent this situation, in a way that tells the users of the code, and tells the compiler, and lets both know not to make assumptions.
By existing, they allow the non-optional variant to exist. In other words, they allow you to formally express “no, really, this will always be an object of type X*”.

What’s so great about this is that it makes it possibly for you to dump a lot of that mental baggage, for a lot of the code. When it makes no sense to deal with the “nothing” scenario, you don’t have to any more. You *require* there to be an object, and leave it up to the caller to deal with ensuring that requirement is met, or not calling you if it can’t be met.

All of this will seem blindingly obvious to anyone who is used to this concept from another language, but what wasn’t completely clear to me at first was some of the implications.

What it took me a while to get was that if we encounter an API or some source of objects that is giving us an optional X, we should probably want to turn it into a non-optional X as soon as we can. We want to push the “deal with the thing not existing” code out as far to the edges of our call graph as we can, and validate things as early as we can, and as close to the source of the objects as we can. This gives us more chance to deal with unexpected situations early before they become bad, and means that the bulk of the code doesn’t have to.

I think that the loss of the mental baggage in the rest of the code will actually be substantial in many cases, and will be a great aid to productivity (plus a small aid to efficiency).

I may be way off course here, mind you. If I am, please correct me!

[Update: I’ve been saying “object of type X”, but of course really I should just be saying “type X” I think. They don’t have to be objects].

[*The funny thing is, it’s kind of like a better, more dynamic version of reference parameters in C++, and I had completely got used to that back when I did a lot of C++, and always tried to use const references rather than pointers when I could. It’s been a while since I’ve had to think in those terms though, and I’d rather got out of the habit :)]

No Single Swift Style?

2014-10-26T00:00:00+00:00

I really don’t agree with this post: No Single Swift Style by Jeremy Sherman.

His conclusion (that we won’t end up with one idiomatic style) may be correct, but I really don’t think it would be a good thing.

In fact I think it would be a pretty damning indictment of the language if it was seen to be "too big to be confined to having a single style”.

It would certainly be grist to the mill for people who argue that Swift is too much of a bastard child formed from many disparate influences. I’m not totally sure I buy into that as a criticism - I don’t think a synthesis of old stuff to form something new is necessarily a bad thing at all - but I certainly do think that not all choice is good. Just because we can do things in ten different ways in a given language, it doesn’t mean we should. I rather hope that we do develop some best practise, and that it falls naturally out of the best features of the language (whatever they turn out to be).

Glassboard

2014-10-26T00:00:00+00:00

Interesting blog post from Justin Williams on mistakes he made when taking on Glassboard.

He talks about some technical issues, but it seems to me that the mistakes made were essentially business decisions, including quite possibly the decision to take it on in the first place.

I was always a bit dubious about how Glassboard could ever make anyone any money, so I was doubly surprised when, having had the original developers fail to do so, Justin then decided to give it a try.

There are situations where taking over someone else's product is a really good way to hit the ground running - for example if it’s got no monthly overheads and clear potential for new features and/or growth of the user base.

This seemed to be the exact opposite in all regards. It clearly had a big server infrastructure cost, was in a fairly crowded market, and had a USP that seemed dubious to me at the best of times.

Still - we only learn by making mistakes, so fair play to Justin for taking one for the team!

Why I think Marcus is wrong

2014-03-18T00:00:00+00:00

Marcus Zarra’s NSConference talk today - about how reinventing the wheel is ok, maybe even to be encouraged - was thought provoking. In fact, I think it was downright provocative. Which is great :)

I recognise a lot of the sentiment behind what he was saying, and have undoubtedly had that same urge to reinvent on many occasions.

I generally feel that it’s useful to understand at least one level below the one at which you are working. If you’re using Objective C, understand C. If you’re using C, understand assembler. If you’re using an open source framework, understand how it works or the technology on which it is implemented.

And yes, it’s hard to really understand a coding problem until you’ve tackled it yourself.

Also it’s quite true that generic code is never going to be as optimal as a bespoke solution.

However…

When you use an open source library, you are typically using code that has shipped in existing products. In multiple existing products. Products that have provided the code with hours/days/months of testing with real world data. Is that worth something? Hell yes.

There is code out there which is broken, unused, or just plain rubbish. Undeniable. That’s an argument for choosing carefully, it’s not an argument for not using other people’s code.

The fact that it’s hard to understand a problem until you’ve tackled it yourself makes it tempting to rewrite. Very tempting. I have ripped out and rewritten plenty of other people’s code in my time. Many’s the time I have simplified “unnecessarily complicated code”: “what an idiot this guy is, look at all this crazy shit he’s doing…”.

That same fact is also why you really shouldn’t, most of the time. Unless you have the luxury of enough hours to rewrite the same thing more than once on the same project, or the luxury of doing the same thing repeatedly on each project. When you rewrite, you will fuck up. The fuck up rate will probably never get to zero, but it will almost certainly only reach an acceptable level after two or more attempts.

It’s uncanny how often that “weird shit” that you ripped out near the beginning makes a quiet return (with the variable names changed to protect the innocent), when the subtlety of your understanding of the problem finally catches up with that of the person who wrote the original version.

When we use someone else’s code, the trade off we’re making is to exchange that deeper understanding that we would have gained by doing it ourselves for the luxury of not having to go down all the blind alleys first.

When it comes to the risk that comes with using someone else’s code, I agree with Marcus that it’s always good to understand it.

But if you’re selling your client on the idea of doing an upfront rewrite of something now on the basis that there’s a 20% chance that they might conceivably need to do it later… well, that’s nice work if you can get it, but 80% of the time it’s stuff that they didn’t need to pay for. Sure the cost of that 20% scenario might be much worse than up-front rewrite, but there are other ways to mitigate against that worst case.

Marcus seemed to be describing a situation where you’re basically doing similar projects again and again. In that case I can also see the attraction of starting from scratch each time. Like a master craftsperson essentially making the same chair over and over again, that zen-like quest for perfection is seductive.

Most of us don’t live in that world though.

I’m not saying that we write code once and let it ossify for ever more, or that we grab the first thing off Github that looks vaguely ok and never question it again. There’s a hell of a lot of room in between that and what Marcus seemed to be advocating though.

Re re Stewardship

2014-02-06T00:00:00+00:00

Jamie posted a reply to Mattt about an article Mattt wrote on stewardship of open source code.

I totally agree with Jamie here. Yes, a well maintained and documented library should be the ultimate goal, and if Mattt’s article had been entitled “How To Create A Successful Open Source Project”, or words to that effect, it would be fine.

But the tone did seem to me to suggest that you shouldn’t put code up there unless you’re prepared to go all the way to full-on well supported open source project, and that is frankly ridiculous (and a little pompous).

I have a lot of source code out there on github. I know that a lot of it is out of date, or not as well packaged and explained as it ought to be, and I sincerely regret that. However, I make no claims for it being in anything other than that state.

Most of my public source is code that has been created as a by-product of past or ongoing work. I would love to polish and refine it into a more drop-in form, but life keeps getting in the way.

Faced with the options of putting it out there in the hope that it will help someone (and perhaps inspire someone to help with it), or removing it until I have the time to polish each library to perfection, I think that there is only one sensible choice.

I would encourage anyone who has even the slightest desire to share in the open source community to do so.

Put aside ego, hubris, and insecurity, and put your code up there.

Be open about the fact that it has flaws, invite constructive criticism, and by all means aspire to the perfection that Mattt is talking about. In the meantime, relax for goodness sake!

Neu 2.0b1

2013-10-12T00:00:00+00:00

There’s a new beta of Neu out on the beta software page.

This one is 2.0, which sounds like an exciting change, but actually at this point it’s a small evolutionary step and not that different from 1.3b4. There are quite a few under-the-hood changes still in the works, but they aren’t quite ready for prime time, so haven’t made it yet.

However, what has changed is that support for 10.6 has been removed. It’s always a tricky decision to drop support for an old system, but as a small developer it is really quite necessary to keep the number of systems that you’re testing with to a manageable minimum. With 10.9 just around the corner, and 10.6 now pretty old, the time had come for a change. Rest assured that if you need to stick with 10.6, the old releases of Neu should continue to work, but moving forward, it’s 10.7+ only.

The one other thing that’s changed is that Finder menu support has been temporarily disabled if you are running 10.9 (Mavericks). It turns out that some changes in the Finder under 10.9 have broken Neu’s menu hack, and this can result in the Finder crashing continuously whilst Neu is running. We definitely don’t want that happening, so the best plan in the meantime seemed to be to simply disable it until I can get it all playing nicely once again. Watch this space on that front…

Ambientweet 1.2

2013-09-25T00:00:00+00:00

Ambientweet 1.2 is out!

Strictly speaking I’m treating Ambientweet as in “maintenance mode” now, as it’s not really being actively developed.

However, Twitter’s recent turning off of their old 1.0 API broke Ambientweet, and I wanted to at least get it working again, hence the release of 1.2.

By all means report any bugs that you find still lurking within this version, and I’ll endeavour to fix them…

Coding Guidelines

2013-08-23T00:00:00+00:00

Some interesting coding guidelines from Richard Buckle (@RichardBuckle).

I’ve not read through the whole document yet, but what I’ve read makes a lot of sense.

Like all these things, much of the value comes not with slavish obedience, but with the thought that is provoked. Introspection about what we do as coders, and how we can do it better, is essential.

(oh, and thanks, Richard, for the ECLogging namecheck…)

Update: I’ve read em all now, and I can’t find anything to disagree with.

Hot on the heels - Ambientweet 1.1.2b8 released

2013-07-13T00:00:00+00:00

As is often the way, a new beta follows swiftly on the heels of 1.1.2b7, which was crashing on startup! I’ll repeat the details of 1.1.2b7 here too, since they are essentially the same:

A few weeks back, Twitter made some changes to their service.

Unfortunately, these changes caught me on the hop slightly, and Ambientweet (which hasn’t been receiving a lot of love recently, I must admit) stopped working.

Happily, I have now released version 1.1.2b8, which fixes these problems, and resumes normal service.

It appears that there may also be a slight problem with the auto-update mechanism at the moment. I’m looking into this, but if for any reason you have trouble getting your existing copy of Ambientweet to update, I would suggest simply downloading it again from our beta software page.

Ambientweet 1.1.2b9 - they're like buses

2013-07-13T00:00:00+00:00

You wait for ages, then along come three at once…

Another beta, this time to fix a problem with the preferences panels.

There is still a slight problem with the auto-update mechanism at the moment, so another beta will probably follow shortly.

I’m looking into this, but if for any reason you have trouble getting your existing copy of Ambientweet to update, I would suggest simply downloading it again from our beta software page.

Ambientweet 1.1.2b7 released

2013-07-13T00:00:00+00:00

A few weeks back, Twitter made some changes to their service.

Unfortunately, these changes caught me on the hop slightly, and Ambientweet (which hasn’t been receiving a lot of love recently, I must admit) stopped working.

Happily, I have now released version 1.1.2b7, which fixes these problems, and resumes normal service.

All Change At Elegant Chaos

2013-07-11T00:00:00+00:00

It’s fairly old news by now, but I’ve recently had a slight change of direction and taken a permanent role at Bohemian Coding. As a result, it is past time for me to mention it here, and to address this important question:

What does this mean for Elegant Chaos and our products?

Ambientweet

I’ve made the decision to effectively retire Ambientweet, and from today onwards it will be free (in fact, I changed the price a couple of days back). Ever since Twitter started discouraging third-party clients, it has been hard to justify continuing to work on it, and some recent API changes at Twitter’s end seem to have broken it completely. I will endeavour to fix it again, if I can do so without expending an insane amount of effort, but after that, it isn’t likely to receive much love.

I actually had a lot of plans for Ambientweet, and had also been hoping to do a version for app.net. Perhaps one day this will still happen, but it’s not likely to be soon.

Neu

On the other hand, Neu is still very much alive, and there are some new features on the way.

However, to streamline things slightly, I am going to move directly to releasing a 2.0 version, which will require OS X 10.7 or greater (or possibly even 10.8). This will allow me to tidy up and modernise the code, which will have some internal benefits as well as speeding the development process.

Additionally, I may well remove Neu from the Mac App store. Neu is in that category of helper applications which have a difficult time complying with Apple’s rules for the store.

It doesn’t do anything particularly complicated, but it integrates quite tightly with the Finder, and in the future I have some plans to integrate it further - both with the Finder and with other apps. Doing this whilst staying correctly sandboxed is quite difficult, and is deflecting time from the development of new features.

Early on, I made the decision that I didn’t want to release a new version on my website until I’d had the same version approved for release on the store, so that store customers didn’t feel that they were lagging behind.

Unfortunately, this has resulted in no new official version of Neu being released since Apple required sandboxing. Given that I now have a limited amount of time to work on it, I would prefer to move it forward by adding features, rather than by working around sandboxing problems.

The most pragmatic way to do this is to remove it from the store, for now at least. This is likely to happen when I release 2.0.

What does this mean if you already own a license for 1.0? The good news is that I intend Neu 2.0 to be a free update. If you bought Neu through my store, it will continue to accept your existing license file.

Unfortunately there is no way for a developer to obtain a list of the people that bought a product on the app store. I have some existing technology in Neu which allows non-app store versions to spot that you once ran a licensed app store copy, and to accept this as proof of purchase. This should continue to work for Neu 2.0, but it does have the slight annoyance that would will have to have downloaded Neu from the store and run it at least once on your machine. I will continue to explore other options - possibly I may be able to add something that allows you to request a new license file when the app spots that you have an app store license on the machine. More on this later, no doubt…

Other Products

I have various other products that I was planning on. Most of these are things that I want for myself, so they may well happen, but unless someone invents a time machine, they will take a while…

## Welcome To The Future

So… that’s my news. Hopefully it’s all good. Do continue to keep the feature requests and bug reports coming in (especially for Neu). Please bear with me if I take a while to respond, but I am definitely still here and beavering away!

Neu 1.3b4

2013-05-28T00:00:00+00:00

Fresh off the press today comes Neu 1.3b4.

The main change in this version is an additional template substitution:

{% clipboard %}

When you create a new document from a template, this substitution will be replaced by any text that was on the clipboard.

This allows you to easily copy some text, fire up Neu and select a template, and have it create a new document containing your selection.

You can download 1.3b4 from the beta software page.

Neu 1.3b3

2013-03-12T00:00:00+00:00

Hot on the heels of 1.3b1 came 1.3b2, which fixed the support for OS 10.6 that had accidentally got broken!

And if that weren’t enough, hot on the heels of that came 1.3b3, which adds support for a new service command: “Add To Templates…”.

This command adds a copy of the selected files into Neu’s templates folder.

As always, you can find this version on the beta software page.

The software update mechanism within Neu will also grab this new version, if you have the “Update to beta versions when available” box checked. Unfortunately, if you auto-updated to 1.3b1 on OS X 10.6, you’ll have to download it, since 1.3b1 is totally broken on 10.6.

Neu 1.3b1... work in progress

2013-03-12T00:00:00+00:00

Things have been quite quiet on the Neu front, but under the surface we have been doing a bit of paddling!

Available now on the beta software page is a new beta - 1.3b1 - which adds some new features.

Folder Expansion

The first of these is the ability to use folders properly as templates, and to have variables expanded in the contents of the folders (including in their filenames).

Applescript

The second big change is applescript support. You can now use Neu via, applescript, doing this sort of thing:

tell application id "com.elegantchaos.neu"
    make new document
        with properties {template:template "MyClass", extension:"hpp", replacing:yes, revealing:false, opening:true} 
        at POSIX file "/Users/sam/Desktop/"
end tell

Currently there’s also an alternative syntax, which looks like this:

tell application id "com.elegantchaos.neu"
    make new document
        duplicate template (template "MyClass") at destination name "Blah" with revealing, opening and replacing
end tell

We’re interested in hearing if people have a preference.

Finally, we’ve added experimental support for a proper menu in the Finder.

This is classed as “experimental” because Apple doesn’t officially support allowing apps to add Finder menus, so it involves installing a small hack to the Finder.

You can enable this support in the new “Finder” preferences pane. Note that you may have to log out and in again before the Neu menu appears. You will also need to have Neu running.

Feedback Required

To some extent all of these features are still experimental, and there may be bugs in them.

We’re really keen to hear what you think about the features, and about any problems you encounter.

Unit Testing With Mock Server

2013-01-28T00:00:00+00:00

In a blog post last month I introduced Mock Server, an Objective-C toolkit to help with the unit testing of networking code.

Basically, Mock Server works by running on a local network port, pretending to be some sort of server (ftp, http, you name it). You point your networking code at it in your unit tests, thus avoiding the need to be connected to the network, and ensuring that you always have an available, predictable server to test against.

Since then I’ve been using MockServer a fair bit, and have expanded it’s capabilities somewhat, with hopefully more to come.

To help everyone else use it, in this post, I thought I’d go into a bit more detail about how exactly to set up a unit test to talk to Mock Server.

Using KMSTestCase

MockServer can be used in a number of different ways, but to simlpify things I’ve tried to make it easy to use it in what I think is the most likely scenario: as part of a suite of unit tests using the standard SenTestKit framework that ships with Xcode.

To do this, you need to make just two things:

a responses file
a unit test class that inherits from KMSTestCase.

Response file examples for ftp, http and webdav can be found in the MockServer project.

In a future post I’ll talk about how to set up your own response file to mimic a custom protocol, or to mimic something like an FTP server behaving in a very specific way (great for testing those obscure bugs that only show up on really weird FTP servers).

For now though, I’ll just use the example FTP response file, which provides responses for most of the common commands than an FTP client is likely to throw at an FTP server.

Imports

After making a new source file, you first need to import the MockServer headers that you need. You’ll definitely need KMSTestCase.h, and probably also KMSServer.h:

#import "KMSTestCase.h"
#import "KMSServer.h"

A Class And A Test

Next, you want to declare your unit test class, inheriting from KMSTestCase:

@interface KMSCollectionTests : KMSTestCase

@end

Set Up Server

As a simple example, we’ll just implement a single test, which performs an FTP request.

@implementation KMSCollectionTests

- (void)testFTP
{

First, we need to set up a new mock server instance, and start it running.

Luckily KMSTestCase has a method that does all the hard work for us: [KMSTestCase setupServerWithResponseFileNamed:].

This method takes the name of a response file, and sets up a MockServer instance, running on a dynamically allocated port, and using the “default” set of responses from the response file, which should be added as a resource to your unit test bundle.

We have to check the result of this call (which is a BOOL), in case anything goes wrong with the setup.

	// setup a server object, using the ftp: scheme and taking
	// the "default" set of responses from the "ftp.json" file.
	if ([self setupServerWithResponseFileNamed:@"ftp"])
	{

Because we’re going to fake a download, the next thing we have to do is to give the server some data to return to us when it pretends to respond to the download request.

This is the way MockServer works generally. It doesn’t actually perform as a server at all in any real sense; instead, you give it the thing you want it to return, then run the real client code that sends a request to it, then check that the client code got back the thing that you asked MockServer to return.

So, to set some data to be returned, we use the [KMSServer data] property:

        // set up the data that the server will return
        NSString* testData = @"This is some test data";
        self.server.data = [testData dataUsingEncoding:NSUTF8StringEncoding];

Make A Request

Next, we need to make an NSURLRequest object that we’re going to use in our client code to do our actual FTP request.

It’s important to note here that we don’t have to use NSURLRequest, or NSURLConnection, or any particular client-side technology. We can formulate the network request in whatever way we like, and can use any sort of third-party library (or even low level socket code) to perform the client part of the transaction (which is the part that we’re trying to test).

However, I’m using NSURLConnection here for simplicity, so I need an NSURLRequest, which means that I first need an NSURL object, including the address of the server and the path of the file we want to download.

We know that the server is running locally (as that’s the point of MockServer). However, because the server is using a dynamically allocated port, we can’t just hard-code the URL into the test; we have to figure it out on the fly. We do this by grabbing the port number back from the server, grabbing the scheme from the responses collection, and building up a URL from that information.

Luckily KMSTestCase provides a helper method to simplify this.

In this case we’re going to pretend to download a file called “test.txt” from the root of the FTP server:

		// setup an ftp request
		NSURL* url = [self URLForPath:@"test.txt"];
		NSURLRequest* request = [NSURLRequest requestWithURL:url];

Perform The Download

Next, we want to actually peform the download.

Because this is a unit test, the first instinct might be to do this synchronously. After all, a unit test is just one method, and we can’t perform a test on what we got back until we’ve actually got it. For an asynchronous case we’d have to give back control to the main loop for a while until we somehow know that the request is done, and that all sounds a bit complicated. There’s a danger that the unit test would just finish before anything had happened.

To test in real-world conditions though, we really do want to do things asynchronously. A synchronous test at this point really isn’t a good idea, since (hopefully) we aren’t going to be writing synchronous downloads in our apps.

Luckily, MockServer and KMSTestCase have this covered. KMSTestCase has a two methods: runUntilPaused, and pause.

The first of these hands back control to the main run loop, and pumps it until something calls pause. If we arrange to call this in our completion handler, we can happily set up an asynchronous request, wait for it to do it’s thing, and then return control to our unit test so that we can check the results.

Here’s the code:

	__block NSString* string = nil;
	
	[NSURLConnection sendAsynchronousRequest:request 
		queue:[NSOperationQueue currentQueue] 
		completionHandler:
			^(NSURLResponse* response, NSData* data, NSError* error)
			 {
			     if (error)
			     {
			         NSLog(@"got error %@", error);
			     }
			     else
			     {
			         string = [[NSString alloc] initWithData:data encoding:NSUTF8StringEncoding];
			     }
		 
			     [self pause];
			 }];
	 
	[self runUntilPaused];

Test The Results

Finally, we can check that we got back whatever it is that we were expecting to get back.

In this case we should receive the test string that we asked MockServer to return to us:

    STAssertEqualObjects(string, testData, @"got the wrong response: %@", string);

And that, as they say, is that.

By using KMSTestCase, most of the setup work and all of the cleanup work is done for us, and we can just concentrate on the code that performs whatever network operation it is that we’re trying to test, and then checks the results to verify that they are ok.

Clearly there’s more going on under the hood of KMSTestCase, but most of it is just housekeeping and for a lot of situations it should be sufficient for your needs.

You can obviously use KMSServer directly if you need to do something more complicated - examining the source code of KMSTestCase should give you everything you need.

For More Info

You can find full source code and documentation for MockServer on github.

The Mysterious Case Of The Unit Tests That Ran At The Speed Of Light

2013-01-22T00:00:00+00:00

Once or twice I’ve had the situation where I ran some unit tests in Xcode, and they seem to run incredibly fast. Too fast.

Spoiler: they didn’t Instead of Xcode’s Log Navigator looking something like this:

it instead shows this:

The observant amongst you will have already spotted the flaw in this otherwise nice, friendly, green display of unit testing happiness.

There are no issues because no unit tests ran.

What’s Going On?

Unless you know where to look, just beginning to discover why this might be is a little challenging.

You might think to yourself: “silly me, I must have turned them all off”. A quick look at the Tests settings in the Scheme will verify this:

Nope, they’re on, but Xcode isn’t running them for some reason.

Console Yourself

Normally when something goes wrong with unit tests my first port of call is the console output, and the way I get to it is via the little button that appears on the right hand side of a test when you hover over it:

But with no tests apparently running, there is no little button, so we need to actually go and look in the separate console area.

If you’re the kind of person who leaves everything visible all the time in Xcode, you’ll probably be wondering why I’m even mentioning this, but if (like me), you have behaviors that hide things at various times, then it’s quite possible to not even remember that the console output is still there once the run has finished.

You can get to it like this:

When you do, you’ll discover that something very odd is going on. The normally copious unit test console output has been reduced to:

Luckily, Xcode tells us what’s happening (I’ve shortened the paths):

    The test bundle at CURLHandleTests.octest could not be loaded because its
    Objective-C runtime information does not match the runtime 
    information required by the test rig.

and why:

    This is likely because the test rig is being run with
    Objective-C garbage collection disabled, but the test
    bundle requires Objective-C garbage collection.
    To enable Objective-C garbage collection for the test rig,
    run it in an environment without the OBJC_DISABLE_GC 
    environment variable.

Thank you Mr Xcode, you are so good to me.

So, I got my build settings wrong and forgot to disable garbage collection right? I’ll just go and fix it and… oh, hang on… no I didn’t. No garbage collection here.

So, what the hell is going on?

Back to the console. It looks like otest is having trouble:

    2013-01-22 18:25:46.057 otest[5433:203] *** NSTask: Task
    create for path 'CURLHandleTests.octest/Contents/MacOS/CURLHandleTests'
    failed: 22, "Invalid argument".  Terminating temporary process.

For those who are wondering, otest is the command line tool that loads and runs your unit tests.

So it seems to be having trouble. It’s failed, with error code 22. Which is an invalid argument.

Well, that clears things up then.

Long Story Short

Ok, so let’s cut to (somewhat nearer) the chase.

What’s actually happening here, in the cases that I’ve encountered at least, is that otest is failing to load CURLHandleTests.octest - which is the bundle that contains your compiled tests.

There are probably a few reasons why it could fail. Clearly Xcode thinks that garbage collection incompatibility is right up there. However, also high up on the list is that the dynamic loader is failing to load it because it can’t find a shared library or framework that your test code depends on.

If you are using the default Xcode setup, the octest bundle is built into the Products directory, along with any other dependent targets. If your tests need to link at runtime to any custom libraries or frameworks, you need to make sure that they end up here (or somewhere else that the dyld loader can find them).

This is fairly obvious in most cases - for example if you are unit testing a framework that you’re building, it kind of makes sense that you’ll need the framework. In these cases Xcode generally does the right thing anyway - your unit test target will probably depend on the framework target, and they’ll both end up in the Products directory.

So most of the time you’ll be ok, and perhaps even blissfully unaware of how your tests are loaded and run.

However, things can get tricky when the library dependencies are a bit more complex. For example if you’re testing a framework and the framework itself relies on some other shared libraries that are custom built, and not just pre-installed in some convent location like /Library/Frameworks/.

It’s possible in this case for the framework to embed these libraries as part of it’s own build process, which makes life a lot simpler for people who use it - and will probably not cause the sort of issues I’m talking about. That doesn’t always happen though, for good reasons such as not wanting to end up with multiple copies of a library embedded in more than one framework.

In this situation, where your framework is looking for a shared library that it can’t find, what will happen is a runtime failure to load your framework. This in turn will cause your unit test bundle to fail to load, which in turn will cause otest to bomb out with the useful error 22, which in turn will cause Xcode to LIE TO YOU BY PUTTING UP A BIG GREEN THING SAYING ALL YOUR TESTS RAN.

Sorry. Deep breaths.

## The Fix

So once you’ve figured out what library you’re missing, the fix is quite simple. Just add a build phase to your unit test target which copies the relevant things into the Products Directory:

(yes, in this case we’re linking against our own builds of libssh2, libssl and libcrypto - they aren’t the normal builds, so if they are missing, everything goes pear-shaped).

Happily, if you’ve got the right libraries, all will now be well and the unit tests will run.

What To Link

Of course, working out what library you’re missing isn’t always trivial, especially if it’s a big codebase and/or you aren’t in control of it all.

How to work out which libraries depend on which is a blog post all of its own, but here are a few handy hints.

otool

You can use otool (not to be confused with otest, which I mentioned earlier), to work out what something links to.

You have to run it on the actual binary inside your octest bundle

otool -l /Caches/DerivedData/CURLHandle-flfotwdrtmazzmdpxjznosnizjlm/Build/Products/Debug/CURLHandleTests.octest/Contents/MacOS/CURLHandleTests 

This will produce a big load of output, including things like this:

          cmd LC_LOAD_DYLIB
      cmdsize 104
         name /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (offset 24)
   time stamp 2 Thu Jan  1 01:00:02 1970
      current version 744.1.0
compatibility version 150.0.0

Which tells you that you’re linking against CoreFoundation - which you could probably have guessed.

All the system ones are probably fairly expected and aren’t likely to be the problem. However, if something shows up in this dump that you weren’t expecting, it’s a good clue where to look.

Update: otool -L actually gives you a much more compact output here, just listing the libraries that are linked to:

    @rpath/Connection.framework/Versions/A/Connection (compatibility version 1.0.0, current version 1.0.0)
    @rpath/SenTestingKit.framework/Versions/A/SenTestingKit (compatibility version 1.0.0, current version 4053.0.0)
    @rpath/DAVKit.framework/Versions/A/DAVKit (compatibility version 1.0.0, current version 1.0.0)
/System/Library/Frameworks/Foundation.framework/Versions/C/Foundation (compatibility version 300.0.0, current version 945.11.0)
    /usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 169.3.0)
    /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 744.1.0)

DYLD_ variables

Another thing that may help is to enable a few DYLD_ environment variables and then run otest manually. Knock up a little script like this:

export DYLD_PRINT_LIBRARIES=YES
export DYLD_PREBIND_DEBUG=YES
export DYLD_PRINT_ENV=YES
export DYLD_PRINT_APIS=YES

export OBJC_DISABLE_GC=YES

xcrun otest /Caches/DerivedData/CURLHandle-flfotwdrtmazzmdpxjznosnizjlm/Build/Products/Debug/CURLHandleTests.octest 

What you’ll see when you run this is a whole load of debug output from the DYLD loader itself.

...
dlopen(/Caches/DerivedData/CURLHandle-flfotwdrtmazzmdpxjznosnizjlm/Build/Products/Debug/CURLHandleTests.octest/Contents/MacOS/CURLHandleTests, 0x00000115)
dlopen(/Caches/DerivedData/CURLHandle-flfotwdrtmazzmdpxjznosnizjlm/Build/Products/Debug/CURLHandleTests.octest/Contents/MacOS/CURLHandleTests, 0x00000109)
dyld: loaded: /Caches/DerivedData/CURLHandle-flfotwdrtmazzmdpxjznosnizjlm/Build/Products/Debug/CURLHandleTests.octest/Contents/MacOS/CURLHandleTests
dyld: loaded: /Volumes/titan/Users/Shared/Applications/Xcode/Xcode46-DP4.app/Contents/Developer/Tools/../Library/Frameworks//SenTest
....

This may give you a clue about which library or framework is failing.

Other References

The topic of libraries, frameworks and dynamic linking is a fairly complicated one in its own right.

If you encounter the problem that I’ve described above, the chances are that you’re working with a fairly large and/or complex project with multiple libraries, and you’ll already know a bit about dynamic linking.

If not, these links may help.

Run-Path Dependent Libraries: This article explains how the system finds out where your libraries are at runtime, and how you can set up any frameworks or libraries that you build so that they can be hosted in the the /Library/Frameworks folder, or inside your app, or embedded in another framework, in a way which still allows them to be found.
Mac OS X Debugging Magic: This tech note gives a lot of helpful advice on debugging.
Friday Q&A 2009-11-06: Linking and Install Names: Mike Ash has also covered this topic.
You can also use man dyld in the terminal to get a full list of the DYLD_ variables.

In Conclusion

This problem can be really tricky to solve.

Just knowing what the problem is would be a good start, but Xcode does it’s best not to tell you.

Of course, I have filed a radar. Feel free to dupe it.

Hopefully, if you’ve stumbled across this blog post whilst tracking it down, I’ll have pointed you in the right direction!

2012 Mini Review

2013-01-07T00:00:00+00:00

Well, 2012 turned out to be quite a busy year for me on the contracting front.

Some of the main highlights:

Most of the coding for the iOS game Bag It and Bin It was actually done in 2011, but it finally came out at the end of the year, and I did a bit more work on an update during 2012. Written using Cocos2D, this was great to work on, and it was nice to get the chance to do a little game from scratch.
On the other end of the spectrum, I spent some time in the summer wrapping up a C++ library and turning it into an Objective-C iOS framework for another client. This couldn’t be more different from writing a game, but designing code to be used by other programmers is also something that I enjoy, so this was also a nice little job.
In the early spring, I started doing some MacOS X work for Karelia, which continued through the year (and hopefully on into 2013!). This work has covered all sorts of territory, from unit testing their libraries to some networking and user interface work for their main product Sandvox. They like to keep me on my toes…
Finally, in the autumn I also started working on the SVG import/export component of Bohemian Coding’s excellent MacOS X vector graphics app Sketch. Once again, a very different area to work in, with a whole new set of challenges, but definitely fun. The SVG standard is a bit of a monster, and making software that can understand everything in the standard would be a monumental task, so we took a pragmatic approach, trying to identify the areas that are most often used and tackle them first. It feels a bit like painting the Forth Bridge, in that by the time we’ve done everything there will probably be a new version of the standard out :)

So a pretty varied year, and those are just the highlights.

The one down side of this is that the time I’ve had to spend on Neu, Ambientweet, and some other as-yet-unreleased projects of my own, has been somewhat limited. I really hope that I can rectify that in 2013, and give them a bit more love.

The only problem is that the work for Karelia and Bohemian is continuing, and I’m enjoying it too! If anyone invents a time machine, please let me know…

Mock Server - Unit Testing Network Code

2012-12-06T00:00:00+00:00

Unit testing is great, but when you’re working on networking code, it can be a bit of a pain in the, erm, transport layer.

I’ve been doing quite a bit of network-related code for Karelia recently, and I wanted to be able to unit test, so this became an issue for me.

If there are two things that you definitely want from a suite of unit tests, it’s that they run fast, and that they are consistent. You want to be able to fire off the tests regularly, you want to be able to trust the results, and you want to be able to run them anywhere.

Throw a network server or two into your testing mix, and these bets are generally off. If the server is internal, the tests may not run when you find yourself on the wrong side of a firewall. If the server is public, you probably have no control over it. In any case, you have no control over the bit of soggy string between the test machine and the server, and any number of problems could cause your tests to fail spuriously.

This problem gets even worse when you consider that one of the best uses of unit tests is to check that your code copes when something goes wrong. Networking code is very good at going wrong in exciting and unexpected ways. Making real servers go wrong on demand in exactly the same way each time so that you can check that you handle it is challenging.

Typically the solution to this problem is some sort of “mocking” - i.e. finding some API or object that you can swap out or inject into the system in such a way that your tests can work as if they are talking to a real server, whilst actually talking to something else.

One approach is to somehow intercept the network request and, instead of passing it on to a real server, fake up the relevant response and return it directly. In the Cocoa world, a custom NSURLProtocol is a good way to achieve this, and it’s the approach taken by things like Nocilla. If you can work at such a high level, this is a nice solution as no actual networking need ever take place. You just intercept the URL request and deliver back an NSData or NSError object depending on what you’re trying to simulate.

The only problem for me was that this approach relies on the networking going via an NSURLConnection, and not all of the code that I wanted to test did this - I was testing code using libcurl, for example.

So I needed another approach, which would work with any networking code without me modifying it.

The second solution is essentially to implement a real (albeit very cut down) server. Bind a local port, listen on it for a connection, and respond to it as if you were a server.

Providing that you’re happy to require that your tests all connect to a custom port on localhost, rather than to real addresses, this solution is infinitely flexible.

There’s only one problem… the bit where I said “respond to it as if you were a server”. Isn’t writing servers a bit tricky?

Well, yes, if you want to write an actual server. But if you think about it, most interactions with servers are just a series of query, response, query, response. In a testing scenario you know what network operation you’re going to do, so you should be able to figure out what query the underlying code will actually send to the server. All you need to do then is to arrange for the right responses to come back.

Even better, in a testing scenario you generally only have to deal with one request happening at once (not always quite true, but near enough a lot of the time).

Subsequently I’ve discovered that there are a few solutions out there that do this sort of thing, but often they seem to just pretend to be a particular kind of server, most commonly a web server. I needed something that could pretend to be an FTP server, a web server, do WebDAV, maybe do SFTP, etc.

More to the point, being a reckless sort, I thought: how hard can it be?

I decided to find out, and thus MockServer was born.

MockServer

In a nutshell, the way MockServer works is this:

at the start of your unit test, you make a KMSServer object, and give it a responder (more on these later)
you then fire off the network operation that you want to test, but instead of pointing it at a known port like 80 on a real server you point it at localhost, at the port that the KMSServer object told you to use
you now tell the KMSServer object to run until your operation is complete - this pauses your test
once your operation has completed (or failed), your normal completion or error routines will be called
in your completion routine you tell the KMSServer object to pause
pausing the KMSServer object returns control to your test, and you can check for the right responses.

Internally, MockServer just opens up a port and listens for connections.

When it gets a connection, it opens up a socket and waits for input.

When it gets input, it passes it off to a responder, which matches it against things that it knows about, and sends back some commands. Mostly of these commands are just “send xyz back to the client”, but they can also include other things like “wait for x seconds”, or “close the connection”.

There is a bit more cleverness going on (for example, MockServer can fake the second connection that an FTP transfer requires), but essentially that’s it.

Responders

As I mentioned above, for the purposes of testing, most server interactions are just a series of query, response, query, response.

MockServer models this query/response behaviour using an abstract KMSResponder class - an instance of which you give to the server object at startup. So depending on the actual responder class, you can pretend to be any kind of server you like.

In real life some servers rely on a lot of state in order to give the correct response to a given query. In theory a responder could keep any state it liked, to deal with this problem. You could even genuinely implement an actual server protocol as a responder class if you were mad enough.

However, for testing this often isn’t necessary. MockServer supplies the KMSRegExResponder class, which is based on performing regular expression pattern matching on the input that it receives, and choosing from a series of stock responses.

It turns out that you can cover an awful lot of ground with this approach alone. Especially when you add in the ability to perform some limited substitution into the stock responses - things like inserting the current time in the correct format, or including part of the query you matched in your reply.

Responses

So now the meat of testing a particular network interaction comes down to identifying the queries that are going to arrive at the server, and arranging the right responses to send back.

This is where some low level knowledge of the underlying network protocol becomes useful, but even here it turns out that a lot of work can be done by trial and error.

You can configure MockServer so that it will log out everything it receives, so when it comes to figuring out the query, you can literally watch to see what arrives, and then write a pattern to match it.

Working out what to send back is slightly harder, as you can’t just watch MockServer, you have to actually do something yourself. The trick here though is to watch a real server, either by sniffing a network connection with it, or by using a tool such as Transmit and examining the log, or even by opening up a telnet session with the real server and pretending to be the client.

The good news is that a lot of this work only needs to be done once or twice for each kind of server.

You ask yourself “how does an FTP client send the password, and what does the server do?”.

Once you know that the client sends PASS thisismypass, and the server responds with 230 User user logged in. or 530 Login incorrect., you can just set up these a couple of alternate responses to match against a pattern like PASS (\w+). Normally you’ll want the first response, but if you want to test your error handling for the wrong password, you use the second one instead.

To support this sort of thing, MockServer provides the KMSResponseCollection class, which lets you define a series of patterns and responses in a JSON file, and then group them into higher level sets of responses.

So far in the course of my own tests, I’ve developed a couple of the JSON files - one for FTP and one for WebDAV, with quite a few useful responses in them. These files are included with MockServer, and can be used as the basis for your own tests. As time goes by I expect that I’ll add more response files for other protocols.

An Example Test

This is a simple example of how you might set up an FTP test, using the FTP responses supplied with MockServer.

#import "KMSTestCase.h"
#import "KMSServer.h"

@interface ExampleTest : KMSTestCase
@end

@implementation ExampleTest

- (void)testFTP
{
    // setup a KMSServer object, using the ftp: scheme and taking the "default" set of responses from the "ftp.json" file.
    if ([self setupServerWithScheme:@"ftp" responses:@"ftp"])
    {
        // set up the data that the server will return
        NSString* testData = @"This is some test data";
        self.server.data = [testData dataUsingEncoding:NSUTF8StringEncoding];

        // setup an ftp request for an imaginary file called "test.txt", on the local port that
		// our mock server is running on
        NSURL* url = [NSURL URLWithString:[NSString stringWithFormat:@"ftp://user:pass@127.0.0.1:%ld/test.txt", (long)self.server.port]];
        NSURLRequest* request = [NSURLRequest requestWithURL:url];

        // perform the request using NSURLConnection
		__block NSString* string = nil;
		[NSURLConnection sendAsynchronousRequest:request queue:[NSOperationQueue currentQueue] completionHandler:^(NSURLResponse* response, NSData* data, NSError* error)
		 {
		     if (error)
		     {
		         STFail(@"got unexpected error %@", error);
		     }
		     else
		     {
		         string = [[NSString alloc] initWithData:data encoding:NSUTF8StringEncoding];
		     }

		     [self pause]; // this call causes runUntilPaused to return
		 }];

		[self runUntilPaused]; // we hang here until the completion block above has executed

        // check that we got back what we were expecting
        STAssertEqualObjects(string, testData, @"got the wrong response: %@", string);
    }
}

Obviously not all tests will use FTP, and not all tests will use NSURLConnection. Some networking APIs may be callback based, or require delegates. As long as you call [self pause] in your callback or delegate method, the test should behave correctly - waiting at the runUntilPaused call until the callback or delegate method has fired.

For simplicity in this test we encoded the user name and password into the request URL. More complicated examples would probably rely on some sort of NSURLCredential callback or delegate method (such as implementing connection:willSendRequestForAuthenticationChallenge:), as a way of obtaining the details to send to the server. The KMSTestCase class has handy user and password properties that you can use to store the details that your authentication method can pass back. This lets you set this information on a per-test or per-request basis, as a way of exercising the different code paths for authentication success and failure.

For More Information

This post is just a quick overview, but there is also a fair bit of documentation which you can find over on github.

Also, of course, you can look at the code.

I wrote this code for Karelia as part of some other work. Many thanks (and much kudos) to them for allowing me to open source it.

Still Thinking Different - ARM Based Mac Pro?

2012-11-06T00:00:00+00:00

In my Mac Pro Update - Think Different post a few weeks ago, I said that I thought we don’t really need a great big new Mac Pro from Apple.

What we need instead (and I admit that I don’t have an exact technical solution for this problem, but nevertheless) is a unit or units that can seemlessly enhance the processing power of a laptop when it’s on our home network.

You could argue that there’s still a need for a big box that can host all sorts of large expansion cards, but I’m not really convinced. Thunderbolt should deal with most of that.

Something else has occurred to me recently.

People have been talking about a potential switch away from Intel to ARM for the Mac range, and assuming that it would happen first with a Mac Mini or an iMac, since they are seen as “consumer”, and there’s a perception that consumer machines will be easier to switch over.

I actually think that the opposite will happen. I think that the first ARM Mac is likely to come in the guise of a new “Pro” desktop box. I also think that it’s likely to be loaded to the gills with cores, to the extent that it blows away anything out there in terms of raw processing power.

With GCD / blocks / NSOperation, Cocoa has a really nice clean model for making use of multiple cores. Better than anything I’ve used before (although admittedly I’m now a bit out of date with regard to the competition).

With the expansion of privelege separation and things like XPC, we’re also moving closer to a world where even a single app could be run seamlessly across multiple machines.

It seems to me that switching to a relatively low power, low cost, risc architecture makes perfect sense in this world.

Before you ask - I’ve absolutely no evidence for this. Just thinking aloud…

NSError and Paranoia

2012-10-05T00:00:00+00:00

Simon Wolf wrote a blog post recently talking about passing NSError pointers into methods like this:

NSError* error = nil;
[something someMethod:@"blah" error:&error];

He talked about the fact that when you do this, you should always check the result of the method before using the error. In other words, you do this:

NSError* error = nil;
if (![something someMethod:@"blah" error:&error])
{
	NSLog(@"got error %@", error);
}

and not this:

NSError* error = nil;
[something someMethod:@"blah" error:&error];
if (error)
{
	NSLog(@"got error %@", error);
}

This is good advice. The method you’re calling isn’t guaranteed to set error to a non-nil value if nothing goes wrong. It might leave it alone, but it might set it to a temporary value, even though it eventually returns an “ok” result.

However, Simon then went on to talk about this line at the top:

NSError* error = nil;

His assertion was that by setting error to nil, you’re indicating that you intend to test it after the call - in other words that it’s an indicator that you don’t know what you’re doing.

He suggested that you should just do this:

NSError* error;

Strictly speaking, this is correct. If the method you’re calling follows the implicit contract for such functions, this should be fine.

However, I always set the error to nil before I pass it in, even though I never test it after the call. Why do I do this?

Consider the following method:

- (BOOL)someMethod:(NSString*)string error:(NSError**)error
{
	if (error)
	{
		if (something)
			*error = [self doSomeStuff];
		else (somethingelse)
			*error = [self doSomethingElse];
		//... lots of other cases here...
		
		//... later
		
		if ([(*error) code] == someCode)
		{
			[self doSomeExtraStuff];
		}
	}
	
	return YES;
}

This is a bit of a contrived example, but the point is that this method might be the one you’re calling, and it might be in a third party library that you have no control over.

Now lets imagine that some refactoring or other code change introduces a bug where *error doesn’t get set in the first block of if/else statements. Let’s also imagine that the developer has the relevant warnings turned off, and doesn’t noticed (shock horror - some people actually do this!).

What happens when the code starts accesses *error? Well, it depends what’s in it. If you did this:

NSError* error = nil;

you’ll be fine.

If you did this:

NSError* error;

you get undefined behaviour, based on what happened to be on the stack in the position that the error variable occupies. It’s a classic uninitialised variable bug, and potentially a bastard to track down.

At this point you may well be saying “if the quality of the library you’re using is that bad, you’ve only yourself to blame”. You might have a point, but bugs do creep in, even to good libraries.

Admittedly too, when you’re accessing an uninitialised Objective-C pointer, you’re way more likely to crash straight away than you would have been if you were just dereferencing a pointer to read a member in C/C++.

However, all of this is the reason why I still do:

NSError* error = nil;

even though I’m not going to test error afterwards.

You could call it paranoia, but don’t mistake it for misunderstanding the NSError** contract!

Another Day, Another Beta: Ambientweet 1.1.2b6

2012-08-28T00:00:00+00:00

So, yesterday’s 1.1.2b5 release didn’t quite go according to plan, as it contained a rather serious bug which corrupted it’s store of cached information. The result of this was that when you relaunched it, Ambientweet encountered problems and generally ended up in a non functional state.

Luckily, I can now bring you 1.1.2b6, now with added not-so-many-bugs-honest.

Since it’s only been a day and people may not have read the previous post, it’s worth re-iterating the changes in these new versions, so here’s an edited version of yesterday’s message:

I’ve been working on some improvements to Ambientweet, for the next version, which I’m currently calling 1.1.2 (although it’s getting to the point where there are enough changes to warrant calling it 1.2).

Internally, the code that manages the communication with Twitter has been modified a fair bit. This was partly a case of tidying up some old code, and partly improving the design to make it easier to support multiple Twitter accounts at once.

I’ve not quite got as far as multiple Twitter accounts in this revision, but I have added support for multiple windows. This means that you can set up more than one Ambientweet window if you wish, and have them track different timelines. One could track your normal timeline, one could focus on tweets from a single user, and one could show the results of a search for some text or a hashtag.

Because you can now have different windows, I’ve removed the window related settings from the preferences, and instead added a “Window Options” menu to the “View” menu, so that you can set them individually for each window.

The other major change that has happened relates to the automatic update mechanism. Unfortunately this got broken in version 1.1.1, and as a result automatic updating may not work, particularly if you are running Mountain Lion. These problems should now be fixed, but if you’re running one of the versions where updating was broken, then unfortunately you’ll have to manually get yourself past this version by downloading the latest beta.

Other than that, there are a couple of minor fixes:

The preview display which says what tweets are being downloaded (before there are any to display) should now make a bit more sense, and shouldn’t show an ugly looking internal description for a user any more.
The preference for centring the window when it shows some sort of sheet (eg when you’re asked to enter the text to search for) now actually does something.

If you are feeling brave, you can download the latest version here.

Ambientweet 1.1.2b5

2012-08-27T00:00:00+00:00

Other than that, there are a couple of minor fixes:

The preview display which says what tweets are being downloaded (before there are any to display) should now make a bit more sense, and shouldn’t show an ugly looking internal description for a user any more.
The preference for centring the window when it shows some sort of sheet (eg when you’re asked to enter the text to search for) now actually does something.

If you are feeling brave, you can download the latest version here.

Moving Git Repos Containing Submodules

2012-08-19T00:00:00+00:00

If you work with submodules in git, and you’ve ever tried to move a repository locally to a different place on your machine, you may have encountered a problem.

In recent versions of git, the embedded submodules don’t have their own “.git” directory. Instead they contain a text file called .git which points git back at the root .git directory, which contains all the information for all submodules. Furthermore, there’s a config file for each submodule, hidden in the main .git directory, which points “forward” to the submodule.

That would all be fine except for one incredibly stupid thing: in versions of git prior to 1.7.10, this path was stored in absolute format.

Which means that if you move the repo on your disk, all of these paths break, and the repo no longer works!

This is, to put it mildly, a bit of a pain in the arse.

What to do?

The long answer is to go through all of the submodules and do the following:

hand edit the gitdir: entry in the .git file to make it relative
hand edit the config file in the directory that the gitdir entry points to, to make the worktree entry relative

That’s all fine and dandy, but it’s a tricky process, and if you’ve got lots of submodules, some of which may even have embedded submodules, it’s a lot of work.

Luckily, there’s a short answer:

do a search and replace across the .git and config files mentioned above, and replace the old broken absolute path with a new correct absolute path.

This isn’t ideal, but it gets you working again.

What’s more, it’s a lot easier to automate. Figuring out the proper relative paths isn’t that easy to automate, but doing a search and replace of one known string with another across a bunch of files is.

Here’s a script I wrote to do it.

It’s not a perfect script, and please be aware that it makes permanent changes to files so you may well want to zip up the whole of your repo first as a paranoid backup.

However, it seems to work for me. It could take the old and new paths as parameters, but I decided to embed them in the script for a couple of reasons.

One, it’s a better example of what the paths should look like. Two, this situation is most likely to occur when you’ve made some sort of global change, like renaming or changing your hard drive. In that case you’ll probably want to do the same replacement lots of times on different repos, so embedded it in the script is helpful (and also reduces the risk of typing it wrong).

Organising Objective C Libraries

2012-08-11T00:00:00+00:00

I asked this question on Stack Overflow earlier, but I think it’s worth posting basically the same thing here too.

I’ve been building libraries, and collections of layered libraries, for a long time, and I’m still not totally happy that I’ve found the best way to organise them. The question I asked was aimed at soliciting some insights from others.

Here it is:

Let’s say that I’ve three libraries, A, B, and C.

Library B uses library A. Library C uses library A.

I want people to be able to use A, B, and C together, or to just take any combination of A, B, and C if they wish.

I don’t really want to distribute them together in one large monolithic lump.

Apart from the sheer issue of bulk, there’s a good reason that I don’t want to do this. Let’s say that B has an external dependency on some other library that it’s designed to work with. I don’t want to force someone who just wants to use C to have to link in that other library, just because B uses it. So lumping together A, B and C in one package wouldn’t be good.

I want to make it easy for someone who just wants C, to grab C and know that they’ve got everything they need to work with it.

What are the best ways of dealing with this, given:

the language in question is Objective-c
my preferred delivery mechanism is one or more frameworks (but I’ll consider other options)
my preferred hosting mechanism is git / github
I’d rather not require a package manager

This seems like a relatively straightforward question, but before you dive in and say so, can I suggest that it’s actually quite subtle. To illustrate, here are some possible, and possibly flawed, solutions.

CONTAINMENT / SUBMODULES

The fact that B and C use A suggests that they should probably contain A. That’s easy enough to achieve with git submodules. But then of course the person using both B and C in their own project ends up with two copies of A. If their code wants to use A as well, which one does it use? What if B and C contain slightly different revisions of A?

RELATIVE LOCATION

An alternative is set up B and C so that they expect a copy of A to exist in some known location relative to B and C. For example in the same containing folder as B and C.

Like this:

libs/
  libA/
  libB/ -- expects A to live in ../
  libC/ -- expects A to live in ../

This sounds good, but it fails the “let people grab C and have everything” test. Grabbing C in itself isn’t sufficient, you also have to grab A and arrange for it to be in the correct place.

This is a pain - you even have to do this yourself if you want to set up automated tests, for example - but worse than that, which version of A? You can only test C against a given version of A, so when you release it into the wild, how do you ensure that other people can get that version. What if B and C need different versions?

IMPLICIT REQUIREMENT

This is a variation on the above “relative location” - the only difference being that you don’t set C’s project up to expect A to be in a given relative location, you just set it up to expect it to be in the search paths somewhere.

This is possible, particularly using workspaces in Xcode. If your project for C expects to be added to a workspace that also has A added to it, you can arrange things so that C can find A.

This doesn’t address any of the problems of the “relative location” solution though. You can’t even ship C with an example workspace, unless that workspace makes an assumption about the relative location of A!

LAYERED SUBMODULES

A variation on the solutions above is as follows:

A, B and C all live in their own repos
you make public “integration” repos (lets call them BI and CI) which arrange things nicely so that you can build and test (or use) B or C.

So CI might contain:

- C.xcworksheet
- modules/
    - A (submodule)
    - C (submodule)

This is looking a bit better. If someone just wants to use C, they can grab CI and have everything.

They will get the correct versions, thanks to them being submodules. When you publish a new version of CI you’ll implicitly be saying “this version of C works with this version of A”. Well, hopefully, assuming you’ve tested it.

The person using CI will get a workspace to build/test with. The CI repo can even contain sample code, example projects, and so on.

However, someone wanting to use B and C together still has a problem. If they just take BI and CI they’ll end up with two copies of A. Which might clash.

LAYERED SUBMODULES IN VARIOUS COMBINATIONS

The problem above isn’t insurmountable though.

You could provide a BCI repo which looks like this:

- BC.xcworkspace
- modules/
    - A (submodule)
    - B (submodule)
    - C (submodule)

Now you’re saying “if you want to use B and C together”, here’s a distribution that I know works.

This is all sounding good, but it’s getting a bit hard to maintain. I’m now potentially having to maintain, and push, various combinations of the following repos: A, B, C, BI, CI, BCI.

We’re only talking about three libraries so far. This is a real problem for me, but in the real world potentially I have about ten. That’s gotta hurt.

So, my question to you is:

What would you do?
Is there a better way?
Do I just have to accept that the choice between small modules and a big monolithic framework is a tradeoff between better flexibility for the users of the module, and more work for the maintainer?

Asking Stupid Questions

2012-08-11T00:00:00+00:00

Often, when I’m contracting for someone, I find myself asking them lots of things.

Sometimes I do it formally by writing a spec, and saying “this is what you want, right?”. Other times it’s just a conversation.

Then, equally often, I find myself apologising for asking so many “stupid questions”.

Then I tell myself off for being insecure and apologising. We all like to feel wise, and it can be hard to admit ignorance, especially if you’re in some sense selling yourself as a person who is supposed to know what they’re doing.

It’s rare, though, for it to be a bad idea to ask a question.

I particularly like asking questions to which I think I already know the answer. It’s instructive how often it turns out that I don’t.

So much of what we do as programmers is reliant on good communication, yet we’re so often not very good at it. A few well placed questions at the beginning of a task can save a world of pain.

Once in a blue moon, you’ll come across a client who reacts to this the wrong way.

They wonder why you’re so fixated on some details that they think are trivial, or why you’re asking them about stuff that they didn’t ask you to do*.

Or they just interpret it as vagueness or ignorance.

If that happens, you probably don’t want that client. Honestly, it won’t go well.

[ *hint: they were so vague that you thought they might be asking you to do it, or you can see that you’d need to do it, in order to do whatever it is that you think they probably do want you to do ]

Gemmell on managing email

2012-08-06T00:00:00+00:00

I like this post by Matt Gemmell on how to manage your email.

He describes pretty much what I’ve been doing for the last few years.

I’m not as good as I should be about replying quickly to the people who do matter, but I’m considerably better than I used to be since I started being more ruthless about the others.

The only thing I’d add to Matt’s suggestions is to do the following:

make a group in Address Book / Contacts
add the important people to it
make a new Smart Mailbox that just shows mail from people in this group
change Mail’s badge to show the unread count from this mailbox
change Mail’s notification sound to only fire for this mailbox
reduce the frequency that it checks mail - does it really matter if you don’t get it for another half an hour?

Restoring iChat Logs in Mountain Lion

2012-07-27T00:00:00+00:00

In Mountain Lion, the new Messages app replaces iChat.

If you’re the kind of person like me who religiously keeps your email, chat logs, etc, but you have chosen not to do a migration of your data when upgrading to Mountain Lion, you’re going to be faced with the problem of how to import your old iChat logs into Messages.

It turns out that Messages stores logs in:

~/Library/Messages/Archive

They appear to be the same format as iChat, so if you move your old logs into this folder, Messages will find them.

Alternatively, you can make a symbolic link to a folder elsewhere and put your logs in there. For example, this will rename the old Archive as “Archive.old”, then make a link to a folder called “iChat” in your Dropbox folder.

 > cd ~/Library/Messages
 > mv Archive Archive.old
 > mkdir -p ../../Dropbox/iChat
 > ln -s ../../Dropbox/iChat Archive

This just leaves the Archive.old folder lying around, but you could obviously move its contents into your new iChat folder then delete Archive.old.

HebCelt festival? There's an app for that.

2012-07-10T00:00:00+00:00

We’ve been pretty busy in recent months, with a number of contracting jobs, including a bit of work with the lovely folks over at Karelia software.

Live music is dear to our hearts though, so we’re planning on taking some time out this week to attend the excellent Hebridean Celtic festival, which happens each year here in Stornoway, on the Scottish island of Lewis in the Outer Hebrides.

The festival is very much a labour of love, and although it attracts a large audience from all over the world, it’s run by a charitable trust. So when they came to me to ask about the possibility of doing an app, I was rather keen to get involved (although a little scared at how little time we had to get something ready for this year).

After a bit of nifty footwork, and a few anxious nights waiting for Apple’s review process, the result is this little app. Given the amount of time we had, we decided to keep it simple, but we’re rather pleased with the results. Hopefully, the festival organisers are too.

Preparing for Mountain Lion

2012-06-27T00:00:00+00:00

Apple are due to release OS X Mountain Lion soon.

One of the features included in it is something called Gatekeeper, which is intended to improve security. By default, when you install Mountain Lion, you will only be able to run newly downloaded applications if they have been signed by the developer, using an official Apple developer identity (note that you can turn this behaviour off, but that’s the default setting).

What this means in practise for you, with regard to Neu and/or Ambientweet?

If you’re using copies of Neu or Ambientweet downloaded from the Apple store, there will be no change.

In any event, there will be no change if you continue to use copies that you’ve already downloaded (ie on another machine or with a previous version of the system).

If you download the current version of Ambientweet (1.1) or Neu (1.2) from our website, they are currently not signed in the correct way, so Gatekeeper will complain about them. There is a way that you can tell Gatekeeper to make an exception, but that’s not really an ideal solution in the long term, so we’re preparing a quick update to bring Ambientweet up to version 1.1.1, and Neu to 1.2.1.

The only change for these versions is that they will be signed correctly and Gatekeeper will recognise them as legitimate.

If you want to test beta versions of these applications, they should be available later today from the beta software page. Barring any reported problems, they’ll then go live in the next few days.

Since the only purpose of these updates is to prepare for Gatekeeper, and since Gatekeeper already accepts applications downloaded from the Apple store, these updates will not be submitted to the store. If you’re using versions from the store, you can safely continue with versions 1.1 or 1.2 of Ambientweet and Neu respectively.

Proper updates, with more features, will arrive in due course, bringing the store and non-store versions back into line again.

Mac Pro Update? Think Different...

2012-06-16T00:00:00+00:00

My first desktop Mac was a IIcx, bought in about 1989, for the astounding sum of nearly £4000 (to put that in context, it took me a year of working between school and university to pay for it, and I could have bought a new Ford Fiesta for less).

Since then, as a Mac developer, I’ve owned and used many more, pretty much always the top spec of the current generation. There was certainly a time when having the latest, greatest desktop machine was essential (not to mention highly desirable for the sheer geeky joy of it).

However, since going back to indy development in 2010, I’ve only worked on my laptop, and I didn’t have a desktop machine at home for a good few years prior to that, despite regularly working from home on a big game with masses of data.

So the recent speculation leading up to WWDC about whether the Mac Pro would get an update, and the subsequent answer (“not really, well maybe slightly, but there’s something new coming next year”), lead me to ponder what I’d actually want. The four things that I can see a desktop might offer me are:

more power (processing and memory)
more storage
more screen real estate
the ability add custom hardware

The conclusion I came to is that with all of these things, what I actually want isn’t really a desktop machine at all.

Conceptually what I want is one or more black boxes (well, maybe shiny aluminium boxes), that I can plug into my laptop (or perhaps hang on the network), which expand its capabilities.

In performance terms, these boxes should behave as near as possible as if they were inside the laptop, hanging directly off the main system bus, but other than that, they really don’t have to be housed in one big lump under my desk.

Whether Thunderbolt is up to the sort of load this would impose, I don’t know. For storage, I suspect that it is, and solutions are already there.

To some extent screen real estate may be too, although I’m not sure what sort of external monitor the new retina Mac Book Pro will be able to drive in addition to it’s own monster display.

If it were possible to house additional video hardware at the other end of a Thunderbolt cable (maybe it is?), so that one could effectively plug in additional video cards via the cable, this would pretty much remove the need for a desktop Mac as a way to drive a silly numbers of monitors.

Similarly, custom hardware, encoders and input cards, and so on, can and should probably live at the other end of a Thunderbolt cable.

All of which just leaves processing power and memory.

Given the bandwidth requirements of both, it’s clearly unlikely to be possible to add these directly to a laptop as if they were on the main logic board, but I’d love to see Apple try to get as close as it could to that.

Wouldn’t it be great to have a Mac-Mini / Airport Extreme sized box that required little or no configuration and just made your laptop more powerful when you plugged it in?

To some extent, things like distributed compilation offer this already. We used to have a pretty excellent setup for the Mac when I worked at SI, distributing to a large number of high end Mac Pros, and it really improved the build times. I think that got a bit broken with Xcode 4 though, and since then I’ve tried it at home with limited results, but only attempting to distribute to a machine with pretty much the same power as my laptop.

Given the increasingly distributed and task oriented nature of the code we all write these days, an approach which we need anyway to take advantage of the multiple cores already in our machines, it seems like some sort of generic solution allowing us to scale up the power of a machine ought to be a lot more achievable now than it was in the past.

Data is the big problem - getting the right stuff to the right processor in a timely fashion is tricky (just ask anyone who’s ever written code for a PS3) - but it’s not insurmountable.

Abstractions like GCD and NSOperation are probably a bit too fine grained to be transparently parcelled out to an external processor, but if we had another abstraction for encapsulating slightly larger chunks of work along with the data needed to do them, we’d have a chance of making this sort of thing fly. Things like XPC could really help here, allowing whole abstract services to run without the client process having to know or care where.

Nothing I’m talking about is revolutionary of course, but packing it up in a really clean, modular, elegant way that can be used by pro-consumers would be. And who better to do it than Apple?

Apple Bug Reporting: An Illustration Of (some of) The Problem(s)

2012-06-13T00:00:00+00:00

Last November I encountered an issue in Xcode whilst attempting a submission.

I was getting an error due to the fact that some embedded bundles in the application that I was submitting were signed with the wrong identifiers.

Xcode, rather unhelpfully, reported this:

“the nested app bundle ECFoundation (Ambientweet.app/Contents/Frameworks/ECFoundation.framework) is not signed, the signature is invalid, or it is not signed with an Apple submission certificate. Refer to the Code Signing and Application Sandboxing Guide for more information.”

I submitted a Radar report, pointing out that whilst having an error message is helpful, it would be even better if it narrowed down the actual cause, rather than suggesting three possibilities and leaving you hanging.

Last week, I got the first response to this report:

“We believe that this issue has been resolved through changes on our side.

Please let us know whether the issue is resolved for you.”

Now first of all, let me say that I appreciate getting the response.

The first sentence is really helpful, as it sets my expectations - if I use the latest Xcode, it’s probably fixed.

The second sentence is a little irksome. If they’d just said “Please let us know if you notice any problems”, I’d be fine with it.

Even as it is, I do realise that it’s probably just a stock response, and I shouldn’t read too much into it, but it does slightly irritate me.

Think about it for a minute. I had an issue seven and a half months ago. It was an issue with submission. Am I really likely to still have something that exhibits that issue?

“What about source control?” I hear you cry (that was you, wasn’t it?). And yes, you’re right, I could wind back in git and get the project into the state it was in then. But hang on, it was a problem with submission. So I’d need to do a submission again. Now forgive me for being a wuss, but I’m not in a great hurry to submit a new version with seven-and-a-half-month-old code. I could probably do it - add a fake new version, hack around the old code to have the right version numbers, perform the submission, cancel it if it worked, and so on.

We’re starting to talk about a bit of an investment in time now though. Maybe only an hour or two, but it’s funny how these sorts of things have a habit of taking longer than you expect. Now I don’t fall into the trap of immediately equating any time spent doing anything in my life with the hourly rate I charge in my working life, but…

And thus we see some of the problems with the bug reporting dance:

the response is so late that the problem is no longer current
the response is so impersonal that I don’t quite know whether to interpret it as a courtesy, or a serious request for help
the response doesn’t appear to acknowledge the difficulty of following it up
incidentally, the response didn’t tell me which version fixes the problem

All of this leads to the feeling of futility / apathy that I’ve mentioned in previous posts.

I’m sure I could make a test case and satisfy myself that the problem is fixed. Maybe offer further feedback on the solution they’ve come up with. If I happen to encounter it again in passing, I will. If I’d had a personal email from someone on the Xcode team, I’m sure I’d do it right now, no matter how long it took (despite the fact that I wouldn’t be doing anything that they couldn’t do themselves), just because it’s nice to feel involved in the future of the tools that I use.

In the absence of that email though, going out of my way just feels, frankly, like it wouldn’t be worth the effort.

Ambientweet 1.1 now in the store

2012-05-05T00:00:00+00:00

I’m happy to say that Ambientweet 1.1 has been approved for release and is now in the store.

The main changes in this version are:

Added ability to view the results of a search.
Fixed bug where a message containing links could get truncated.
Fixed clipping of author info when it spills onto three lines.
Fixed auto-completion window popping up again straight away after a completion.
Improved tweet caching so that mentions are always kept - this stops the badge being shown again for mentions that have already been seen.
Added a “psuedo tweet” which is a reminder that you’re running a pre-release or unlicensed version of Ambientweet (if you are). It is inserted into the tweet stream every now and then.
Improved application security.

You can find more details, and download links, on the Ambientweet home page.

Ambientweet 1.1b5

2012-04-27T00:00:00+00:00

I’ve just found and squished another little Ambientweet bug, which could occasionally cause problems when posting a tweet with a URL in it.

Consequently, there’s a new beta available - 1.1b5. Unfortunately a new beta also means resetting the clock on the Apple store review process, so we’ll just have to cross our fingers and hope it doesn’t take too long…

Birthdays not showing in Lion iCal

2012-04-19T00:00:00+00:00

I just had a weird glitch where I realised that iCal wasn’t showing the birthdays calendar, even though the relevant preference was turned on.

I looked on iCloud, and sure enough the birthdays were showing up there - proving that I hadn’t somehow lost the data from my address book.

In the end I found a simple solution:

Open iCal preferences
Turn off “Show Birthdays calendar”
Quit
Launch iCal again
Open iCal preferences
Turn on “Show Birthdays calendar”

Lo and behold, back came the calendar, and the events showed up as they should.

Go figure…

Ambientweet 1.1b4

2012-04-15T00:00:00+00:00

Hopefully this is the final beta of 1.1.

I’ve now turned on Apple’s sandboxing, in preparation for submitting it to the Mac App store.

Sandboxing makes it run in a slightly more secure environment, though hopefully users will not notice the difference.

Pick up a copy of the latest beta from the beta software page…

Ambientweet 1.1b3

2012-04-15T00:00:00+00:00

And hot on the heels of 1.1b2, here’s another beta.

This one shows a reminder “tweet” every now and then if you’re running a beta version, or a demo version (this is in preparation for finally charging something for the App Store version, at which point there really needs to be some difference between it and the demo!).

Grab the new version from the beta software page.

Ambientweet 1.1b2

2012-04-14T00:00:00+00:00

Another beta, with a couple of bug fixes.

The auto-completion when posting new tweets now behaves a bit more sensibly and doesn’t keep popping up when you’ve just done a completion.
The mentions badge should be a bit saner now, tweets that are mentions get cached properly so it should always know if incoming mentions are actually new or not

Grab the new version from the beta software page.

New Ambientweet Beta 1.1b1

2012-04-10T00:00:00+00:00

I’ve been rather busy recently with all sorts of contracting work, but I’m glad to announce that in between it all I’ve been slowly working away on additions to both Ambientweet and Neu.

First up for public preview is Ambientweet 1.1b1.

The main thing that’s new in this release is the ability to focus Ambientweet on a search term and have it cycle through tweets that contain it.

This opens up a pretty handy new use for Ambientweet - if there’s a term trending or a topic that you particularly want to follow, you can set Ambientweet up to display just those tweets (perhaps whilst you continue to view your own stream with another client).

Feel free to give the beta a try….

NSConference 2012

2012-03-21T00:00:00+00:00

So, NSConference 2012 has finished. I wanted it to go on for another week, but alas it was impossible, and that many cooked breakfasts probably wouldn’t have been advisable.

I won’t go for a full on summary of the whole thing - a few other folks have been doing a good job of that.

Here’s a quick take on my initial thoughts though. After further reflection I may have more, or may revise these ones.

The Good

Seeing old friends
Making new friends
Some great inspirational talks
Lots of valuable Nuggets Of Information (tm)
Tom bringing me my favourite beer

The Un-Good*

Not enough technical content for me this year
Some (undeniably brilliant) speakers essentially doing the same talk as last year
Quality of the blitz talks varied wildly, and I think I preferred them being elsewhere
The conference clearly needs more QR codes

(*this may be a bad translation)

Overall though - another fantastic experience. Many thanks to Scotty and the whole team for looking after us all so well.

Radar? GTFO!

2012-03-02T00:00:00+00:00

Daniel Pasco just blogged an article named Radar Or GTFO.

The gist of his argument is that it’s all very well moaning about Xcode 4, but unless you file Radar bugs on it, you don’t have a leg to stand on.

Since I’ve recently been moaning about Xcode 4, I have a view on this. Not that I have anything against Daniel, but from the title of this blog post, you can probably guess what it is.

Funnily enough, a few weeks ago, I was also moaning about Radar.

You can read the post if you want the full details, but my basic point about Radar is that it’s a crock of shit for anyone outside of Apple. It’s an impenetrable black hole - bugs go in, but little or nothing ever comes out. It takes a lot of time to make a well formed bug report, yet Apple won’t even let you know whether or not they know about an issue unless you go through all of that effort.

If you are lucky enough to have someone publicise a radar number that you can file a duplicate on, you can avoid some of this work, but even then it’s far harder that it ought to be to just post a “me too” report. You can’t even see the content of the bug report so you have to take someone’s word that it’s actually the issue you are complaining about (Open Radar is a little help in this, but it’s run by us, not by Apple, it’s entirely voluntary and therefore very incomplete, and using it is potentially even more work).

I have every sympathy for Michael Jurewitz (developer tools evangelist at Apple), and the Xcode team - it’s not their fault that they are stuck behind The Great Wall of Cupertino. Unless Apple give some serious love to the bug reporting process though, it is they who don’t have a leg to stand on.

Coincidentally, John Gruber made a tangental but highly relevant post on this the other day.

Apple’s resorting to moaning about a lack of bugs filed in Radar is either them being deliberately obtuse, or it exposes a fundamental misunderstanding of human psychology (something that Apple usually can’t be accused of).

People do things (or fail to do things) for a reason. If we’re not filing bugs, it’s because the bug reporting process is fundamentally broken. Fix that, and we’ll drown Apple in bug reports.

Then maybe I can have a development environment that doesn’t crash on me at least once per hour.

What The Hell Happened To Xcode?

2012-02-27T00:00:00+00:00

I’ve been using Xcode for a good long time (since before it was called Xcode).

It has many detractors, and though I understand why people get frustrated with it, on balance I’m not one of them these days. I say on balance because there are undoubtedly things about it that drive me mad. That’s has been equally true in the past though of Visual Studio, Eclipse, Metrowerks, Think C, etc.

I think a lot of the criticism of Xcode is unjust.

A lot of it comes from switchers who are actually saying “I don’t understand this because it doesn’t look like insert-my-favourite-IDE-here”. Well, no shit! Different it is, but different is not necessarily worse.

A lot of it also comes from the kind of programmers who don’t really understand build systems, don’t like the fact that making large software projects is complicated, and would really like it if someone else just made it all work. These people are probably happy with a simple project (maybe taken from a sample and hacked about a bit), but as soon as they have to make a change to a build setting, or work with a multi-target, multi-project setup, and something goes wrong, they get snarky and blame the IDE for being crap. These people would probably not be much happier with anything else, but for whatever reason they’re using Xcode, so they bitch about Xcode.

So, anyway, hopefully we’ve established that I quite like Xcode. Xcode 4, in particular, is a bit improvement in lots of ways over Xcode 3.

Except. Except.. ah..

WHAT THE FUCK HAPPENED TO XCODE 4.3?

On the face of it, not a lot has changed. Except for one really big thing, which is that almost every file that it uses has moved to a new location on the disc, inside the application bundle. But that shouldn’t affect us right? All we generally do is launch the application, and a few other tools, and some command line utilities, and - well, it shouldn’t affect us too badly, surely?

Well, you’d hope not, but you can see how it might flush out certain incorrect assumptions about where things are relative to each other.

Whether that’s the reason, or it’s something else, this version appears to be really, really crashtastic! It crashes when I do stuff. It crashes when I don’t do stuff (seriously, it crashes when I just leave my computer sometimes). It throws up random modal alerts with obscure error messages. It gets confused and refuses to build projects that it happily builds if you approach them from a different direction. Worst of all, it ignores previous settings about where I want my curly brackets god dammit!

It may be that I’m alone in this, and there’s something strange about my setup. I do admittedly have four different generations of Xcode installed on my machine to cope with the different requirements of various clients.

I doubt that’s it though. The word I’m hearing from all the other developers that I know suggests that my experience is pretty typical.

To me this raises some concerns about Apple’s internal Q&A for their tools department. How exactly did this thing end up going out in this state? The first beta of it was actually quite solid, but the last one and the actually release have been woefully unstable.

What concerns me more though is that Xcode is one great big monolithic lump of closed-source code. It may be modular under the hood (there’s clearly some sort of plug-in architecture), but it’s clearly not modular in a very rugged sense, since Apple only ever seem to want to distribute releases of the whole damn thing in one big lump. If that policy is based on the danger of instability if pieces are updated individually, that’s bad in what it hints about the true state of the code. It’s also bad because, frankly, that policy ain’t working.

To be fair, the decision to keep the releases monolithic may be based more on the complexity of supporting lots of versions of Xcode. If all the components were changing all the time that could get messy I grant you, but if that’s the case then they really really need to make sure that official releases are stable.

Because new releases don’t come along that often.

Betas aren’t that rare (indeed, I have one right now, which is even crashier than 4.3), but betas don’t always help. For every thing that is fixed, a couple of others are either broken or in a state of temporary flux.

I don’t have any simple solutions, but I really hope that this is a blip, since currently, working with Xcode is a profoundly unpleasant experience for me.

Actually, I do have a couple of solutions, but I doubt that Apple will go for them.

One is to open source the bastard. How I would love to be able to just dive in and fix the bugs that are biting me. If there’s one thing that really ought to make sense to open source, it’s a development environment, given the skill set of the user base!

Assuming that’s not going to happen, the next thing I’d really like to see is for Apple to split up the release cycles a little bit.

Fair enough maybe we can’t have a crazy free for all where I can download the latest beta of IDEGit.ideplugin and install it individually, but why on earth do I have to wait for iOS 5.1 to be released before someone will fix some bugs in Xcode’s text editor? It’s ridiculous. No reason at all, except for the monolithic release model.

It would be great if at least the release cycle of the editing environment, the compiler tool chain, the sdks and the ancillary tools were independent, so that the IDE could be updated far more frequently.

There are signs that they’re attempting to go that way, but I’d really like to see them pursue it more aggressively. In the meantime, I live in hope of a new, more stable, beta.

The Run Loop In Cocoa Unit Tests

2012-02-26T00:00:00+00:00

Following on from my post on parameterised unit tests, another little utility hidden away in ECUnitTests is some support for running the run loop in unit tests.

Why would you want to do that, I hear you ask?

The answer is: because you want to do something asynchronous in a unit test, which relies on the run loop. An example would be using NSURLConnection to download something.

The Problem

Each unit test typically runs as a single, discreet test method.

If you need to use something like NSURLConnection that relies on the run loop to post notifications or call delegate methods, then you have a problem doing so in a unit test method. The asynchronous stuff won’t run until after your test method has exited (if at all), at which point it’s too late to test the results.

The Solution

The solution is simply to set things up, call your asynchronous method, then run the run loop until something flags that it’s time to stop, and finally test your results.

This is actually very simple. All you need is to set up some sort of boolean variable which you can test to see if it’s time to exit the loop.

You then set it to false, and enter a loop like this:

exitRunLoop = NO;
while (!exitRunLoop)
{
    [[NSRunLoop currentRunLoop] runUntilDate:[NSDate dateWithTimeIntervalSinceNow:0.1]];
}

In order that you don’t get stuck in this loop forever, you need a way for one of your delegate methods or completion blocks to set the value of exitRunLoop to YES.

The easiest way to do this is to make exitRunLoop a property on the test class, and make sure that the test class is also the delegate (or accessible from the delegate or completion block).

To simplify things, the ECTestCase handles this for you, and provides two methods that you can call.

- (void)runUntilTimeToExit;
- (void)timeToExitRunLoop;

If you inherit from ECTestCase in your test class, you can simply call the first one of these when your test needs to wait for something to happen, and call the second one when the thing has happened and you want to stop waiting.

You can see an example of this in ECUnitTests.

Parameterised Unit Tests

2012-02-26T00:00:00+00:00

Normal Tests

If you’ve used the Objective-C unit testing framework (OCUnit, also know as SenTestKit) , or for that matter any other xUnit testing framework, you’ll be familiar with the basic way it goes:

define a class that inherits from SenTestCase
add some test methods which take no arguments, return no results, and have names prefixed by “test”
at runtime SenTest makes a test suite for each SenTestCase subclass it finds
it adds each test method that it finds to the suite
it then runs each suite in turn, running each test once, and reporting the results

Parameterised Tests

The normal tests work great if you want to run each test once, but what if you have a set of test data and you want to run each test multiple times, applying each item of test data to it in turn?

The naive approach is to define lots of test methods that just call onto another helper method supplying a different argument each time. Something like this:

- (void)testXWithDataA { [self helperX:@"A"]; } 
- (void)testXWithDataB { [self helperX:@"B"]; } 

That gets tired quickly, and it doesn’t allow for a dynamic amount of test data determined at runtime.

What you really want in this case is to add the following abilities to SenTest:

the ability to define parameterised test methods using a similar naming convention to the normal ones
the ability to define a class method which returns a dictionary of test data
have SenTest make a sub-suite for each parameterised method we found
have the sub-suite contain a test for each data item
iterate the suites and tests as usual, applying the relevant data item to each test in turn

SenTestKit is very flexible, but it’s also a bit obscure in the way it’s written, so it’s not immediately apparent how to achieve these goals. After a bit of investigation though it turns out to be pretty simple, and I’ve figured it out so that you don’t have to!

You can find the following classes as part of a small open source module I’ve created, called ECUnitTests. This module includes some other utilities too, but the thing I’m concentrating on in this blog post is the ECParameterisedTest class.

ECParameterisedTest: How To Use It

inherit from ECParameterisedTest instead of SenTestCase
define test methods which are named parameterisedTestXYZ instead of testXYZ (they still take no parameters)
either: define a class method called parameterizedTestData which returns a dictionary containing data
or: create a plist with the name of your test class, which will contain the data

At runtime the data method will be called to obtain the data. The names of each key should be single words which describe the data. The values can be anything you like - whatever the test methods are expecting.

To simplify the amount of modification to SenTest, the test methods still take no parameters. Instead, to obtain the test data, each test method uses the parameterisedTestDataItem property.

ECParameterisedTest: How It Works

To make its test suites, SenTestKit calls a class method called defaultTestSuite on each SenTestCase subclass that it finds.

The default version of this makes a suite based on finding methods called testXYZ, but it’s easy enough to do something else.

Here’s our version:

+ (id) defaultTestSuite
{
    SenTestSuite* result = nil;
    NSDictionary* data = [self parameterisedTestData];
    if (data)
    {
        result = [[SenTestSuite alloc] initWithName:NSStringFromClass(self)];
        unsigned int methodCount;
        Method* methods = class_copyMethodList([self class], &methodCount);
        for (NSUInteger n = 0; n < methodCount; ++n)
        {
            SEL selector = method_getName(methods[n]);
            NSString* name = NSStringFromSelector(selector);
            if ([name rangeOfString:@"parameterisedTest"].location == 0)
            {
                SenTestSuite* subSuite = [[SenTestSuite alloc] initWithName:name];
                for (NSString* testName in data)
                {
                    NSDictionary* testData = [data objectForKey:testName];
                    [subSuite addTest:[self testCaseWithSelector:selector param:testData name:testName]];
                }
                [result addTest:subSuite];
                [subSuite release];
            }
        }
        
    }

    return [result autorelease];
}

Most of this is self explanatory, but some key things to note are:

parameterisedTestData is a class method which returns a dictionary containing the data
the method as a whole returns one SenTestSuite object
that SenTestSuite object contains more SenTestSuite objects, which in turn contain the actual tests

To make things simple, we want to use the existing SenTestKit mechanism to invoke the test methods. Since SenTestKit expects test methods not to have any parameters, we need another way of passing the test data to each method. Each test invocation creates an instance of a our class, and we do this creation at the point we build the test suite, so the simple answer is just to add a property to the test class. We can set this property value when we make the test instance, and the test method can extract the data from the instance when it runs.

Obtaining Test Data

To obtain the test data, we’ve added a method parameterisedTestData that we expect the test class to implement.

This method returns a dictionary rather than an array, so that we can use the keys as test names, and the values as the actual data. Having names for the data is useful because of the way SenTestKit reports the results.

Typically it reports each test as [SuiteName testName], taking these names from the class and method. Since we’re going to use the name of the test method for each of our suites, we really need another name to use for each test. This is where the dictionary key comes in.

Where the test data comes from is of course up to you and the kind of tests you are trying to perform. There is a simple scenario though, which is that we want to load it from a plist that we provide along with the test class.

Since we need a default implementation of the method anyway, we can cater for this simple case automatically. We look for a plist with the same name as the test class. If we find it, we load it, and return the top level object from it (expecting it to be an NSDictionary).

+ (NSDictionary*) parameterisedTestData
{
    NSURL* plist = [[NSBundle bundleForClass:[self class]] URLForResource:NSStringFromClass([self class]) withExtension:@"plist"];
    NSDictionary* result = [NSDictionary dictionaryWithContentsOfURL:plist];
    
    return result;
}

An Example

Here’s a fully worked example.

###ExampleTests.m:

@interface ExampleTests : ECParameterisedTestCase
@end

@interface ExampleTests

- (void)parameterisedTestOne
{
	STFail(@"test one with data %@", self.parameterisedTestDataItem);
}

- (void)parameterisedTestTwo
{
	STFail(@"test two with data %@", self.parameterisedTestDataItem);
}

@end

###ExampleTests.plist:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
	<dict>
		<key>NameA</key>
		<string>ValueA</string>

		<key>NameB</key>
		<string>ValueB</string>
	</dict>
</plist>

When run, this should produce something like the following log output (simplified for clarity here):

Run test suite ExampleTests
  Run test suite parameterisedTestOne
    test NameA failed: test one with data ValueA
    test NameB failed: test one with data ValueB
  Run test suite parameterisedTestTwo
    test NameA failed: test two with data ValueA
    test NameB failed: test two with data ValueB

Git Repository Migration

2012-02-24T00:00:00+00:00

I’m migrating some of my git repositories over from github.com/samdeane to github.com/elegantchaos.

Unfortunately that means a change of URL, and there’s no automatic redirection in place to ease this transition.

I’ll try to edit any URLs on this website to point at the new locations, but apologies if I get it wrong.

ECLogging now standalone

2012-02-23T00:00:00+00:00

A while ago I blogged about an Objective-C logging system I’ve developed, called ECLogging.

Previously it was available on github as part of my ECFoundation framework, but recently I’ve been going through ECFoundation and splitting it up into modules, with each one stored as a separate git repository.

Some of these split-out modules still depend on each other, but luckily ECLogging isn’t one of them, which means that you can now adopt it a bit more easily, since you don’t have to take any other baggage with it.

You can find it on github. You can also find a wiki on there with some documentation in it. The github repo also contains a couple of example projects to help you get started.

Using NSSocketPort with AF_UNIX for on demand IPC

2011-12-22T00:00:00+00:00

In my recent post on OS X Helper Applications, I provided some sample code for installing and launching a helper application.

The host application and the helper communicated using distributed objects (NSConnection), running over mach ports (NSMachPort).

At the time, I was aware of the advice in Apple Tech Note 2083: Daemons and Agents which recommends against using mach ports, but I wanted to get something up and running, and the NSMachPort method was the one that seemed to be best documented.

However, after a lot of searching around and poring over some very sparse documentation, I’ve now got a version of the code working using NSSocketPort with unix domain sockets (AF_UNIX style sockets), which the tech note recommends as a good way to do IPC.

Tracking down all the details for this proved to be pretty hard work, and I had to glean various bits of information from a lot of disparate sources - tech notes, email threads and sample code.

The tough bits included:

getting NSSocketPort to use unix domain sockets
then getting NSConnection to work with them
getting launchd to launch a task on-demand when a unix domain socket is hit
getting the actual socket out of launchd in the server task

I’ve now updated my sample code to include all this good stuff - hopefully it will help someone else out in the future.

As a result of this the sample is a tiny bit more complex, and I’ve also taken the opportunity to refactor some stuff into utility classes. Because these are pretty useful I’ve pulled in my ECFoundation framework, and located some of them in there. However, the sample still doesn’t need the vast majority of ECFoundation, so I haven’t linked to the framework, I’ve just pulled in the source files that I need.

Check out the github repo for more details.

Happy Holidays!

2011-12-22T00:00:00+00:00

Whatever this time of year means to you, for us it means a few hard-earned days of rest, and probably far too much eating and drinking!

As a result, things will be quiet round here for the next week or two. If you happen to attempt to get in touch, please forgive us if we’re a bit slower than usual in responding.

If you’re on holiday too, we hope you have a good one! If not… well we hope you have a jolly nice time whilst it’s so quiet!

See you all in 2012 I hope.

OS X Code Injection

2011-12-20T00:00:00+00:00

Following on from my Helper Applications sample project, I’ve created another sample which focusses on how to do code-injection.

In particular this sample makes and installs a helper application who’s job is to inject some code into another application (by default the Finder).

The injected code adds a menu to the application’s menubar, with a single item in it that just logs stuff to the console.

Once again the project is based on existing examples, but attempts to present things in a slightly cleaner / simpler way.

It uses Wolf Rentzsch’s mach_star to do the heavy lifting for the injection task.

You can find it on github.

Once again I should point out that this example pretty much ignores security when it comes to the inter-process communication between the host application, the injection helper, and the injected code. Which isn’t to say that it’s not useful, simply that you’d have to batten it down far tighter if you want to avoid opening up some potentially horrible security holes.

In the sample, the helper is launched on demand, and only does the injection in response to a command from something else. In a real-world scenario, I guess it would make more sense for it to watch for the target app to launch and do the injection then. It would also make more sense if the various parameters were embedded into the helper to lock it down. However, in it’s current form, I guess the sample could form the basis of some sort of application-enhancer style system allowing multiple client applications to inject code into multiple targets.

OS X Helper Applications

2011-12-16T00:00:00+00:00

I’ve recently been looking at how to set up a helper application (one that either runs all the time as a background task, or on demand, potentially with system permissions).

Apple has a few samples in this area, but they’re not all that complete, sometimes not cocoa-centric, and I found them a bit confusing.

After a fair bit of puzzling it out I’ve created my own sample (a modified version of a couple of theirs), and I’ve put it up on github in the hope that it might help someone else. Here’s the read me file for the project:

About ECHelper

This is based on Apple’s SMJobBless example, which shows how to cleanly install a helper tool that needs to run with privileges as a launchd task.

Staying Consistent

The biggest problem with this task is that the helper tool and the host application that’s going to install it have to be set up very carefully to have the right code-signing details and bundle ids.

If you wanted to change these in the SMJobBless example you had to do it in lots of different places - and it was easy to miss one.

This sample gets round that problem by setting three user-defined values at the project level, in the Settings.xcconfig file:

HELPER_ID = com.elegantchaos.helper.helper
HOST_ID = com.elegantchaos.helper.host
HELPER_SIGNING = 3rd Party Mac Developer Application: Sam Deane

You should set HELPER_ID to the bundle id that you want to use for your helper application.

You should set HOST_ID to the bundle id that you want to use for the host application - the one that installs the helper (and which probably makes use of it, although that’s not necessarily the case).

You should set HELPER_SIGNING to the code signing profile that you want to use to sign everything. Note that if this profile is associated with a bundle id pattern (eg. com.elegantchaos.*) then the HELPER_ID and HOST_ID settings must match the pattern, otherwise xcode will refuse to sign the applications.

The name of the profile is embedded in various places, and it’s important that they all match exactly what’s in the certificate. For this reason you need to specify an exact profile name here (like “3rd Party Mac Developer Application: Sam Deane”), rather than a wildcard (like “3rd Party Mac Developer Application: *”).

Building The Plists

The tricky part of all of this is that the host application needs one plist, and the helper application two, and both of them make reference to the three values defined above.

What’s more, you can’t rely on Xcode’s variable substitution to fill in all of these values. Two of the plists are embedded in the __TEXT section of the helper application, which is a simple binary and therefore doesn’t live in a bundle. These are embedded with some special linker commands, and it appears that Xcode (and the underlying linker tools) don’t pre-process the lists in passing.

Even with the other plist (which is used for the host application) has a complication to it, because one of the special keys in it is a dictionary where a key needs to be replaced by the value of HELPER_ID. It appears that Xcode’s plist pre-processing doesn’t affect keys, only values, so you can’t use a variable in the key to fill in the correct value.

The way I’ve got round this is to add build scripts to both targets which take the original plists and perform the substitutions. The various linking stages then use the output of these scripts as their inputs.

In theory these derived plists should probably go into ${DERIVED_FILE_DIR}, so that they are hidden out of the way.

However, it seems that Xcode needs to be able to find them when the build starts, because they are set as the Info.plist to use for the targets. When I tried hiding them away in the build folder, Xcode couldn’t seem to cope, so instead I’ve added them to the project as “proper” resource files. Each time you build, they’ll be regenerated. Luckily though, Xcode (and git) don’t see them as having changed unless they actually do change (because you change the values of the variables mentioned above).

Example Helper

I’ve also expanded the example to illustrate one way to actually communicate with the helper application.

This example uses distributed objects, which is NOT a particularly secure way to do things. Assuming that your helper is running with enhanced priveleges of some sort, it obviously becomes a potential attack vector for things that are trying to take over your system. DO is big and powerful, and complicated, and thus hard to secure.

In reality you should probably use something simpler, like sending simple bytecode across a NSMachPort or NSSocketPort.

Neu 1.2 Released

2011-12-16T00:00:00+00:00

I’m very pleased to announce that Neu 1.2 is now live on the Mac App store, as well as available for download from this site.

This version has quite an extensive list of changes and improvements, including reorganised preferences, some new substitutions, and a substantial overhaul of the main UI.

Check out the release notes for full details.

Ambientweet 1.0.2 Released

2011-12-14T00:00:00+00:00

Ambientweet 1.0.2 is now available from the Mac Application store.

This is a minor revision, with a fix to a bug that caused it to stall and stop refreshing tweets after an hour or so.

Adding missing framework methods at runtime

2011-12-14T00:00:00+00:00

Let’s say that you are building with a modern SDK, but you are targeting some older platforms. For example, you’re using the iOS 5.0 SDK, but you want your app to run on iOS 4.2.

When you do this, you have to take care not to call routines which are only implemented on iOS 5. You can set your deployment target to 4.2, but that won’t prevent you from calling 5-only routines, they will simply be weak linked and will be nil on platforms that don’t support them.

Occasionally I slip up on this, and during testing I discover that I’m using a routine that doesn’t exist everywhere.

The simple solution when this happens is to stop using the routine in question, and find some workaround, but that’s a bit rubbish. After all, the routine has probably been added precisely because it’s useful.

Also, the routine, when it does exist, might be better than your workaround. Apple wrote it, so it must be good right? So you’d prefer to only use the workaround where necessary.

Finally, this is a temporary problem that will probably go away at some point. The routine is present on new systems, and eventually time will move on and you’ll stop supporting the ones where it is missing.

So rather than changing your code, how about finding a way to add the missing implementation when necessary? Luckily, Objective C is wonderfully dynamic, so this is entirely feasible. Here’s a little example taken from ECFoundation.

UIColor has a method getRed:green:blue:alpha: which only seems to be present in iOS 5 (although the header helpfully fails to mention this). How can we arrange to add an implementation for it, if it’s missing?

The first thing we want to do is add the replacement implementation. We can put this anywhere really, but for neatness it might as well go into a private category on the class we’re fixing up. The actual code here doesn’t really matter for the purposes of the example - the point is that you’ve now got a method implementation that you can install if you need to:

- (BOOL)ecGetRed:(CGFloat *)red green:(CGFloat *)green blue:(CGFloat *)blue alpha:(CGFloat *)alpha
{
	const CGFloat* components = CGColorGetComponents(self.CGColor);
	BOOL ok = components != nil;
	if (ok)
	{
		*red = components[0];
		*green = components[1];
		*blue = components[2];
		*alpha = components[3];
	}
	
	return ok;
}

So how to we do the installation? We only want to do it once, and we want to do it before the routine is called, otherwise it won’t be much use.

This sounds like a job for the +initialize method, but there’s a slight problem. What happens if you add an initialize method in a category and there’s already one defined in the system elsewhere. Presumably you get a name clash and one version won’t get called. Nasty.

Luckily, there’s also the +load method. This gets called super early, and has some interesting properties. Here’s a quote from a great Mike Ash article:

An interesting feature of +load is that it’s special-cased by the runtime to be invoked in categories which implement it as well as the main class. This means that if you implement +load in a class and in a category on that class, both will be called. This probably goes against everything you know about how categories work, but that’s because +load is not a normal method. This feature means that +load is an excellent place to do evil things like method swizzling.

What a piece of luck! The system frameworks will be loaded by the time it runs, so we can test for the routine’s presence, and if it’s not there, add it:

+(void)load
{
	SEL apiSelector = @selector(getRed:green:blue:alpha:);
	if (![self instancesRespondToSelector:apiSelector])
	{
		SEL replacementSelector = @selector(ecGetRed:green:blue:alpha:);
		Method replacementMethod = class_getInstanceMethod(self, replacementSelector);
		IMP replacementImplementation = method_getImplementation(replacementMethod);
		const char* encoding = method_getTypeEncoding(replacementMethod);
		class_addMethod(self, apiSelector, replacementImplementation, encoding);
	}
}

[Updated Feb 2012 to fix some github links that have changed]

Radar: Why Closed Bug Reporting Sucks

2011-12-06T00:00:00+00:00

Apple’s bug reporting system (imaginatively called Apple Bug Reporter to the outside world, but known as Radar internally I believe), is essentially closed.

What this means is that you can’t check up on a bug that you didn’t originate, you can’t share your bug report with other developers for testing, discussion or corroboration, and you can’t search for existing or similar issues.

This really really sucks. Recently I’ve been filing quite a few bugs, mostly on the development tool that I use most often - Xcode.

Previously I’ve been quite bad at reporting bugs. Two of the main reasons for that were lack of time, and lack of feedback.

Lack of time boils down to the fact that in the past I generally didn’t work for myself, and I was usually massively overworked / or at least my time was greatly over-committed by my bosses. This lead to the perpetual feeling that I was running to catch up and had no justification to indulge myself with anything time consuming that wasn’t worth the effort.

Generally I was actually very good at resisting the (mostly illusory) sense that it wasn’t worth doing something that had long-term benefits, because of short-term pressure. However, often I didn’t bother filing bugs to Apple, because of the combination of my lack of time with the second problem - lack of feedback.

Filing bugs in Radar quite often feels like sending text direct to /dev/null, in terms of the sense one gets that the effort spent doing it is worthwhile.

One reason for this is that the interface for doing so is incredibly primitive, and consequently filing bugs feels slow and awkward.

The other reason is that, as a developer, I’m only too aware of the need to not only be precise in the report, but to include lots of information. It’s hard to find the time to do this properly at the best of times (indeed many full-time testers that I’ve worked with seem to find it impossible, despite it being their job, but that’s another story). It’s even harder to find the will to do it if you can’t check first whether you’re wasting your time reporting something that already has 1000 duplicate reports.

Now that I’m indy (yay!), I have been making more of an effort. If I find a bug, or have a feature suggestion, I try to report it.

It still feels painful though. When I do report an issue, I often don’t supply nearly as much information as I’d like if I was the developer on the other end, because I still feel that I haven’t got the time to waste on taking a punt that someone somewhere actually gives a damn about what I’m saying.

Lest anyone should misunderstand, it’s not that I think that the Apple engineers, chained to their desks in their darkened coding dungeons in Cupertino, don’t care. I’m sure that they very much care (well, most of them), but most of the time the bug reporting system is a bloody great big emotion filter that manages to prevent any hint of that caring seeping out across the divide back to us mere mortals.

There are many things that Apple could do to improve this process.

When it comes to the reporting of bugs, the web interface could be better. Standalone Mac/iPhone/iPad applications could be written which can automatically fill in most of the contextual information that one needs to provide. The “Report A Bug” menu command could become standard in Apple applications, and a de-facto standard everywhere, with coding support to make it easy to implement.

Best of all though, the bug reporting system could be partially opened up.

I don’t want some sort of open source anarchic free-for-all where any Tom, Dick or Harry can get into a flame war with the developer responsible for implementing a feature or fixing a bug.

At the very least though I’d like to be able to do some sort of web search on a problem description and be told “45 issues match this problem” - even if I couldn’t see those issues.

Better still, I’d like to be shown the summaries of those bugs, if not the full bug reports.

Of course it would have to be possible to mark bugs as private (if they contain proprietary information), and in any case I’d only expect to be able to do this after logging in with my developer id - so Apple would have full control over what level of access I was given to a particular bug. Perhaps based on some internal concept they had of my level of experience, trustworthiness, the help I’d given tracking down previous bugs, etc.

Ideally I’d also like to be able to explicitly share bugs with colleagues, by attaching them to the bug so that they have full access.

And I’d like the bug reporting system to be integrated with the developer forums so that it was possible to have a forum thread linked to one or more bugs in a way that allowed a discussion surrounding a particular issue to be clearly associated with the bug id.

I’d like to be able to +1 a bug as a very quick indication that I also had it, or +1 a feature request as a way of adding my vote to it.

This stuff isn’t rocket science. I know that Apple has a lot of important stuff to do, but the strength and helpfulness of the developer community is one of it’s hidden assets. It would be great if Apple could show us diligent bug reporters a little bit of love.

New Betas

2011-12-03T00:00:00+00:00

A couple of new betas are available from the beta software page.

Neu 1.2b7 includes a couple of minor tweaks to the UI, and a revised manual. I know that there are a few typos to fix in the manual, but I’d definitely appreciate any other feedback on it, or the changes in Neu 1.2. I’m hoping to submit this version to the App store soon.

Ambientweet 1.0.2b1 fixes a stall that has been reportedly happening in Ambientweet - where it simply stops working after an hour or two. Again, all feedback greatly appreciated.

Thanksgiving Sale!

2011-11-24T00:00:00+00:00

I’m pleased to announce that over this weekend, we’re running a 50% off offer on Neu. For this period it will be available for the bargain price of $5!

We’re also holding Ambientweet at it’s current price (free!) over this period - after which it will actually cost something!

Go get em whilst they’re hot…

Ambientweet 1.0.1 Released

2011-11-22T00:00:00+00:00

Ambientweet 1.0.1 is out!

This version has a whole raft of improvements, including:

Reduced the frequency of refreshes, to avoid running over the Twitter call limit.
Post windows now use the same style as the main tweet window.
The length of posts is now calculated intelligently to account for link shortening. So if you paste in a long link, it won’t use up all of your character count, as the link will be shortened in the actual post.
Auto-completion list for @users in post window now includes all users that the app has encountered.
Now correctly restores cached tweets and users when loading.
Added subtle gradient to window backgrounds.
Fixed beta Sparkle feed URL (the app was looking at the non-beta feed instead).
Updated the manual with screenshots of new posting UI.
Fixed window resizing.
Improved sorting, refreshing with new items.
Changed user guide code to be sandbox-friendly (copies guide into the Caches folder and opens it from there).

There are lots more improvements and new features on the way in a future version, please give 1.0.1 a try!

Signing nested app bundles

2011-11-21T00:00:00+00:00

During a recent submission I came across a problem.

I got an automatically generated email back from Apple with this sort of thing in it:

Invalid Signature - the nested app bundle ECFoundation 
(Ambientweet.app/Contents/Frameworks/ECFoundation.framework)
is not signed, the signature is invalid, or it is not signed with an Apple submission certificate. 
Refer to the Code Signing and Application Sandboxing Guide for more information.

It’s a bit annoying that the email is capable of spotting that I have one of three possible problems, but not narrowing the actual cause down for me. It’s also a bit annoying that this only runs when you’ve submitted, and not when you verify an application with Xcode.

Be that as it may… … the main point is that it looks like Apple are starting to require that all code inside your application bundle is signed. This includes any frameworks or plugins you’ve embedded, even if you didn’t actually build them.

I’m slightly mystified as to why Xcode doesn’t do this for you at the point when you submit - since it asks you for an indentity to use, and presumably does some re-signing with it at this point.

However, as luck would have it, it’s fairly easy to write a script to do what we need.

for f in "${CODESIGNING_FOLDER_PATH}/Contents/PlugIns/"*
do
    BUNDLEID=`/usr/libexec/PlistBuddy -c "Print :CFBundleIdentifier" "$f/Contents/Info.plist"`
    codesign -f -i ${BUNDLEID} -vv -s "${CODE_SIGN_IDENTITY}" "$f"
done
 
for f in "${CODESIGNING_FOLDER_PATH}/Contents/Frameworks/"*
do
    BUNDLEID=`/usr/libexec/PlistBuddy -c "Print :CFBundleIdentifier" "$f/Resources/Info.plist"`
    codesign -f -i ${BUNDLEID} -vv -s "${CODE_SIGN_IDENTITY}" "$f"
done

This script goes through the build app’s PlugIns and Frameworks folders and signs everything it finds. It uses the application bundle id from each framework/plugin when signing. From one reading of the signing instructions, it sounded like everything bundled in with your app has to have the same bundle id, so this might not be quite the right approach - the script might need to take the bundle id from the app, sets the bundle id of each embedded framework to it, and then signs the framework.

However, apps submitted after building with this script don’t seem to generate the warnings, so for now this one seems to do the job.

I’m running the script with a final Run Script build phase in the target for my main application. This means that the bundles and plugins inside the app will always get signed, whatever configuration I choose.

Alternatively, you could probably run it as a post-action script in the Archive part of your build scheme, so that the extra signing only gets done when you’re about to submit. Actually I think that’s probably more in line with Apple’s advice, which seems to be not to sign things until you’re actually submitting them, so as not to give legitimacy to development builds which accidentally escape into the wild. However, I’m not certain that all the environment variables that I’m using will be set at that point, so it may be that the script would need modifying. I’ll leave that bit as an exercise for the reader…

Lazy Properties Revisited

2011-11-16T00:00:00+00:00

A while ago I blogged about how I’d really like Objective-C to have built in support for lazy properties.

My ideal solution would be something like this:

@property (nonatomic, retain, lazy) NSString* name;

The synthesized getter for this property would automatically call a method

 - (NSString*)nameInit

when the property was first accessed, and would use the resultant value to initialise the property.

This has some advantages over any hand-rolled solution:

it can remain efficient
it can support the retain/copy semantics and atomicity of the property
it avoids the need for boiler-plate code for every property to test if it needs to be initialised

In the absence of this solution in the language, I offered a rather complicated set of macros to attempt to implement something similar.

Since then I’ve thought about it a bit more, and come up with another couple of ways of tackling the same task… What Do We Want To Do? ———————-

Let’s say we’ve got a class:

@interface MyClass : NSObject

@property (nonatomic, retain) NSString* name;

@end

and we want to make the name property lazy.

What we need to do is to somehow insert some checking code into the -(NSString*)name getter function, that does something like this:

id value = self.name; // <-- this is a call to the ‘original’ getter
if (!value)
{
    value = [self nameInit];
    self.name = value;
}

return value;

Note that we need to be able to call the original getter here. We could attempt to access the underlying ivar directly, but using the original getter is much better as it preserves the property semantics regarding retain, copy etc.

Using Inheritance

One clean way to do this would be to insert a class in-front of our class, and implement our modified getter there. That allows us to cleanly call on to the original getter.

So we end up with something like this in our header file:

@interface MyClass : NSObject

@property (nonatomic, retain) NSString* name;

@end

@interface MyClassLazy : MyClass
@end

And this in our implementation file:

@implementation MyClass
@synthesize name;
// normal implementation here

// lazy initialisation function for name property
- (NSString*)nameInit
{
    return @"blah";
}
@end


@implementation MyClassLazy

- (NSString*)name
{
    id value = [super name];
    if (!value)
    {
        value = [self nameInit];
        self.name = value;
    }

    return value;
}

@end

Getting Things The Right Way Round

That’s quite neat, but there’s a problem. Our class is called MyClass, but in order to get the lazy behaviour, we need to use the class called MyClassLazy.

We can fix this with a bit of renaming. The header now looks like

@interface MyClassNonLazy : NSObject

@property (nonatomic, retain) NSString* name;

@end

@interface MyClass : MyClassNonLazy
@end

And the implementation:

@implementation MyClass
@synthesize name;
// normal implementation here

// lazy initialisation function for name property
- (NSString*)nameInit
{
    return @"blah";
}
@end


@implementation MyClass

- (NSString*)name
{
    id value = [super name];
    if (!value)
    {
        value = [self nameInit];
        self.name = value;
    }

    return value;
}

@end

Wrapping It Up Nicely

So that, in a nutshell, is a solution that works. With a bit of judicious macro definition, we can generalise this:

#define lazy_interface(class,parent) interface class##_nonlazy : parent
#define end_lazy_interface(class) end @interface class : class##_nonlazy @end

#define lazy_implementation(class) implementation class##_nonlazy
#define lazy_properties(class) end @implementation class
#define end_lazy_implementation(class) end

#define lazy_synthesize(prop) \
    class Dummy__; \
    - (NSString*)prop \
    { \
    id value = [super prop]; \
    if (!value) \
    { \
    value = [super prop##Init]; \
    self.test = value; \
    } \
    \
    return value; \
    }

So now, we can do this in the header:

@lazy_interface(TestClass, NSObject)

@property (nonatomic, retain) NSString* name;

@end_lazy_interface(TestClass)

And this in the implementation:

@lazy_implementation(TestClass)

@synthesize name;

// lazy initialisation function for name property
- (NSString*) nameInit
{
    return @"blah";
}

@lazy_properties(TestClass)

@lazy_synthesize(name)

@end_lazy_implementation(TestClass)

Can We Do Better?

Which is kind of nifty, but still, well, ugly. All our nasty macros make it not look like normal Objective-C.

So how about a different approach? Tune in to my next post…

Lazy Properties Revisited (Again)

2011-11-16T00:00:00+00:00

In my previous post I presented a way to implement lazy properties.

It was a bit macro-heavy though, and looked kind of messy. I wondered if there was another way. Objective C is marvellously dynamic and introspective. Shouldn’t we be able to do something clever at runtime to achieve what we want?

In a word, the answer is yes, we can… Let’s say we adopt a standard pattern for the initialisation methods for any lazy properties, like we did in the previous implementation:

 <prop>Init.

So for a property called name, the initialisation method would be called

nameInit

At class initialisation time, we can use Objective-C’s introspection routines to run through a list of all our class’s properties, looking for ones that have a lazy initialisation method.

For those that we find, we can swap out the normal implementation of the getter method for one which calls on to the original getter to get the current value, initialises it if the getter returned nil, then returns the value.

That sounds neat, but we will need to define a slightly different getter method for each property - since they all have to call on to a different original getter. Where are we going to put this code?

The answer turns out to be simple - we put it in the lazy initialisation method that we use to indicate that the property is lazy! So not only does the presence of the method tell us that it’s a lazy property, but the body of the method actually performs the lazy get.

Implementing The Replacement Getter Method

Each of these new getter / lazy-initialisation methods is going to look pretty much the same - call the original getter, check the value, initialise if necessary, return the value. Most of the code is boilerplate, but the getter method that we need to call onto changes for each property.

We could probably do something clever here to work out the name of this property at runtime by performing some selector to string conversions, but we’d have to do it every single time the getter was called. As we don’t want the performance to totally suck, it would be good if we could just make a normal call to the original method, check the returned value, and only do potentially slow and clever things on the one occasion when we need to do some initialisation. We can achieve this quite easily with a macro that expands to a few lines of code for every lazy getter.

So what do we end up with?

The interface for our lazy class looks like this:

@interface TestClass : NSObject

@property (nonatomic, retain) NSString* test;

@end

Ok, that’s… erm… pretty normal!

The implementation looks like this:

@implementation TestClass

@synthesize test;

+ (void)initialize
{
    if (self == [TestClass class])
    {
        [self initializeLazyProperties];
    }
}

@lazy_synthesize(test, @"test value"); 

@end

That’s also pretty normal!

We only have two odd things going on here. First, we need to perform a one off initialisation when the class is first loaded, to hook up all the magic.

We can do this in the +initialize method of the class. We could make this step even briefer with a macro, but in this case I figured that it was cleaner to just put the code in.

The second thing we’re doing is “synthesizing” the lazy getter. We could easily just put in the actual boiler plate code each time, but since writing boiler-plate is really what we’re trying to get away from, a macro makes sense.

This solution is much cleaner, and it works. In my next post I’ll show you the implementation.

Lazy Properties Dynamic Implementation

2011-11-16T00:00:00+00:00

In my previous post I showed a relatively clean way to implement lazy properties generically using the dynamic runtime.

So how does this dynamic implementation actually work? The interface is pretty small:

@interface NSObject(ECLazyProperties)

+ (void)initializeLazyProperties;

#define lazy_synthesize(name,init) \
class Dummy__; \
- (id)name##Init__ \
{ \
id value = [self name##Init__]; \
if (!value) \
{ \
    value = (init); \
    [self setValue:value forKey:@#name]; \
} \
return value; \
}

#define lazy_synthesize_method(name,method) lazy_synthesize_value(name,[self method])

@end

Most of the work here is in one macro.

Essentially what this does is define a lazy initialisation method for a property. The method:

calls on to the original property getter
if it’s got a value, we just return it
if not, we first KVC to update the original property with its initial value

We define the name of the method by appending Init__ to the property name. We could have appended just Init, but this method really shouldn’t be called publicly, so I went for a slightly more obscure name.

How it calls the original getter is kind of interesting. If you unpick the macro you’ll see that it appears to be calling itself. Isn’t this going to lead to some hideous recursion?

The answer is no - because the method will have been swizzled. When we’re actually running in the method, it will be as a result of a call to the original getter method. So if we want to call the original getter method, we need to actually call the init method instead!

If we do discover that the value is uninitialised, we then use KVC to call the proper setter with an initial value.

So all of this relies on some swizzling. How do we set that up?

The implementation of initializeLazyProperties looks like this:

+ (void)initializeLazyProperties
{
    uint count;
    objc_property_t* properties = class_copyPropertyList(self, &count);
    for (int i = 0; i < count ; i++)
    {
        const char* propertyName = property_getName(properties[i]);
        SEL init = NSSelectorFromString([NSString stringWithFormat:@"%sInit__", propertyName]);
        if ([self instancesRespondToSelector:init])
        {
            SEL getter = NSSelectorFromString([NSString stringWithFormat:@"%s", propertyName]);
            Method initMethod = class_getInstanceMethod(self, init);
            Method getterMethod = class_getInstanceMethod(self, getter);
            
            method_exchangeImplementations(initMethod, getterMethod);
        }
    }
    free(properties);
}

We iterate through a list of properties, looking to see if there is a corresponding lazy initialise method for it. If there is, we simply swizzle the original getter method with the init method.

When the user calls the getter, they’ll actually get the init method. This will call on to the getter, check the value, and initialise if necessary.

Complex Initialisation

In the example, we’re just supplying a fixed initial value, but of course the real point of lazy initialisation is for situations where we need to do something time consuming in the initialisation, and we only want to actually do it if we have to.

For this situation you can use the second macro - lazy_synthesize_method. Instead of passing in an initial value, you pass in a method to call to return the value.

Conclusions

This code seems to actually work, but I’ve not tested it intensively to see whether it behaves property in all situations - for example with KVC/KVO.

I think it should do, but right now I view it largely as a thought experiment.

Right back at the beginning I said that the way I really think it should work is

@property (nonatomic, retain, lazy) NSString* name;

I still think that would be ideal. I think that what I’ve done illustrates that it ought to be entirely possible to add a hack to Clang to implement this, but I leave that as an exercise for the reader…

If anyone is interested in the full source code to my implementation, you’ll find it in the ECCore module of my open source ECFoundation library.

[Updated Feb 2012 to fix some broken links]

ECLogging - Better Logging For Objective-C

2011-11-16T00:00:00+00:00

There comes a time in every programmers life when all the fancy symbolic debuggers let us down, and we have to fall back to that old standby, logging. Or to give it its full name “printf debugging”!

Actually there are lots of reasons why logging can be useful. Sometimes we have what I like to call a “heisenbug” - where the very act of stopping in a debugger to look at the problem causes the problem to disappear. Sometimes we have other timing issues. Sometimes we simply have too much data to analyse in real time, and we need a chance to process it or manually sift through it later.

In these cases the typical solution is to insert print statements into our code which output some text to the console or a file. In C this is typically printf(), or perhaps fprintf(). In Objective-C, we use NSLog instead. Problems ========

There are a few problems with logging code though:

we typically only want it whilst debugging
sometimes we only want it for a short time during a particular run
some log messages are more important than others
a large, well modularised, program may be under the control of many programmers, each with their own logging; often it all comes out together, leaving you drowning in output

Traditional Solution

Some of these problems can be solved, but it can get messy.

A typical solution is to define one or more macros which do the logging. By varying the definitions of the macros we can turn the logging on or off. This is a bit clumsy. You have to know that the macros exist, and where they are defined, and you have to recompile the program to change something. In a big program this may not be trivial - you may not even have source code for every framework.

In some solutions the logging macros that are defined include some concept of a level - for instance “warning”, “error”, and so on. This might allow us to vary the logging at runtime by only allowing through the log messages that match (or perhaps fall under) a given level. This can help, but it’s still clumsy. Logging messages don’t necessarily fall neatly into levels. Furthermore, messages from different modules will match the same level, so if you ask for a level you get everything at that level.

The ECLogging Solution

The ECLogging module sets out to solve these problems in a different way.

Channels

All logging is sent to a named channel, and there can be as many named channels as are needed. Named channels are intended to group logging messages by their purpose or their module, rather that by some concept of “level”.

Logging channels can of course be turned on and off. This can be done at runtime, and the particular configuration of channels that are enabled is saved on each machine, so that a programmer can set things up with only the channels that they want to see enabled. In fact, because of this, all channels default to being disabled, so you don’t get spammed with log output that you didn’t ask for.

Because channels in ECLogging are actual named objects, the logging system can iterate through them and present a user interface for their configuration.

Channels also come in two flavours - log and debug. Log channels exist in all builds. Debug channels only exist in debug builds, and the code associated with them is completely compiled away in release builds.

Handlers

So log output goes in to the system via channels, but where does it come out?

The answer is handlers.

The logging system contains a number of predefined handlers, which know how to output to the console, the standard output and error streams, and to files on the disc. Custom handlers can be defined and added to do anything conceivable with log output - send it to another process over the network, for example.

By default, all channel output is sent to a global set of handlers. By choosing the handlers in this set you can easily influence where most of the logging output goes.

However, any logging channel can be configured to use it’s own custom set of handlers. In this way you can determine that some channels just go to the console, some just go to disc, and so on.

What Does It Look Like?

In use, the system is very similar to NSLog, except that you have to first define a channel, then direct output to it.

ECDefineLogChannel(MyChannel);

@implementation MyClass

- (void)myMethod
{
    ECLog(MyChannel, @"some message with params %@ %d", self, 123);
}

@end

Add to this a small amount of code in your application delegate to initialise the system and install the handlers, and you’re good to go.

Drop in some additional iOS or Mac OS X user interface items, and you’ve got a way of configuring channels at runtime. Alternatively, you can just turn channels on and off in code, or by setting NSUserDefaults values.

Where To Find Out More

ECLogging can be found on github, including some decent documentation.

The ECLoggingExamples projects illustrate how to use the ECLogging module on both the Mac and iOS. They should be built from a workspace which also includes the ECLogging project.

Both of these samples are quite basic, and should definitely be regarded as a hint in the right direction, not a definitive illustration of everything that’s possible.

Status

The library design is quite mature - it’s a meme I’ve implemented many times over the last decade or more, in different environments.

The implementation is quite immature, for the simple reason that I’m doing bits when I need them. It does work though, and I use it extensively, both in my own products and others that I contract on.

If anyone shows any interest in it, no doubt I’ll be motivated to get my finger out and improve it further!

[Updated Feb 2012 to fix github links, which have moved]

Ambientweet 1.0.1 submitted

2011-11-14T00:00:00+00:00

Ambientweet 1.0.1 is on it’s way, and has been submitted to the Mac App store today for review.

It brings with it a slew of syncing/caching updates which should result in more reliable fetching of tweets (especially after it’s been running for a while), and a faster startup.

The posting interface has also been improved - it looks more like a normal tweet window now, and it correctly deals with long urls, allowing you to post tweets that appear to be longer than 140 character, but which will be under the limit after Twitter has shortened the links.

File A Bug And Get A Free License

2011-11-04T00:00:00+00:00

It’s hard to make good products if you’re working in a vacuum. That’s why one of the things that we like most is hearing from people who’ve used our software, even if the message is “it’s broken”!

So if you send us a good suggestion, or a report of a genuine bug that we didn’t know about, there’s a good chance that we’ll send you a free license in return.

No promises mind you - if you send fifteen reports, each about an individual spelling error in the manual, then you probably won’t get fifteen licenses back :)

Sandboxing... climbdown?

2011-11-03T00:00:00+00:00

So, Apple have announced the deadline for developers to have their Mac App store apps work with sandboxing, and suddenly it’s March 2012, and not the November 2011 date that was widely understood amongst the developer community.

I guess I should be grateful, since the current sandboxing is distinctly half-baked as a plan. The only problem is, working on the assumption that we had to have sandboxing implemented by November, I’ve now got a relatively major update to Neu which uses it, and is waiting on just one sandboxing bug to be sorted out. Having removed the November deadline from developers, Apple have also conveniently removed the urgent need to fix the bug that’s holding me up.

So now I have to decide whether to hold back the Neu update - potentially for four months - whilst I wait for the bug fix (not a particularly attractive option), or to do more work turning sandboxing off again in Neu and splitting out the sandboxing-inspired changes from those that can ship regardless.

Sigh.

As I’ve said before, I’m broadly in favour of sandboxing, or at least the the motivation behind it. Anything we can do to protect users from malicious software is fine by me, as long as it doesn’t detract from what users actually want to do.

The way Apple have gone about implementing it though, has left a lot to be desired, and smacks of a certain hubris. They told us, rather than asked us, what we needed. Perhaps more worryingly, they got the answer quite substantially wrong. It’s good that they’ve seen sense and delayed it, but by announcing a new date they are still holding up a hostage to fortune, since we don’t have a fully working system yet.

Personally I think that they made a fundamental error in even considering using a metaphorical stick to force developers to adopt it. It would have been far better to use a carrot. If the Mac App store showed a subtle badge on sandboxed apps, with supporting material explaining that apps with the badge had “enhanced security” or some such phrase, it would create a positive incentive for developers which ought to translate directly into increased sales.

Using an application with “enhanced security” is going to be attractive to most users, regardless of exactly how much they understand about what the “enhancement” actually represents - but it leaves the choice in the hands of the user.

Whilst I’m not normally a great fan of marketing bullshit, or of overloading users with more information than they need, I think that this is a situation where presenting a simplified (but accurate) picture of the benefits of sandboxed apps is far preferable to forcing all apps to be sandboxed.

Neu 1.2b3...

2011-10-28T00:00:00+00:00

… and another one!

Neu 1.2b3 is available for the brave of heart!

This one has a rough implementation of the new UI for managing your templates.

Rather than a separate templates window, you can now right-click on a template in the main UI, and do various things to it directly (rename, delete, reveal, open), or you can choose “Edit Template…” to open an editing pane. This pain allows you to set some per-template preferences, including a keyboard shortcut.

Known Issues

In grid view, the context menu commands on templates currently operate on the selected template, which might not be the one that you clicked on! So be careful with the delete command!

Per-template keyboard shortcuts don’t work yet.

The “Rename Template…” command doesn’t work yet.

Due to a bug in sandboxing, when overwriting an existing document you will be asked “Do you want to replace?” but if you say OK the creation will fail. The workaround for now is simply to give the new document a different name. Neu will always default to a unique name for the file, so this shouldn’t affect many people.

Neu 1.2b2

2011-10-27T00:00:00+00:00

Another day, another beta…

Neu 1.2b2 fixes some issues with 1.2b1, and adds some additional features.

Added “Keep Window Centered” option.
Fixed keyboard shortcut preferences.
Neu now spots if the templates directory is deleted whilst it is running - it will recreate the directory and fill it with the default templates.
Improved window and grid appearance by adding some subtle gradients.
Grid items now have tooltips with full name of the template.
Added userName and userFullName to substitution variables. Added some more text filters to convert MixedCaps text - useful for coding templates.
Added an example template for an objective-c header file, which illustrates how you can modify the name of the new file to format it to suit your coding conventions.

Known Issues

This version has a few issues that I know about.

Another issue is that user guide doesn’t show properly at the moment. This is actually another Sandboxing issue, but it’s probably a good thing as the user guide hasn’t been updated to reflect the changes in Neu 1.2!

Neu 1.2b1 Released

2011-10-26T00:00:00+00:00

Hot off the presses is a early beta of Neu 1.2.

Warning! This is an early version, and there are some known problems, so please don’t use it unless you are prepared for a bumpy ride.

The major changes in this version revolve around some upcoming security changes that Apple are making.

From November onwards, applications in the Mac App store must use something called Sandboxing. This is a system which asks applications to deliberately limit what they can do - for example which areas of the disk they can write to - to the smallest set of things that they need to do. The idea behind this is that if something nasty manages to hijack an application, it will be able to do less damage since it won’t have complete access to the machine.

In theory this is a nice system, but in practise it imposes some limits on applications which can cause applications like Neu some difficulty. In particular, it means that in some cases Neu can’t write new files to a directory unless it shows you a “Save As” dialog.

For this reason the Neu user interface has changed a bit.

Using Drag & Drop

The biggest change is that you can now use drag & drop to drag items directly out of the templates window and onto the Finder.

For people who like using the mouse/trackpad, I’d suggest that this is probably now the best way to use Neu. The reason for this is that when you drag items directly into the Finder, there won’t be any Sandboxing issues. If you set the preferences accordingly, you won’t have to go via a “Save As” dialog.

You can also use drag & drop to drag things into the Neu templates window - when you do this it will take a copy of the dragged items and add them to its templates list.

Using Menus & Keyboard

The old ways of working with Neu via menus are still there. You can still set a keyboard shortcut to bring the template list up, and you should be able to use the keyboard to select a template and create a document.

The only change is that sometimes you don’t get the choice to skip the “Save As” dialog. If you look in the preferences you’ll see that the preference relating saving has been renamed to reflect this difference. It now says “Hide when possible” to reflect the fact that sometimes it isn’t possible!

Using Services

When you launch Neu via the services, the Sandboxing system doesn’t need to force you to use a “Save As” dialog, so again this way of working should remain unchanged.

Other Changes

Neu now requires the 64-bit version of Mac OS X 10.6. This shouldn’t affect many people, as most of the machines that are capable of running 10.6 are 64-bit already. However, there are a few rare machines that run 10.6 but only in 32-bit mode. I’m afraid that Neu 1.2 won’t work on these machines. Neu 1.1.1 will continue to work fine though.

The template list now refreshes itself automatically if you change the contents of the underlying templates folder, so I’ve removed the Re-scan menu item.

The templates window UI now allows switching between list and grid view directly from within the window. It remembers the way you left it last time, so I’ve removed the corresponding view preference. The template window also has a “Open after creation” checkbox on it now, which you can use to change whether or not you want to open the resulting document after it is created.

I’ve re-organised some of the other preferences - a couple of more technical ones are now on a panel called “Advanced”, and some of the other Appearance and Behaviour options have been merged back into a General panel.

The small status menu icon is now see through like other icons!

If you only want to use Neu with a single template, we simplify the menus, and we don’t bother showing the “choose template” dialog. This streamlines things.

Still To Come

Some more features that I’m hoping to get into 1.2:

drag & drop to rearrange the order of template items
individual per-template preferences and keyboard shortcuts
more text substitutions
auto-centring the templates window

Known Issues

This version has a few issues that I know about.

Coming Up Soon: Neu 1.2

2011-10-25T00:00:00+00:00

I’ve been doing a fair amount of contracting recently, including this crazy project, so development of Elegant Chaos products has had to take a bit of a back seat.

Happily though, I’m now in the position to spend some time on it again, and now that Ambientweet has made it to 1.0, the next update on the agenda is Neu 1.2.

This update of Neu is going to be quite substantial, not least because Apple are about to change the rules on how an application in the App store is allowed to interact with the Finder and the file system. This is going to require me to change the user interface a bit. There is a slight danger that it’s going to get a bit more clumsy, and I may have to permanently enable some settings that have until now been optional, such as whether you get asked for the file name / location when making a new item.

I’m hoping to have a beta version ready for release in a few days, and I’m especially keen to get some feedback on this version.

If you’d like to test it, please either check the beta software page, or get in touch.

If you have purchased an App Store version of Neu (and have updated to version 1.1.1), you should now be able to download betas from this website and test them. Although they have come from this site and not the App store, they ought to recognise that you have had a paid version of Neu on your computer from the App store, and won’t annoy you with reminders.

As always, if you have any problems with this, or anything else to do with Neu, please get in touch and I’ll try to help. Likewise if you have any feedback or feature suggestions for version 1.2 or beyond.

The iOS simulator and case-sensitivity

2011-10-19T00:00:00+00:00

One of the most common problems I find when switching between testing on the simulator and testing on the device is the fact that the device’s file system is case-sensitive, whereas the simulator uses the user’s default file system which generally isn’t.

This can throw up mysterious bugs where things appear to work fine on the simulator, and then fail unexpectedly on the device because one or more files are actually being referred to by the wrong names (or rather, the right names, in the wrong case).

It occurs to me that the simulator could instead create and mount a private DMG disk image to use as the simulated device’s disk. It would then be able to format this image using a case-sensitive file system, which might provide a more realistic simulation of an actual device’s behaviour.

For what it’s worth, I’ve filed an enhancement request.

Ambientweet 1.0 Released

2011-10-03T00:00:00+00:00

Hurrah!

After a beta period even longer than Apple’s iOS 5 (well, maybe), I’m pleased to announce that Ambientweet 1.0 has been released, and can be downloaded from the Mac App store.

As this version still has relatively modest functionality, and as I’m mostly interested in getting some user feedback, I’ve decided that it will initially be available for free.

Eventually I plan to start charging, but of course if you grab it from the store now, you will be able to get any future updates to this version for free too.

Ambientweet getting close

2011-09-28T00:00:00+00:00

I’ve released another two betas of Ambientweet in the last couple of days, and hopefully it’s now getting close to being ready for submission to the Mac App store.

The current plan is still to release it as a free app to begin with, as the main thing I’m interested in at this stage is getting a bit of feedback.

Ambientweet 1.0b27 released

2011-09-14T00:00:00+00:00

It’s been a while since I’ve had a chance to do much to Ambientweet, but it’s still trundling along in the background. It’s a fairly experimental project, and as such it definitely suffers a bit from the “there’s one more thing” syndrome.

In other words, every time I think it’s time to release it, I have another idea (or spot another bug).

Having said that, hopefully you’ll find that the latest version is fairly close to being ready for submission to the store.

As always, any feedback appreciated…

Neu 1.1.1 release

2011-08-21T00:00:00+00:00

I’m happy to say that Neu 1.1.1 has been released, and is now available from this website and Apple’s Mac Application Store.

This version is pretty much the same as 1.1, except that it fixes a bug that caused the service names to have the wrong values. See the release notes for full details.

Sandboxing and Neu

2011-08-11T00:00:00+00:00

For those who don’t know, Sandboxing is a new facility introduced in Mac OS X Lion, which requires applications to list what abilities they want from the system, and then restricts them at runtime so that they can’t do other things. At the moment sandboxing is optional, but fairly soon it’s going to be obligatory for applications that are sold through the Mac Application Store (MAS).

I have a utility application (Neu), currently on sale in the MAS, which is going to run into major problems with sandboxing as it currently stands.

The utility allows the user to create files quickly from templates. Really it just does a glorified copy of a stationery file into a location of the user’s choosing - but it presents the user with a simplified interface for doing this.

The problem with sandboxing comes with how the utility chooses where to create the new file.

Initially Neu was written to be triggered by a Services menu selected from the Finder. It would then create the new file in the location represented by the Finder window that the user had open when they opened the services menu.

I am concerned that this may not work under sandboxing - unless the sandbox system automatically adds any paths that it supplies to an application to the list of allowed accesses.

However, there is a more serious problem. The Finder/Services integration isn’t brilliant, and in particular, there’s no way to right-click on a Finder window and have it show a services menu for the folder that the window represents (see rdar:7719410).

Because of this, I’ve added some other mechanisms for triggering Neu. These mechanisms rely on looking at the front window of the Finder, and inferring the location that the user wants to create files in from it.

I’m assuming that this isn’t going to work with sandboxing. Instead I’m going to have to throw up a “Save As” dialog and ask the user where to save the new file. This is already an option in Neu, but requiring it is going to annoy the users that turn it off by default.

It seems like sandboxing is a great idea, but if we’re not careful it is going to destroy a whole class of utilities which work by integrating with other applications. The Finder is the most obvious of these other applications, but I’m sure that there are many additional examples - for example utilities that watch what track is being played in iTunes, and then do something with it.

Because of this, I think we need the possibility of specifying an entitlement allowing the application to interact with another process id - in a way that then allows the application to do the sorts of things that I’ve described.

In the case of Neu, right now I’d have to request permission to interact with the Finder. However, in future versions I was also planning to allow Neu to infer a save location from the currently open document in the front application - thus letting it integrate more intelligently with whatever program the user is currently using. This would require yet-more flexible permissions.

I have some ideas for a different UI which will get round some of these problems for Neu, but these changes will only work at the expense of some of the intelligence that I had hoped to build into it. Essentially they will force the user to explicitly determine save locations at all times. This is a shame - there are times when we want the computer to make intelligent default choices for us. It seems like Sandboxing may curtail our ability to implement those default choices.

User Signup

2011-08-04T00:00:00+00:00

Unfortunately, because of persistent signups by spam users (probably driven by a script somewhere), I’ve decided to disable the facility to sign up to this site.

Really the only reason for allowing people to sign up in the first place was to in turn limit posting of comments and forums posts to registered users, again to avoid spam.

As a result, turning off sign-up may make it impossible for people to leave comments or use the forum. This is a real shame, but the alternative is just too annoying to police. In the meantime I’ll look into some alternatives. I may re-enable anonymous posting, along with captcha forms, but I fear that doing so will also result in a torrent of spam.

Very irritating…

Neu 1.1.1 beta

2011-08-04T00:00:00+00:00

I’ve had a few reports of the Services menu commands not showing up properly in Neu 1.1.

Instead of saying things like “Create Document…”, they are showing up as things like “create.name”.

If you’re having this problem, you might want to look at the new beta of Neu 1.1.1, which attempts to fix the problem.

For people who’ve bought Neu from the Mac Application store, I’m afraid that this beta will show up as unlicensed, so you shouldn’t remove your original 1.1 version of Neu until version 1.1.1 has made it’s way through the approval process.

I’m working on a way to sort this problem for future beta versions (so that they know that you’ve got a proper licensed copy from the MAS), but they will require you to already have version 1.1.1 installed, so there’s a bit of a chicken-and-egg problem.

Neu 1.1 Released

2011-07-12T00:00:00+00:00

I’m happy to say that Neu 1.1 has been approved by Apple, and is now available from the Mac App store, and from this website.

This version of Neu includes a number of improvements:

Added “behaviour” and “appearance” preference panes, removed “general” pane, reorganised preference items.
Improved handling of symbolic links, added support for Finder aliases, and added options relating to them. Added “Create Document On The Desktop…” and “Create And Open On The Desktop…” service menu commands.
When documents are created on the desktop, they are now just selected on the desktop in the Finder, rather than another “Desktop” window being opened.
Added some more substitution variables.
Switched Help to use an embedded PDF manual instead of Apple Help / web pages.
General bug fixes and improved OS support.

It doesn’t stop there though, work on Neu 1.2 is underway, so watch this space.

Neu 1.1 in review

2011-06-23T00:00:00+00:00

Good news - Neu 1.1 has been submitted to the Mac App store, and is currently in review.

Meanwhile, for the brave-of-heart, beta version 1.1b5 is available from the betas page.

Work on version 1.2 is also well under way… more on that soon.

Neu 1.1b2

2011-06-15T00:00:00+00:00

Hot on the heels of the first one, I’ve just released a second beta of Neu 1.1.

In addition to the changes in 1.1b1, this version has a few tweaks to the default preferences, and a new pdf-based manual, available from the Help menu.

Neu 1.1 in beta

2011-06-14T00:00:00+00:00

I’m happy to announce that I’ve just released the first beta of Neu 1.1.

This is a relatively minor update, but there are a few new features as well as some bug fixes, hence the upgrade from .0 to .1…

Hilights from the change log include:

Added “behaviour” and “appearance” preference panes, removed “general” pane, reorganised preference items.
Improved handling of symbolic links, added support for Finder aliases, and added options relating to them.
Fixed a couple of bugs relating to template files and file extensions.
Improved OS X Lion support.
Added “Create Document On The Desktop…” and “Create And Open On The Desktop…” service menu commands.
When documents are created on the desktop, they are now just selected on the desktop in the Finder, rather than another “Desktop” window being opened.
Added some more substitution variables (templateFullName, templateName, templateActualName, templateType).

Debugging Services in OS X

2011-06-08T00:00:00+00:00

When you’re developing an application that provides Services in OS X, you sometimes need to change the definitions of the services.

These are stored in the info.plist in your application, but just changing it and rebuilding the application isn’t enough, you also have to force the system to re-index the services so that it spots your changes.

You can do this by calling NSUpdateDynamicServices() in your application, or by running /System/Library/CoreServices/pbs on the command line.

I recently came up against a problem with this though. When pbs scans, it seems to just pick up the definitions from the first version of a given application. If you’re developing, you often have multiple copies lying around, and you don’t seem to have any control over which version pbs will come across first.

I’m guessing that pbs simply scans your volumes in alphabetical order. I had an issue where it was picking up an old version of an app in the “Archived Applications” section of the “Derived Data” folder on a different partition, before finding the version that I had just built.

I’ve found a simple fix which worked for me - simply move the new application into /Applications temporarily, and run a scan. This seemed to be enough to get pbs to pick up the correct app. Whether it will always work, I’m not sure, and it’s a pain to have to keep copying it there if making multiple changes, but luckily the services settings are something that you tend not to have to tweak very often.

Relocated

2011-06-03T00:00:00+00:00

Finally, we’ve completed our relocation, and are now based in Stornoway, in the Outer Hebrides. This is quite a change of scenery compared to South London!!

The move has been a great distraction over the last few weeks, so there hasn’t been a great deal of development work on Neu, Ambientweet, or any of our other projects.

However, the internet connection here is just fine, and as the dust settles I should once again be able to focus on development. First priority is checking both Neu and Ambientweet for Lion compatibility. After that, I’m hoping to finally get an initial version of Ambientweet into the store, and to work on some updates for Neu.

Relocation

2011-05-10T00:00:00+00:00

At the end of May, we are finally completing our relocation.

We’re moving all the way from South East London (down at the South East end of the UK), to Stornoway in the Outer Hebrides (right up at the North West end of the UK), so it’s quite a major operation.

Hopefully it will all be done by the 1st of June, and within a week or two of that the dust will have settled. In the meantime though, we might be a bit slow responding to email! Please bear with us…

Neu 1.0.4 Release

2011-04-10T00:00:00+00:00

I’m happy to announce that Neu version 1.0.4 is now available, both from this website and the Mac App store.

Version 1.0.4 is a bug-fix release, identical to 1.0.3 with the exception that the preferences should now work correctly on the Mac App store version!

To find out more, or give Neu a spin, please see the main Neu homepage.

Neu 1.0.4 submitted for review

2011-04-05T00:00:00+00:00

I’ve submitted Neu 1.0.4 to Apple for review.

The only difference between it and 1.0.3 is that it fixes the bug with the Mac App store version of Neu that prevented some of the preferences panels from showing.

I’ve also made it immediately available here, for non Mac App store users. It really is no different from 1.0.3 for them though.

Neu 1.0.3 Mac App store problem

2011-04-05T00:00:00+00:00

Hi all,

I’m sorry to announce that we’ve spotted a slight problem with the Mac App store version of Neu 1.0.3 - only two of the preference panels load successfully.

This problem doesn’t affect the version of Neu that you download from here, so if you’ve bought Neu directly from us, you can safely update.

However, if you’ve bought from the Mac App store, we’d ask you to hold of updating until we can work out what’s going on and submit a fix.

If it’s too late and you’ve already updated, Neu should continue to work ok, you simply won’t be able to adjust certain preferences. Which is obviously not ideal, so we’ll get a fix out to you as soon as we possibly can.

Neu 1.0.3 Released

2011-04-04T00:00:00+00:00

I’m happy to announce that Neu version 1.0.3 is now available, both from this website and the Mac App store.

Version 1.0.3 is a bug-fix release, containing the following minor improvements:

Remove/add buttons update state correctly in templates and keywords preferences
Fixed window resizing
Improved communication with the Finder to work out what folder the user wants to create items in.
Improved text of “Add License…” menu item and file dialog (This change does not apply to the Mac App store version of Neu, which does not have an “Add License…” menu item).

To find out more, or give Neu a spin, please see the main Neu homepage.

Updates coming

2011-03-26T00:00:00+00:00

Things have been a little quiet around here lately, as we’ve been tied up with quite a bit of contracting work.

However, we’re happy to say that some updates to our own applications are also on the way. Both Neu 1.0.3 and Ambientweet 1.0 have now been submitted to Apple for release to the Mac App store.

Beta releases of both of these can also be found on our Beta Software page.

With Ambientweet, we needed to decide on a pricing policy as part of the submission.

Eventually Ambientweet will be sufficiently feature-rich to justify charging for it, but for now it is quite simple, so we’ve decided to make it the initial version free on the Mac App store! If you download it there when it becomes available, you should be able to get subsequent 1.x updates free as well.

When we change over to charging for it, we’ll probably create a new “Pro” version of the application which you have to buy. At that point we may well also introduce some sort of “Lite” version which contains advertising.

NSConference, designers, and stuff

2011-03-24T00:00:00+00:00

I was going to post something about NSConference 2011 anyway, because I agree with the general consensus - that it was bloody brilliant.

Danny Greg’s post has spurred me into action though, because I sort of agree with him, and sort of disagree!

It is certainly true, as Danny said, that it was great to add some more non-programming perspective to an already excellent conference format.

Having some entrepreneurial input was great, as so many of us are dipping our toes in the indie waters. I found both Kevin Hoctor and Ken Aspeslagh’s talks reassuring, inspiring and challenging in equal measure. They managed to pull off a rare trick - talking to developers about business in a way that made the developers want to listen.

And as Danny pointed out, having some more design/interaction sessions was also fantastic. There were lots of great speakers on this topic (including those like Matt Gemmell who’ve spoken in previous years), but Cathy Shive and Aral Balkan were my favourite sessions.

Where I slightly disagree with Danny is in the idea that need for good design is a change that’s just been happening “for a while”. The implication being that whilst it didn’t happen yesterday, it’s happened in the last few years.

The fact is, the need for good design has always been there, and good design has always been evident in the truly good software. Apple have known this all along. I tweeted to Aral about the fact that I thought he was a fan of Don Norman based on the content of his session (he is). The session’s title was a big hint, come to think of it (if you haven’t read The Design Of Everyday Things, do so!). Don was already in my mind though, as a lot of what Cathy said in her talk the day before also made me think of him. The reason I know about Don Norman at all was that he was knocking around at one of the early WWDCs that I went to, in about 1995. That was because Apple had just hired him (it didn’t stop them having a good many design related disasters over the subsequent years, but in theory at least, they’ve always known the value of good design!).

I believe that the best programmers, like the best graphic artists, the best engineers, the best musicians and authors and scientists, have an ability to transcend technique. They may be brilliant technicians in their chosen field, but they also have the ability to strip away all that skill and get to the essence of what’s important- which is why that Antoine de Saint-Exupéry quote is so relevant.

They have an aesthetic appreciation for function, as well as form, and they can whittle away at a design until all it is clean, and elegant, and entirely user centred. This manifests itself in software as something that does the what the user wants - intuitively, delightfully, effortlessly.

I believe that this ability, this particular sort of focus and clarity of thought, is inter-disciplinary, or perhaps cross-disciplinary, or, to put it in nerd terms, it’s orthogonal to the other stuff.

I also believe that having this ability is what we mean when we say “designer”. You can be an average artist, programmer, entrepreneur, and also a great designer. Or you can be a prodigy in your chosen field, and a piss-poor designer. I’d hazard to say that I don’t think you can be truly great at whatever it is you do unless you have both the technical chops and that focus on design.

In a conversation outside the sessions (that I was loitering on the edge of!), Aral and Cathy were both talking about the fact that programmers have a tendency to confuse the ability to draw with the ability to design. Cathy (I think) was also saying that many programmers have that design ability - and that it manifests itself in other ways, like designing nice APIs. That we tend not to give ourselves the credit that we deserve, and also tend not to recognise how we can transfer these skills.

I think that’s true - many programmers are too hard on themselves when it comes to design - but there is also a converse truth. In my experience, many programmers are actually piss-poor at designing APIs! The same thing that makes them bad at designing user interfaces actually makes them bad at designing code too. They overcomplicate, or they over-focus. They fail to see the cheesy wood texture for the b-trees. They are actually not good designers.

I’m generalising of course, but I do actually think that this is a character trait that’s quite common in programmers, and whilst we’re being generous about ourselves in other ways, we also have to recognise the fact that many otherwise good programmers can’t design. They do lots of things that are absolutely essential, but they can’t, and shouldn’t, be responsible for the design!

An appreciation for design is critical to making good iPhone apps. It’s also critical to making good frameworks. Or good chairs. It is therefore essential that we are aware of the importance of design. It doesn’t mean that we’re all going to suddenly become designers - some of us will have to settle for being really good programmers who work with really good designers.

It does mean that it’s really great to have such excellent design related sessions at NSConference!

Writing Code With Text - What's That All About?

2011-02-04T00:00:00+00:00

Recently I’ve started some contracting work with a new client, and rather impressively they have a coding standard. This is most definitely a good thing.

However, this standard, like most, includes some stuff relating to text formatting, and it happens to be different from the way I like to do it.

I tend to agree with Sutter and Alexandrescu on this sort of thing. In paragraph 1 of item 0 of their C++ Coding Standards book, they say:

“Issues that are really just personal taste and don’t affect the correctness or readability don’t belong in a coding standard. Any professional programmer can easily read and write code that is formatted a little differently than they’re used to.”

They go on to say that you should try to use consistent formatting within a project, because jumping between styles is jarring. I agree in principle, but in practise, I’d go a little bit further than them. People are different. Experienced programmers can read code in pretty much any format. Experienced programmers also have years (or decades) of muscle memory which means that with the best will in the world, they’re likely to slip and use the formatting that they’re used to.

My advice would be - use common sense, and be forgiving. This cuts both ways of course. Whenever I’m working with other people’s code I try to adopt the style that seems to be prevalent in it. When I was younger I would dive in and reformat files - these days I just try to live with it. Having said that, when I write any substantial amount of new code, I’m not thinking about where to put the brackets, so I slip up and use my own style sometimes. What can I say? Live with it.

Thinking about all of this reminded me of something I’ve often thought. How bloody ridiculous is it that we’re still storing source code as plain text?

I don’t care if we input it as text, or view it as text - both are quite natural ways of expressing code (though things like Prograph prove that they are by no means the only way).

The thing I find annoying is that because we still store it as plain text, we can still get into arguments about whitespace, formatting etc, which is just ridiculous in this day and age.

If we stored it as structured text (eg xml), or some sort of binary format like a parsed syntax tree, then code editors would be free to present it to us for editing using our own formatting preferences, safe in the knowledge that it would be stored in a formatting-neutral way, so the next person coming along could view and edit using their preferences.

Of course, in theory we can do this now even with plain text source files, but in practise we can’t because everyone is so used to being able to open the text in any old editor and see exactly what they inputted, down to the last space or tab character. Which leads to polite disagreements about the position of curly brackets, etc, especially when one person accidentally changes what another person had done. Archaic or what?

And that’s not even to mention the obvious inefficiencies of parsing the bloody stuff every single time… we’re stuck in the 70s!

There are partial solutions to the formatting problem. For example, you can set up git hooks to reformat source files (using things like astyle) automatically when you grab them, and before you commit them. If I was running a coding shop that employed multiple people, that’s what I would do, as it would kill these sorts of arguments stone dead.

That sort of thing only works though if everyone buys into it. Automatic formatters tend not to pass the Turing test (!), so sometimes they get things wrong. If you’re running an automatic formatter on your machine alone, everyone else may not even realise it - and you will be perceived as an arrogant and ruthless re-formatter of everyone else’s code!

Anyway, I’m off - got lots of spaces to remove from my method declarations…

Neu 1.0.2 has reached the Mac App store

2011-01-27T00:00:00+00:00

I’m happy to announce that the 1.0.2 update for Neu has now been approved for the Mac Application store, and can be found here.

This brings it into line with the latest version that’s available from my website here. In future I will try to synchronise update releases in both places, to avoid confusion…

Mac Application Store - Frequently Asked Questions

2011-01-27T00:00:00+00:00

I’ve created a page detailing our answers to some frequently asked questions about the Mac Application Store.

In particular, this explains what the differences are between software you buy from our store, and from the Mac Application store.

Another Ambientweet update

2011-01-25T00:00:00+00:00

I’ve just released another new beta version of Ambientweet.

You can now switch between the main tweet stream and a view which only includes your mentions - so that you can catch up on what people have been saying about you. Ambientweet also bounces the dock icon and shows a badge when it spots new, unread mentions.

In addition, you can now set Ambientweet’s window to auto-size. If you do this, it will keep the width that you gave it, but change height automatically so that the full text of the current tweet is always visible.

Neu 1.0.2 submitted to the store

2011-01-18T00:00:00+00:00

I’ve submitted Neu 1.0.2 to the Mac application store… here’s hoping that the review process for updates is a little bit quicker than it was for the first version…

Ambientweet updated

2011-01-18T00:00:00+00:00

I’ve just released a new beta version of Ambientweet.

For those who don’t know, Ambientweet is a minimalist twitter client that I’m working on.

Unlike most clients, Ambientweet shows just one tweet at a time. This is a deliberate design decision, as it is intended to just be left running in the background, as a way of watching your twitter stream pass by. Showing just one tweet allows for a minimal user interface. This, in turn, means that the text can be bigger and there’s no need for extraneous clutter like toolbars or buttons whilst you watch the twitter world go by.

You can use Ambientweet to post, reply and re-tweet, so it’s perfectly adequate for adding to your stream and responding to what you read. What it doesn’t include is lots of complicated functionality for viewing profiles, following people, managing lists and accounts, and so on. The Twitter website is great at that stuff, so why try to replicate it?

Neu is now available on the Mac Application store

2011-01-13T00:00:00+00:00

I’m pleased to announce that after a rather lengthy review, Neu is now available on the Mac Application store. Unfortunately, due to the time lag involved in the approval process, the current version there is 1.0.1, whereas the version available for download here is 1.0.2.

Apple does not provide a way for developers to allow existing users of an application to “cross-grade” over to the Apple store version. So if you have paid for Neu via the Elegant Chaos store, please continue to download new versions through the built-in update mechanism. If, for some reason, you want a Mac application store of Neu, you will need to purchase it again. However, there’s no need to do that - your existing version will stay up to date (in fact, at the moment, it’s more up to date!).

If you bought Neu via the Apple store, please do not download the newer version from this website. Applications bought in the Apple store should only be updated via the store. The 1.0.2 update of Neu will be in the store just as soon as I can get it through the submission process.

I realise that this situation is a bit confusing and I apologise, but I had no way of knowing if/when Neu would be approved for the Apple store, so it wasn’t possible (and wouldn’t have been fair) to hold back the update for customers who had already paid for it.

In future I will try to synchronise releases in both places, to avoid confusion.

One day I may also choose to only sell through one or the other store, but that decision will wait until the developer community as a whole gets a bit more experience of the Apple store and how it’s doing.

NSConference 2011

2011-01-10T00:00:00+00:00

Once again I’m planning on attending NSConference 2011, which is happening in March this year.

It’s a great conference, and I’d highly recommend it if you are a Mac OS X or iOS developer.

You can find out more information here.

Best Wishes To You All

2010-12-24T00:00:00+00:00

Elegant Chaos would like to wish you all a fantastic Christmas.

Unless, of course, you don’t celebrate Christmas, in which case have a fantastic not-Christmas.

Unless of course, you recognise Christmas, but simply don’t celebrate it, because that misses the point, in which case, have an appropriately sober Christmas… look, just have a good few days why don’t you, for some appropriate value of good!

Oh, and we do hope that you have a great New Year and a brilliant 2011.

Unless of course you don’t follow the Julian calendar, in which case…

Neu 1.0.2 Released

2010-12-18T00:00:00+00:00

I’m pleased to announce the release of Neu 1.0.2.

This update contains a number of small changes:

Neu now handles the .neulicense file type. Double-clicking on a license file will automatically launch Neu and register the license.
Neu now respects symbolic links in the path to the Templates folder, and for individual templates.
Added Quit button to “Presence” preference when in “Stealth” mode, since the normal Quit menu item is unavailable.
Added tooltips to the preferences panels. Added application name to preference window title.
Window changes: added fades, increased the default size, made resizable,changed the buttons to a white-text-on-black-background style to match the windows.
Updated sparkle and release notes URLs.
Fixed old reference to Replicator in the Updating preferences panel.
Updated the in-application Help to reflect the current state of the website.
Changed the default Pages template to an iWork ‘08 version, and added a Numbers template (note that this will not affect you unless you throw away your existing templates folder, since these default templates are only used when Neu can’t find a templates folder).
Fixed a bug in the build process which was bloating the size of the application.

Beta Versions

2010-12-12T00:00:00+00:00

I’ve added a page to the Elegant Chaos website which lists the latest available beta versions of our software.

I’d like to avoid having people accidentally download betas without realising that they are getting untested software, so the main product pages will always link to the latest release version.

Which is why, if you’re the adventurous sort, you may want to keep an eye on the beta versions page.

Sneak peak of Neu 1.0.2b2

2010-12-08T00:00:00+00:00

I have a new version of Neu in testing, with a number of minor tweaks and bug fixes:

Fixed old reference to Replicator in the Updating preferences panel.
Neu now handles the .neulicense file type. Double-clicking on a license file will automatically launch Neu and register the license.
Updated the in-application Help to reflect the current state of the website.
Template selection windows now fade in/out.
Neu now respects symbolic links in the path to the Templates folder, and for individual templates.
Changed the default Pages template to an iWork ‘08 version, and added a Numbers template (note that this will not affect you unless you throw away your existing templates folder, since these default templates are only used when Neu can’t find a templates folder).
Fixed a bug in the build process which was bloating the size of the application.

It isn’t available via the in-application update mechanism, because I don’t want to dump beta code onto people who’ve paid for the application, but if you’d like to give the beta a spin, you can find it here.

Please let me know if you discover any problems!

Unit Testing with SenTestingKit with framework dependencies

2010-12-01T00:00:00+00:00

This is a bit of a technical one… you have been warned…

I’ve got some internal Objective-C frameworks, and they have some unit tests.

I’ve have targets set up for both SenTestingKit and GHUnit, because I like to have the choice of which one to run (I use SenTestingKit from continuous integration scripts, and GHUnit when I want to run the tests manually).

I recently came across a problem when I wanted to test one of my frameworks which uses another one of my frameworks, using SenTestingKit.

The problem is that SenTestingKit builds your tests into a bundle, then runs another application to load that bundle and execute it. This means that, by default, the application that is running your tests doesn’t know where to find any frameworks that your tests depend on.

There are probably a few ways of solving this, but the one I came up with was this:

First, add a copy files phase to your target, and make it copy any dependent frameworks into Contents/Frameworks/ inside the octest bundle that you’re making.

This gives you a known location containing a copy of any frameworks that you’ll need, but it doesn’t automatically tell the unit test running application to look there.

Second, edit the run script which actually invokes the unit test running app.

By default this script looks like this:

    # Run the unit tests in this test bundle.
    "${SYSTEM_DEVELOPER_DIR}/Tools/RunUnitTests"

The trick is to add our plugin’s Frameworks folder to the search path for frameworks, before running the tool:

    # Add the unit test plugin's Frameworks/ path to the search paths for dynamic libraries
    export DYLD_FRAMEWORK_PATH="${CONFIGURATION_BUILD_DIR}/${FULL_PRODUCT_NAME}/Contents/Frameworks:$DYLD_FRAMEWORK_PATH"

    # Run the unit tests in this test bundle.
    "${SYSTEM_DEVELOPER_DIR}/Tools/RunUnitTests"

Bingo - the unit test app now finds the dependent frameworks and runs the tests ok.

Neu 1.0 Released

2010-11-25T00:00:00+00:00

Neu 1.0 is now officially out of beta and released.

Neu is a simple application that makes it easier to create new documents in the Finder, without you having to open an application first. You can find out more here.

Although it’s a relatively small product, Neu represents a bit of a landmark for me as it’s the first piece of commercial software that I’ve released since giving up the “day job” and returning to life as an independent developer. As such it was a deliberately small project that was at least partially intended as a way for me to get up to speed with various technologies such as the Sparkle update framework and integration with the FastSpring online store. It’s also had to take a back seat for quite a while as I’ve been distracted with some other projects.

That said, Neu is a real tool that I wanted for my own use, and hopefully it will prove to be of some use to others.

So what’s next?

For Neu, I’ve had a couple of reports of minor cosmetic bugs, so I’ll work on a 1.0.1 release which should follow fairly quickly. After that, I’ve got a number of features that I’d like to fit into a 1.1 version, including support for more complicated templates, and possibly some smart detection of the “best” place to create new files when you aren’t working in the Finder. I’m also planning on trying to do a Mac Application Store edition of Neu - it’s possible that that will result in some changes to the application, which will also find their way back to the version I’m publishing myself.

Other than Neu, I’ve also got quite a few Mac and iOS applications in development, at various stages of completion! I’ll post some more news here as they get nearer to seeing the light of day. Currently I’m working on three things Twitter related (one on the Mac, two on iOS), but I’ve also got all sorts of other ideas. If there’s something that you want to see done though, feel free to send me your ideas too (although bear in mind that if you do, there’s every chance that I’ll run off with them and attempt to make money!).

Neu 1.0b23 Released - Hopefully it's a GM

2010-11-22T00:00:00+00:00

Fairly hot on the heels of version 1.0b22 comes version 1.0b23.

Essentially the only change for this version is that I’ve enabled the “Buy Neu Online…” button, which now actually takes you to our online store - where you can indeed purchase a license for Neu, for the princely sum of $10 (which works out at about GBP6.50 if you’re in the UK).

Barring any reports of problems I expect to officially anoint this version as release “1.0” in the next few days.

If you feel inclined to buy a license in the meantime, that would be splendid, and the license file that you receive should work fine with the b23 version and later versions. It’s probably worth me pointing out that this is the first product that I’ve sold through my new FastSpring store, and whilst I’ve every confidence that it will work ok, please don’t panic if you do encounter any issues. Just get in touch with me and I’ll sort things out for you right away.

Neu 1.0b22 Released

2010-11-18T00:00:00+00:00

I’ve just released a new version of Neu - 1.0b22.

You can now run Neu with neither a menubar or a presence in the Dock, and the Dock menu for Neu now has a slightly different set up of items in it compared to the status menu.

Double-clicking on an item in the grid view now behaves as if “Create” was clicked.

I’ve added a Hot Key for “Create and Open” as well as “Create”.

There’s a new preferences panel to edit which filetypes get keyword expansion applied to them.

I’m still working on the infrastructure for purchasing the application, and there’s now a reminder window for unlicensed copies. Whilst it is still free, if you would like to help me test neu, please download it. If you’d like a free license, to help test the registration system and to get rid of the reminder window, send me some feedback and in return I’ll send you a license file which will continue to work once the application has been released.

In addition to this I’m also working on a Mac Application Store version of the application, which I hope will be available when the store opens.

New, Simpler Site

2010-10-11T00:00:00+00:00

As you may have noticed, I’ve been trimming down the content of this site now that I’ve split my personal pages off onto Born Sleepy.

For now at least I’m aiming for a very simple front page with just a list of products and a small news section on the right-hand side bar. Other areas of the site are still accessible from the navigation links at the top-right, but even these should now be minimal.

Let me know what you think of the new look. Is it too simple? Is there some vital information missing?

My Forking Blog!

2010-10-08T00:00:00+00:00

I’ve decided to split off my personal pages and blog from this site, and move them to a sister site Born Sleepy.

For now the new site is a clone of the old one, but they will gradually diverge. I want to simplify the main Elegant Chaos website and remove some of the more personal stuff from it so that it’s a little less confusing for people who arrive at the site simply wanting to know about Elegant Chaos and the software we sell.

If you have an RSS/Atom subscription to my blog (or the whole of this site), you might want to head over to Born Sleepy and subscribe there instead.

For the sake of history, and to preserve old links, I will continue to host all previous posts on both sites, but as time goes on I will set up some permanent (http 301) redirections to try to teach the search engines etc where the definitive copy now lives.

Disruption Over (Hopefully)

2010-10-07T00:00:00+00:00

Well, the electrical work seems to have finished, and other that a bit of local DNS weirdness that I’m experiencing here, everything seems to be ok.

Let me know if you have any problems viewing this site.

Expect Disruption...

2010-10-06T00:00:00+00:00

We’re having some electrical work done here at Chaos Towers in the next day or two*, which I fear will mean that our servers may be coming and going a bit, since they are hosted locally.

So if everything disappears for a while on Thursday/Friday, fear not - it should be back by Saturday at the very latest.

(* it’s amazing how much power killer death rays draw - turns out that world domination can only be achieved with three-phase power**)

(** actually, that’s a lie, we just need a new fuse box***)

(*** or is it…?!)

What I've Been Working On

2010-10-01T00:00:00+00:00

You might have been wondering what I’ve been working on since leaving Sports Interactive.

Yes, ok, you haven’t been wondering at all, you’d forgotten that I exist, but I’ll tell you anyway…

The answer is “lots of stuff”.

One thing I’ve been doing is an iPhone application for my friends at Pod Point. Pod Point make cool high-tech charging posts for electric vehicles, and they wanted an application that allowed users to find the nearest post on their iPhone. Version 1 of the application is done, but isn’t in the store yet. Unfortunately there are some issues with having a product that legitimately has the word “pod” in its name…

Something else I’ve been working on is Neu, which I’ve blogged about a little bit before. It’s quite a small application, but I’ve used it to get up to speed with a number of Cocoa technologies. I’ve been juggling time between projects recently, but the proper release version of 1.0 should be coming just as soon as I finish sorting out my online store.

I’ve also been working on a number of my own iPhone projects. Some of these have been vehicles for learning about one or other iPhone technology, but others are intended to make it into the store as fully fledged products. The one I’m nearest to completing at the moment is code named “Ici”, and it’s a sort of special purpose Twitter client. There are quite a few other projects on the drawing board too.

All of this development has also resulted in me developing a fair amount of shared libraries and utility scripts, many of which can be found on either my public git repository or the Elegant Chaos git repository.

I’m the kind of programmer who likes to build tools and generalise code into libraries. If taken to extremes this can distract from the product that I’m working on, but it can also lead to serious increases in productivity. Computers are great at doing the same thing again and again, yet I’m always amazed by the number of programmers who are quite happy to continue to do tasks repeatedly because they don’t think they’ve got time to write a script to do it for them.

Of course there are times when you just have to knuckle down and finish the job you’re on, and I’ve certainly learnt something from the Extreme Programming mantra of “the simplest thing that works”, but like most things in life it’s a balancing act, and there are times when taking a day up front will save you a week or a month later.

Luckily, most of my time pressure at the moment is self-imposed, so I’ve had the luxury to write tools and generalise code into libraries. I really hope that this pays off, but even if it doesn’t, I’ve had fun along the way.

I’ve made some of this stuff open source, not because I particularly thing that it will be an off-the-shelf solution for anyone, but more in the general spirit of sharing. In particular my Cocoa libraries ECFoundation, ECTouch and ECAppKit are things that I am using myself, and will continue to maintain for my own benefit. They aren’t brilliantly documented at the moment, but my plans do include producing some better documentation eventually, for my own benefit and for the eventuality that I end up collaborating with other coders on a project that needs them. It would be hard for people to use all the code in these libraries right now without knowing what it does, but some chunks might come in handy. If you find something that you’re interested in, feel free to get in touch.

[Updated Feb 2012 to correct some broken links]

High Level Git Subtree Scripts

2010-09-30T00:00:00+00:00

A while ago in my post Managing Dependent Libraries with Git I talked about some shell scripts that I’d developed to help me to work with the git subtree command.

Since then I’ve updated them quite a bit and rewritten them in python, so I thought I’d revisit them slightly on the blog… There is now just one command line command ‘subtree’ which takes a number of options, like so:

> subtree <command> <tree>

Tree Configuration

The tree parameter is used to look up a configuration file at the path

subtrees/<tree>.subtree

relative to the root of the project.

The configuration files are very simple, and basically just define three variables which the commands use. For example, the ECFoundation.subtree file for my foundation library looks like this:

TREE="ECFoundation"
LOCAL="frameworks/ECFoundation"
URL="git://github.com/samdeane/ECFoundation.git"

The first variable gives a name for the branch that the git subtree system will use to track the module. The second variable gives a location to place the subtree, relative to the root of the local project. The third variable gives the URL location of the repository containing the master copy of the module.

Subtree Commands

Add

> subtree add <tree>

This command grabs a subtree from a remote depot and adds it to your project.

For example, when I first set up a new project and want to import ECFoundation, I first make a “subtrees” directory and copy into it ECFoundation.subtree. Next I cd into the root folder for the project in the terminal, and run:

> subtree add ECFoundation

If all goes well, I end up with a new folder called frameworks/ECFoundation/ containing the latest version of the library.

Push

> subtree push <tree>

This command pushes back local changes to the remote repository that the subtree came from.

For example, if I’ve changed ECFoundation in my project and I want to push my changes back to the master repository, I again cd to the root folder of the project, and run

> subtree push ECFoundation

Pull

> subtree pull <tree>

This command grabs any remote changes to the subtree and merges them into the local project copy.

For example, if I want to pull the latest version of ECFoundation into my project, I cd to the root folder of the project and run:

> subtree pull ECFoundation

Log

> subtree log <tree>

This command runs git log over the local directory containing the subtree.

Commit

> subtree commit <tree>

This command runs git commit on just the local directory containing the subtree. It’s a good idea to commit local changes to the subtree separately from other project changes, so that you don’t end up with weird project-specific commit messages when you push back subtree changes to the master repository for the subtree.

Status Of These Scripts

These scripts are still pretty simple (though more complicated than they were when I started), but they do the job for me.

You can find them on github here.

Comments and improvements to the scripts are welcomed!

Neu 1.0b20 Released

2010-09-08T00:00:00+00:00

I’ve just released a new version of Neu - 1.0b20.

Recent changes include a new “Launch at Login” preference, and a new UI for adding, removing and editing templates.

I’ve also started adding the infrastructure for purchasing the application - it’s free a the moment but I intend to charge something for it once it is officially released.

Whilst it is still free, if you would like to help me test it, please download it. If you’d like a free license, to help test the registration system, send me some feedback and in return I’ll send you a license file which will continue to work once the application has been released.

Lazy Properties In Objective-C

2010-07-23T00:00:00+00:00

Lazy Properties

There was one other motivation for thinking about all of the Objective-C property stuff: Lazy Initialisation.

Lazy Initialisation is a GoodThing™, especially in the iOS world where memory is tight, and performance isn’t always stellar.

So why the bloody hell didn’t Apple add a lazy attribute to the property system, so that you could do this:

@property (retain, nonatomic, lazy) id foo;

In my world, that would synthesise a getter which performs a lazy initialisation the first time it’s called, by calling a standard method (called fooLazyInit or something similar).

It seems to me that this would be a great boon to iOS developers. To some extent it also gets round the instance variable vs setter problem for the init method - the answer becomes simply not to initialise the property at that point.

All of this stuff has been mulling around in my brain for a while, so I was thinking, would it be possible to add some more property macros to my existing ones, like so:

ECPropertyDeclareLazy(name, type, attribute);
ECPropertySynthesizeLazy(name);

The answer is yes, it’s a bit tricky, but it is possible, and here’s how.

Implementing Lazy Properties

My main consideration was that I didn’t want the macro to have to write all the proper getters and setters, since that would mean losing all of the retain/copy/nonatomic goodness that @property gives us (or recreating it in the macros).

So the trick is to wrap the generated getter with one that checks if the instance variable is nil. If it is, it initialises it directly by calling a special initialiser method. It then just calls on to the generated getter.

The solution I came up with looks like this to use:

@interface MyClass
{
	ECPropertyDefineVariable(foo, id);
}

ECPropertyDefineLazy(foo, id, retain, nonatomic);
@end

@implementation MyClass
ECPropertySynthesizeLazy(foo, setFoo, id);

- (id) fooInitLazy
{
	return [[[SomeClass alloc] init] autorelease];
}

@end

If you don’t care about backwards compatibility with the old runtime, you can get rid of the ECPropertyDefineVariable bit.

The ECPropertyDefineLazy macro is defined like this:

#define ECPropertyDefineLazy(name, type, ...)		\
	@property (__VA_ARGS__) type name##Lazy; \
	@property (__VA_ARGS__) type name; \
	- (type) name##LazyInit

Here we actually declare two properties - one called foo, the other fooLazy. We’re going to let Objective-C synthesise fooLazy for us, so we get the right retain behaviour etc.

We also forward declare our fooLazyInit method here. It’s not strictly necessary, and we don’t actually want anyone calling it, but it allows the actual implementation of the method to appear anywhere in the .m file, which is more convenient.

The ECPropertySynthesizeLazy macro is defined like this:

#define ECPropertySynthesizeLazy(name, setter, type)	\
	@synthesize name##Lazy = _##name; \
	- (type) name { if (!_##name) self.name##Lazy = [self name##LazyInit]; return 	self.name##Lazy; } \
	- (void) setter: (type) value { self.name##Lazy = value; }

We let Objective-C synthesise the fooLazy method, but we ask it to use the foo instance variable. This isn’t strictly necessary (it doesn’t really matter what the variable is called), but it makes it consistent with the other property macros, and means that if you do try to access the instance variable directly it will be there and be called what you were expecting.

We then manually implement the getter and setter for the actual foo property.

The setter just calls on to the synthesised version. Annoyingly we have to pass in the name to use for this, since we can’t automatically turn foo into setFoo as it involves turns “f” into “F” in a macro. We also have to pass in the type of the property to the synthesize macro, since it needs it to declare our getter and setter methods. In an ideal world it could work this out for itself.

The getter checks the instance variable directly to see if it’s nil. If so, it calls fooLazyInit to get a value, and assigns it to the synthesised property. From then on, we just call on to that property to return a value.

To maintain the correct semantics for the property, we assign the result of the fooLazyInit method to the synthesised property using its setter. We could just set the instance variable directly, but that might not be atomic when it was supposed to be.

Assigning to the instance variable directly might also break the retain/assign/copy behaviour, so it’s safer to go through the setter. To stick to the standard naming conventions, fooLazyInit should really autorelease any object that it allocates. This isn’t ideal from an efficiency point of view, but it is cleaner. It allows us to call on to the synthesised setter and have it call retain if it’s supposed to. Bear in mind that fooLazyInit is perfectly allowed to return an existing instance, so this isn’t an irrelevance - we need to be certain that the retain/release semantics of fooLazyInit will always be consistent.

So What’s The Point Of All This

Essentially, the point is that I’ve now got a relatively easy, and totally consistent, way of declaring properties to be lazy initialised.

There is a performance penalty, of course - since we have to check the instance variable.

If used wisely though, that’s weighed against the performance gain achieved by not bothering to initialise properties until they’re needed - which may be never.

It also means that you can clear properties more aggressively in response to low memory situations. You don’t have to worry so much about the consequences when you know that the property will get created again on demand.

These macros are pretty new, and might turn out to be a complete cul-de-sac, but I’m going to give them a go.

If you want to try them out as well, I’ll upload them to github

Some Musings On Objective-C and Properties

2010-07-22T00:00:00+00:00

In The Beginning

Traditionally with Objective-C, if you wanted to define a property on a class, you did it by

adding a instance variable to store the value of the property
adding a getter method which returned the value of the instance variable
optionally adding a setter method to set the property

These getter and setter methods followed a simple naming convention - if the property was called foo, they would be called foo and setFoo.

The instance variable could be called anything you liked, since it would only be visible to the outside world via the getter/setter methods, which you wrote. By convention some people called it _foo, or mFoo, or just foo.

Here’s a concrete example.

@interface MyClass
{
id foo;
}

- (id) foo;
- (void) setFoo: (id) value;
@end

@implementation MyClass
- (int) foo { return foo; }
- (void) setFoo: (int) value { /* assign value to foo here, being careful about retain counts */ }
@end

One problem with this approach of course, is that most getters and setters are boilerplate code. The setter can be a bit complicated if the property needed to be retained, or copied, or atomic, so that’s one source of bugs - but essentially they’re the same pattern repeated again and again, which is tedious.

You also have to remember to initialise your property somehow, probably in your init method. This turns out to be slightly subtle. Calling your setter in init is a bit dicey, since a subclass might override it and you’ve no way of knowing if the new version of the setter is safe to run before init has completed. So generally people just perform an assignment to the instance variable here.

And then you must remember to release your property when dealloc is called, if you retained it. You could call your setter here, but again you can’t be sure it’s safe. It’s probably overkill anyway - the setter is likely to zero out the instance variable which is pointless when you’re being deallocated. So generally people just do access the instance variable directly and do something like: [foo release];

Finally, in your code, you have to remember to call the getter/setter, rather than just accessing the instance variable directly. Why? Well, if a subclass overrides your getter & setter to do something fancy, you will break the subclass if you just access the ivar directly.

This can get tricky though - don’t forget that the instance variable can be called the same thing as the getter. It’s easy to write x = foo instead of x = [self foo]. That’s one reason why people might call the instance variable _foo or mFoo instead - just to make it a bit clearer.

Oh, one other thing. Doesn’t it feel a bit like breaking the DRY principal, having to repeat the name of the ivar/property and also repeat the type? Surely this isn’t clever?

Objective 2 Properties

So Objective C 2.0 came along and introduced the @property, @synthesize and @dynamic keywords, as a way of taking the leg-work out properties.

Now, you can do this sort of thing:

@interface MyClass
{
	id foo;
}

@property (retain) id foo;
@end

@implementation MyClass
@synthesize foo;
@end

Which is really nifty. No more writing boilerplate getters and setters. What’s more, there are some handy attributes like assign,retain,copy,monatomic that you can tag your property with to tailor the generated getter/setter. And you still write your own if you need to.

Problem Solved Right?

So that’s all fine and dandy, but there are still some aspects of it that aren’t entirely great. This stuff just writes the getters and setters for you.

It doesn’t actually manage your instance variable. You still need to define a instance variable to store the property in. It can’t help you with your init or dealloc either. You still need to access the instance variable directly for these routines, and you still can’t assume that it’s safe to call the getter or setter.

And there’s still that confusion between the name of the property and the name of the instance variable. Along with the other changes came a new syntax for calling the getter/setter - you can now say object.property instead of [object property]. Still, it’s just as easy to write x = foo instead of x = self.foo.

You can tell @synthesize that you used a different name for the instance variable, but this is a bit of a pain, you have to write things like:

@synthesize foo = _foo;

This just looks ugly, and it feels like it is breaking the DRY principal again.

The New Runtime

So along comes a new runtime, and things move forward again a bit.

Now, you don’t actually need to define the instance variable at all, so you can do:

@interface MyClass
@property (retain) id foo;
@end

@implementation MyClass
@synthesize foo;
@end

Now we’re motoring! No more repeating the name and the type in multiple places.

We’ve still got the instance variable though, it’s just generated for us by the compiler. But we can’t get at it unless we declare it ourselves, so I guess we sort of don’t have it, philosophically speaking. Which means that we have to just bite the bullet and call self.foo = blah in our init method, and self.foo = nil in our dealloc method.

Hang on though. Wasn’t that supposed to be bad? Well, uh, yeah, maybe. So does this step forward make it ok?

(tumbleweed passes)

Ok, so this is a bit weird. It seems that the answer is “it’ll be alright, probably”. Just call the setters.

This is kind of good news, but leaves me feeling a bit uneasy.

There’s another problem. If you’re writing shared code, you might not be able to rely on the new runtime, so you might have to declare the instance variable anyway. So now you’ve still got the problem that you can accidentally use it directly.

There’s another more subtle problem. If you used to do foo = [[SomeClass alloc] init] in your init method, and you retain foo generally, that was probably the right thing to do. But if you do self.foo = [[SomeClass alloc] init], that’s probably not the right thing to do if foo is retained. You probably want self.foo = [[[SomeClass alloc] init] autorelease] (which isn’t very efficient), or to call release explicitly afterwards (which is ugly, and easy to forget).

What’s A Poor Boy To Do?

So all of this has me wondering quite what the official way to behave is, right now.

Currently this is what I do:

declare my instance variable explicitly
call it mFoo (I use the m prefix because I have many years of C++ code behind me)
access mFoo directly in init
use @synthesize foo = mFoo
release it directly in dealloc with [foo release]
try to remember to use self.foo elsewhere

Future Proofing With Evil Macros

So, feeling a bit nervous, I started wondering whether I should wrap this stuff in some macros. That way I can change my approach later without having to rewrite all my code.

Yeah, I know, the C preprocessor is the work of the devil - I completely agree - but since this isn’t Dylan we’re working with here and we don’t have a nice clean macro system for defining new syntactic sugar, there’s not much other alternative.

Surely it ought to be possible to do something like:

ECPropertyDeclare(name, type, attributes);
ECPropertySynthesize(name);
ECPropertyInit(name, value);
ECPropertyDealloc(name);

Maybe?

Well, of course, it’s not that simple. One of the main reasons for this is that the whole damn thing is a hack anyway. Objective-C I mean. Really, if you were designing a syntax, you wouldn’t start from here would you?

First problem: the instance variables and the properties have to be declared in different places - separated by the trailing closing brace of the instance variables section. So you can’t define a variable and a property with one macro. Arse.

So we need another macro: ECDeclarePropertyVariable(name, type). So much for the DRY principal.

Still, I’ve actually implemented this, and it sort of works.

I say sort of, because there are two more stumbling blocks.

Second problem: XCode doesn’t apply the preprocessor when parsing the code for its fancy popup menus and whatnot, so it doesn’t really know that your properties are properties anymore. Not great, but not a killer.

Third problem: Interface Builder has no bloody idea at all about the macros. You can put IBOutlet into the macro, or as part of the type passed to a macro, or next to a macro, and it makes no difference. Interface Builder scans the text file, it doesn’t run the preprocessor.

This is a major pain in the arse. There is a workaround, but it’s a bit crap. Essentially you need to repeat the definition of the properties just for IB’s benefit, like this:

#ifndef ECPropertyDefine
	@property () IBOutlet id foo;
#endif

The compiler will ignore this, but IB won’t, so it’ll work out that you’ve got some outlets. It doesn’t really care about the property attributes, so you can omit them, but you still have to repeat the type of the property. Another nail in the coffin of the DRY principal.

Apart from all that though, you can sort of get away with this. Whether it’s any better than just doing it all by hand, I’m really not sure. The jury is still out. One thing though - if I do decide to rip them out, it’s going to be a lot easier to use a regular expression to expand my macros out of existence than it would be to rewrite any hand-written property code, which might well be less consistent.

Macro Definitions

Here are the basic macro definitions:

#define ECPropertyDefineVariable(name, type)			type _##name
#define ECPropertyDefine(name, type, ...)			@property (__VA_ARGS__) type name
#define ECPropertySynthesize(name)				@synthesize name = _##name
#define ECPropertyVariable(name)					_##name
#define ECPropertyDealloc(name)					[_##name release]
#define ECPropertyInit(name, value)				_##name = value

They use _foo for the instance variable. I’d prefer mFoo myself, but that’s impossible. Why? Because it would involve capitalising the first letter of the name of the variable in a macro. Arse.

Coming next… the real reason I did all of this…. Lazy Properties…

Neu 1.0b17 Released

2010-07-17T00:00:00+00:00

I’ve been beavering away on Neu, and the result is version 1.0b17, which you can download here.

There are three main changes for this version:

First, I’ve changed the “Create Document…” and “Create and Open Document…” menu items. They are now submenus which contain a “Choose Template…” command, but also contain a list of the actual templates. This allows you to choose a template directly without going through any additional dialogs.

Second, I’ve added these menus to the Neu menu in the Dock, and the small status menu that Neu can optionally show on the top right of the menu bar. This means that you can use these menus instead of the Finder’s Services menu. Since the Services menu is a bit clunky and can’t contain dynamically generated content, this is a Good Thing(tm).

Finally, I’ve added “Protect from accidental Quit” option which changes the Quit menu shortcut to Cmd-Shift-Q instead of Cmd-Q. This makes it harder to accidentally quit Neu, if you want to leave it running all the time.

Replicator renamed

2010-07-14T00:00:00+00:00

I discovered that there’s already a piece of software out there called Replicator, so it now has a new name.

In fact, it doesn’t just have a new name, it has a neu name, since the new name is Neu.

Still with me?

Visit the Neu (the application formerly known as Replicator) homepage for more details.

Update:

I forgot to mention that the change of name broke the auto-updating mechanism, so you will need to manually download it again.

Neu 1.0b15 Released

2010-07-14T00:00:00+00:00

Hot on the heels of the previous blog post… it turns out that there was a bit of a major bug in 1.0b14, so 1.0b15 is now out.

If you managed to download 1.0b14, then the automatic update process should get you to 1.0b15.

If not, you can download the new one here.

Visit the Neu (the application formerly known as Replicator) homepage for more details.

Replicator 1.0b13

2010-07-10T00:00:00+00:00

I’ve just released the latest update to Replicator, a utility that lets you create new documents directly from the Finder, using predefined stationary.

The new version (1.0b13) lets you choose the template to replicate using a grid view, rather than a list view, which looks a bit cleaner and reduces the likelihood of nasty scrollbars appearing.

If you’ve got an existing version, it should pick up the update automatically when you next run it. If not, you can grab it from here) page.

Managing Dependent Libraries with Git

2010-07-07T00:00:00+00:00

When I’m developing something, I like to split out related functionality into modules or libraries whenever I see the opportunity, so that I can share code between projects and generally promote re-use.

Doing this presents a number of interesting challenges, one of which is how to manage changes to these shared modules.

If I have three projects using some module, I can either choose to have them all link to the same version of the module on my disk, or I can give each project its own copy.

On the face of it, linking to the same version sounds like the way to go. After all, we’re sharing right? If we make a fix to the shared module, we want to get it in all projects don’t we?

Well, sort of, but not really. It’s how I used to do it, but not any more, and I’ll explain why in a moment. First though, a diversion:

Code Locations

Even if you do want to do it, there is one problem with having multiple projects link to a common module, which is the need to know the relative location of that module from each project, in any given development environment, so that you can get include paths etc set up correctly.

If you are the kind of person who is happy to just hardcode the location of the source code into your projects and force every developer who ever works on it to set up their disks like you do, then you can stop reading at this point. You are a bad person, and I don’t want to work on your projects!

If you are sane, however, then this poses a little bit of a problem. The way I used to get round this was to organise all my source code by reverse domain, in the java style, relative to some nominal root location (in my case, this was the “projects” folder, which could be anywhere on the disk). So all of my code lived in projects/com/elegantchaos, anything I was using from Matt Gemmell would live in projects/com/mattgemmel/, and so on.

Whilst this gets a bit cumbersome in some ways - the directory hierarchy can get quite deep - it does neatly solve the relative location problem. I can place the projects folder anywhere on my disk and as long as my project uses relative paths, it can still find anything that it’s depending on since it knows that the relative path between the two will remain constant.

I do still organise my work directory this way, but not specifically to solve the shared module problem, as I’ve realised that the best thing to do is for each project to have its own copy of the shared module, and to manage the propagation of changes between the projects using source control, which brings us back from the diversion…

Take Your Own Copy

So why is it better for each project to have its own copy of the shared module? The reason is fairly obvious really - and has to do with the law of unintended consequences.

Basically, if you set things up so that you are physically sharing the same copy of code between multiple projects, it doesn’t change the fact that you are always working on that code in the context of a specific project at any one time. That means that you’ll make changes and check that they work for the project you’re working in, but you won’t necessarily have time to check the other projects. You especially won’t have time to fix the problems that you’ve almost definitely introduced in the other projects without meaning to - if, that is, you are lucky enough to have created problems that are sufficiently obvious that they come to light immediately.

To a certain extent you can mitigate these problems with nice things like unit tests and continuous integration, both of which I’d highly recommend. They will help you find the problems that changes to shared code cause, but they still won’t alter the fact that your immediate deadline is based on a particular project and you just need to get it working now dammit, and clean up after yourself later.

This is hard enough to manage if you’re working on your own, but add in a team and/or multiple platforms, and it becomes almost unmanageable to share the same code. Imagine if, before checking in any change to your shared module, you had to test every permutation of every project that your team is working on, on every platform you support, to make sure that you haven’t broken anything. You could, quite literally, do a one hour code change then spend the rest of the day integrating and testing (I’ve been there, and it’s not fun).

All of which is a long winded explanation for why it makes sense to have a local copy of the source code for a shared module in each project that uses it: doing things this way gives you control over when you take changes to the shared module from other projects, and when you push back your changes to the shared module to them.

When you’re working on the change, you can just concentrate on getting the job at hand done and ensuring that your project is working.

Later, when the dust has settled and you have some allotted time to work on the library, you can integrate your library changes up from your project into the “master” copy of the library and perhaps perform additional testing or cleanup to ensure that it fully meets the standards and guarantees set out by that library.

Later still, at a point that is entirely convenient and safe, other projects can grab the modified library, integrate it into their source base, and resolve any issues.

How I Cope With This In Git

So that’s all fine and dandy, but how do we set things up in Git to make this possible?

Lets say that I have two projects A and B, and shared module X.

What I want is a repository for A, a repository for B, and a repository for X.

I want the project A and project B repositories to contain their own copy of module X so that any changes to it are isolated to just that project. This also means that someone else can just grab project A from a server and get everything they need to build it.

However, I want be able to push and pull from either project’s copy of module X to/from the master module X repository so that I can propagate changes.

Essentially there are (at least) two ways to solve this in Git: Submodules and Subtrees (scroll down to “Subtree Merging”). There are pros and cons with both approaches which are beyond the scope of this post, but are well described in the Pro Git book (follow the links above).

For a number of reasons I’ve chosen to go with the subtree approach, which means that I have installed the git-subtree support, and then have to execute various git subtree commands when I want to work with a shared module.

What it boils down to is that there are three module related actions that I typically have to do in a project: add a module, pull a module, push a module.

To make life simpler, I’ve made myself some shell scripts: subtree-add, subtree-pull and subtree-push. They expect to be run from the root of my project repository, and they expect to find a subfolder called “subtree”, containing a configuration file for each module.

The configurations are very simple, and basically just define three shell variables which the scripts use. For example, the ECFoundation.subtree file for my foundation library looks like this:

TREE="ECFoundation"
LOCAL="frameworks/${TREE}"
URL="<path to your repository>/ECFoundation.git"

The first variable gives a name for the branch that the git subtree system will use to track the module. The second variable gives a location to place the subtree, relative to the root of the repository. The third variable gives the URL location of the repository containing the master copy of the module.

So when I first set up a new project and want to import ECFoundation, I first make a “subtrees” directory and copy into it ECFoundation.subtree.

Next I cd into the root folder for the project in the terminal, and run:

> subtree-add ECFoundation

If all goes well, I end up with a new folder called frameworks/ECFoundation/ containing the latest version of the library.

Later if I want to push changes I’ve made to ECFoundation in my project, I again cd to the root folder of the project, and run

> subtree-push ECFoundation

Finally, if I want to pull the latest version of ECFoundation into my project, I cd to the root folder of the project and run

> subtree-pull ECFoundation

These scripts are pretty simple at the moment, but they do the job.

You can find them on github here.

Comments and improvements to the scripts are welcomed!

Update: These scripts have been updated and rewritten in Python. See my post High Level Git Subtree Scripts for details.

Anonymous Comments Broken

2010-07-02T00:00:00+00:00

For some as-yet-unknown reason, anonymous comments don’t seem to be working on my installation of Drupal 6. Which is a bit of a pain in the arse.

I’ll try to figure it out, but for now, if you want to comment on a post, you’ll have to register (which is quite safe - I don’t use the email addresses or do anything else nasty with your data).

Update

I’ve worked out what was going on - somehow my anonymous user had got removed during my upgrade and server migration.

These Drupal threads helped:

http://drupal.org/node/243423
http://drupal.org/node/501286

The solution was just hack the users table manually and add back a user with uid = 0. Scary!

Upgrade Complete

2010-07-01T00:00:00+00:00

Right, hopefully I’ve finished with upgrading my server and this website is back in the proper place.

If you notice any broken links, please let me know…

Content Coming And Going?

2010-06-29T00:00:00+00:00

If you’re noticing that some content on www.elegantchaos.com keeps appearing and disappearing mysteriously, it’s because I’m switching back and forth between the original version and the new Drupal 6 version which is running on an old (and slow) server.

Apologies if this confuses your rss news reader! Eventually I’ll finalise the update and switch over permanently to Drupal 6, at which point it will go onto my proper server and things will stop changing!

Drupal 6 Experiment

2010-06-27T00:00:00+00:00

As an experiment, I’ve ported this site across to Drupal 6.

The new version of the site is running on a test server at the moment, so you might notice a bit of a drop in performance, and one or two old links may be missing since I’ve not bothered to copy the old static content over yet.

Hopefully though, the basic stuff will work. I’m also hoping that moving to Drupal 6 will allow me to re-enable comments, and to add various other bits of functionality.

If you notice anything that seems to have gone badly astray, give me a yell!

A little project I've been working on...

2010-06-10T00:00:00+00:00

I’ve got a number of projects on the go at the moment, but most of them aren’t really ready for prime-time.

I do have one small Mac OS X project that I would like to get some feedback on.

It’s called Replicator (at the moment), and it is a simple utility intended to give the Finder a “Create Document…” menu that allows you to right-click in any folder and make a new file of a given type.

You can find the details here, and download a free beta copy to test. And find bugs. And, hopefully, give me some feedback about what I’ve done wrong and how to do it better.

Software Update Errors in Snow Leopard

2010-03-31T00:00:00+00:00

Occasionally I’ve had errors where Software Update refused to download something, telling me that I didn’t have permissions to save the updated file.

In previous systems I think these files went into the /tmp folder, but in Snow Leopard they don’t, and I’ve finally managed to work out where they do go - /Library/Updates.

On my machine, somehow, I’d ended up with a couple of folders inside this one which were owned by another user, and which didn’t have group write permission.

To fix the problem, I did the following in a terminal window:

sudo chown -R :admin /Library/Updates
sudo chmod -R a+rwx /Library/Updates

You need to replace with your actual user name. In fact, I suspect that it doesn't matter who owns the files as long as they belong to the admin group, so if you want to be a little bit more security conscious, you could probably do this:

sudo chown -R :admin /Library/Updates
sudo chmod -R g+rwx /Library/Updates

Coding Standards

2010-03-27T00:00:00+00:00

I’ve been thinking about coding standards again recently. For me a coding standard should definitely be more than just about formatting and naming conventions. That stuff matters, but it’s also relatively trivial to agree upon and then stick to.

Essentially I think the standard should clarify “the way we do things here” (wherever “here” might be for you).

The objective should be twofold:

to educate everyone in your team, and teach them about good techniques and common pitfalls
when there is more than one way to do something, to clarify which way to choose, so that code is designed in a consistent way and a programmer working in any area can intuitively work out how another area is likely to work, where to look when there are problems, and how to approach extending an area or adding a new one

My suspicion is that the exact choices that one makes when designing and implementing something often aren’t the most important thing. In fact, too much choice can (and often does) paralyse me. Obviously avoiding really stupid decisions is a requirement, but after that, there may well be a number of adequate solutions to a problem.

The important thing when working in a team is consistency. If you’re consistent, and agile, you can move fast. If everyone can understand the designs and the code, it will always be easy to revisit the occasional choice that turns out later to be wrong.

To achieve that, you need good standards, and you need good discipline. People need to buy into the idea of a standard, and try their best to stick to it, even the bits that they might not like.

In my experience, once you let go and just follow the standard, it’s amazing how quickly a lot of those disagreements seem to become trivial and you are left free to concentrate on the important stuff.

That doesn’t mean that designing a standard for a team is trivial. It’s pretty tricky, but very important. Once it’s done though, just go with it.

I'm Going Indy!

2010-03-05T00:00:00+00:00

After three and half action-packed years of my second stint at Sports Interactive (which makes it about six and a half in total by my reckoning), I’ve decided that the time is right to take the plunge and go it alone.

So from June 2010 onwards, I will officially be an independent Mac and iPhone/iPad developer!

No doubt I’ll blog more about my plans as the time approaches, but for now I’ll simply say that I’m really excited about the prospect of returning full time to Mac/iPhone development, and working on my own applications.

I’ll also be up for a bit of collaboration and/or consulting from time to time, so by all means get in touch if that appeals.

NSConference 2010 - day 2

2010-02-03T00:00:00+00:00

Day two started with an illuminating session by Jeff LaMarche about the Objective-C runtime, and some of the nice dynamic and introspective things you can do with it, such as iterating through your own properties, methods, etc. A couple of days ago in Drew’s concurrency workshop I was thinking about the need for some good Objective-C wrappers for OpenCL, as there’s a lot of boiler plate setup code that you have to do. It’s there for flexibility, but probably you do the same thing for most apps, so why not wrap it in some simpler objects. Some wrappers probably already exist somewhere, but it struck me that one of the techniques Jeff was describing would be perfect for supporting OpenCL kernel parameters. Instead of having to write code to poke the parameters into the kernel, wouldn’t it be great to just be able to subclass an OCLKernel class, define some properties on it, and automatically have setters implemented which loaded the property values into OpenCL, and getters which retrieved the values.

Next up was Andy Finnell, giving a session that was supposed to be about Core Image, but was actually more about OpenCL. Everyone reported that this was an interesting session, but I sort of got distracted because I was starting to hack together the OpenCL wrapper classes that I’d thought of in the previous session (I’ll post them up when I’ve done them). Having said that, if there’d been more actual OpenCL in Andy’s presentation I would probably have been drawn back in to it, but actually he mentioned himself that he’d wrapped the calls in a class hierarchy, and went on to focus on the physics, which whilst interesting wasn’t really of much general use. In the end, from what I heard, it was more about the physics of watercolour pigments and their interaction with paper - which whilst interesting, wasn’t really what I was there for!

Next up were two half-length sessions thrown together at short notice because one of the other speakers had to drop out.

First a quick overview of some source control systems by Dave Dribin. This was a tough job in such a short time, and I felt that it was a bit light on the right details. He spent too long on the purpose and history of version control, and not enough time on a discussion of the idioms that we use in the real world, and how they do or don’t fit into the models provided by Svn, Git, Mercurial or Perforce. Not that the history and purpose isn’t interesting, but in half an hour you can’t really do it justice on its own, so it was probably better to focus on telling people who understand version control and use one every day how the one they use compares to the other major options. I found this frustrating because I use a very large Perforce system at work (it’s pricey, but excellent, and I’m not convinced that the other systems could do what we do with it), and run a Subversion server at home (which is simple, but I want to change to something else, probably git or mercurial). What I really wanted was answers to some fairly complex situations that happen in the real world - like “how well can you manage multiple concurrent release branches”, “how does it cope when you have twelve devs continuously hacking on a branch”, “how well does it deal with large amounts of data - e.g. graphics assets”, and “how does system XYZ recommend managing your own shared library modules that are used by more than one project that you also have under source control?”. None of which were really answered.

This was followed by a very whistle stop tour of code signing from Graham Lee. Although brief, this was actually useful as he told me how to do something that I didn’t know how to do before. He delivered it with his customary humour, and the session was most notable for re-christening Wolf Rentzsch as “The Dog Spanner”.

After another excellent lunch we had our final “proper” session, from Aaron Hillegass, on data persistence. He started with an excellent set up, posing some interesting questions about how relevant the file system is any more, and whether “the file” is actually dead, to be replaced largely by the data cloud. Quite possibly, he suggested, our local machine hard drives are just local caches from here on in. I think this is a great topic which could have been explored further - for example, if files on our local hard drive are just caches, shouldn’t we be trying to find ways to structure them better so that they can be synchronised with the cloud in a non-monolithic way. One big fat 10Mb file is a bit hard to sync unless you’ve got a detailed schema describing the internal structure, and permission to mess with it. Anyway, I think that side could have been explored further, but Aaron switched tack somewhat to talk about the BNRPersistence framework that he’s created. Which was an interesting topic in it’s own right.

Finally, we had the “Cocoa Rumble”, which was a light hearted competition between three teams composed jointly of the session presenters and volunteers from the crowd. I didn’t really feel that it worked that well this year, despite generating some good presentations at the end. Somehow I don’t think we’ve yet managed to find a way to end NSConference which matches the old “Stump The Experts” sessions at WWDC (which were kind of cool back in 1993 when I first went, and still good fun back in about 2002 when I last went!).

So there you have it - two days of great Mac related sessions. I’ve learnt a fair amount, met some nice people along the way, and had a splendid time. I’ve eaten far too much, and drunk lots of bad beer and bad cider. And tomorrow, there’s another whole day of iPhone stuff. Phew…

Index of regular UK Mac/iPhone developer meetings

2010-02-03T00:00:00+00:00

Speaking to various folks at NSConference, I realised that there are quite a few regular developer meetings around the country, in addition to the one that I used to organise in London (which still exists but has mutated into the NSCoder london night).

I figured that it would be handy to list them all in one place, for anyone who might find themselves in a strange town for a while in need of geek-based social contact…

I’ve updated my uk-mac-dev-meeting page accordingly, so that it now lists all of the ones I’ve heard about.

If you know of one I’ve missed, please let me know.

NSConference 2010 - day 1

2010-02-01T00:00:00+00:00

Day 1 of the conference was excellent, though I spent the second half of it feeling absolutely shattered, having had less sleep over the last week than I would have liked through a combination of illness, early morning starts, and perhaps a tad more beer than was wise. Which is why I’m writing this now in my room, when by rights I should be in the bar.

We started in a refreshingly human manner, with talks by Scotty and then Mike Lee, focussing on the big picture rather than the nitty gritty details. Of course the details are our meat and drink, and we’ll get plenty of them this week, but it’s good to have our eyes dragged off the floor and onto the horizon every now and then.

I particularly liked Mike’s talk, as much for the effect that it had on the room by confounding our expectations. It was emotional, and personal, and reminded us that there’s more to life than programming, but also that the skills that we love to use in our programming can be applied to life.

I think it wrong-footed a few of us introverted programmer types, who were expecting nerdery and got philosophy instead. “What is this emotion of which you speak, human? I have no time for this frippery. Take me to your loader (sic)”…

Perhaps it was also just a bit too heartfelt for us cynical Europeans, who get a bit confused when someone is open and honest with us. There’s usually a little voice in the back of our head wondering if we’re having the piss taken out of us in some way which is so fiendishly subtle that we haven’t worked it out yet. Not so in this case. Anyway, good on you Mike, nice job.

Luckily for those in the room teetering on the edge of an existential crisis, this was session was followed by a full on geek fest from Wolf Rentzsch, who gave us a whistle stop tour of the tools and techniques needed to rummage around on the internals of OS X and applications written for it. Hurrah for insanely micro-focussed detail. Now were back on familiar ground. There were some excellent pointers, particularly tantalising for me since there are any number of really irritating things about XCode that I could probably sort out now given sufficient time. Like why the fuck my Data Formatter plug-in only works when there’s a full moon.

Rentzsch’s talk was followed by a session from Dave Dribin on clean code, which I’ve already blogged about (http://www.elegantchaos.com/node/405), so enough said here.

After which was another good lunch, including another far too nice pudding. May have to hire a people carrier to haul me back to the station at the end of the week…

One thing I definitely think that they’ve got right this year is the amount and length of breaks between the sessions, and the overall pacing, with just two sessions in the afternoon. Today these were a talk by Drew McCormack on data presentation, and a brief overview of Core Animation from Marcus Zarra.

Drew’s talk started with an overview of some existing ways to present data, but was interesting to me mostly as an introduction to a framework he’s been involved in creating for graphing and plotting. This looked like a great resource when you’ve got any set of data that you want to display as something better than a big table of numbers.

Marcus’ talk was tantalising, as I felt I knew a bit more about what Core Animation was than he assumed, and was hoping for a little more in the way of code examples or teccy stuff.

To be fair though, it was late by then, and he probably pitched it perfectly for the level of fatigue we were all experiencing at that point. And I guess if I want the details I can buy his book :)

All in all it was an excellent first day. The speakers and the general conversation was of the highest quality - something that came as no surprise after last year, but which still takes a great deal of effort to sustain.

So many thanks to the organisers one and all. I can’t wait for tomorrow.

Clean Code at NSConference

2010-02-01T00:00:00+00:00

Dave Dribin was preaching to at least one of the converted (me), in his “Clean Code” session at NSConference today - but then I have to wrestle with very large bodies of C++ code every day, much of which was written by other people not following any of his guidelines!

Most of the stuff he said was uncontroversial, but it was amusing to see a few people in the audience bristling at one or two of the suggestions that Dave made - you could hear the cogs turning in their brains as they thought “but I do that all the time, and I’m smart, so he must be wrong”.

I can’t really offer an opinion on KVO vs Delegates, not having done enough Cocoa work, although I think it’s all a storm in a teacup anyway as he was simply offering a personal order of precedence. Having had to debug very complex systems I can see where he was coming from though. KVO sounds like a lovely way to decouple systems, but it can get scary when just “setting values” can cause a rich ripple of side effects that extends way beyond the object you thought you were playing with.

I do think Dave’s right about singletons - at the end of the day they are globals, but somehow we’ve ended up with design pattern books telling us about them, and people like Scott Meyers telling us how to implement them efficiently, and it’s blinded us to the fact.

The point that some people seemed to be missing was not that they aren’t useful (they can be), or even that they aren’t the right or pragmatic solution to some problems (they can be). It’s just that they often aren’t the best solution, and yet it is dangerously easy to fall into the trap of making them when you don’t need to - I know, I do it all the time, with the best of intentions. And Dave was right - using a singleton you can easily end up with annoying hidden dependencies and inflexibilities which you can avoid if you instead use a pattern where you pass in the services/resources that you need. And it does make life particularly hard if you want to do test driven development and unit test the hell out of everything.

I never felt that Dave’s tone was in any way preachy, but maybe some people wondered why he was telling them stuff that “we already know”. Perhaps it’s also just a very healthy skepticism in the audience towards received wisdom, which is fair enough - I’m all for that. But just because it’s received wisdom, it doesn’t always make it wrong, any more than it makes it right!

In any case I’m not at all convinced that everyone in the room does know all that stuff. There were plenty of indy developers in the room, who are working on there own a fair amount of the time, many of whom are 25 or younger. I’ve also met plenty of developers in my life who’ve been doing coding professionally for years and still haven’t got a clue!

When I was 21 and just out of college I was lucky enough to get a great job working in an university research department, writing educational software in Hypercard (with plenty of lovely XCMDs that I wrote too). It was a brilliant job and a fantastic environment in many ways, with loads of freedom for someone just out of college. At the tender age of 21 though, I was the only full-time developer and with the benefit of hindsight I realise that I missed out on a great deal of stuff that was going on at the time in the wider software engineering world, because there was nobody at work to point me in the right direction. I was totally focussed on what I was trying to make, and not on the process of how it got made. Things are a lot better now than they were in 1992 of course, and it’s a lot easier to stay in touch with the current trends in computer science through the sort of social networking that we now have. But you still occasionally need people to tell you stuff, even stuff that you’re supposed to already know.

There’s definitely no harm in being reminded of some of the basics from time to time, and having some of your assumptions challenged - whether you’ve been doing programming for 1 year or 5 years or 30.

The power of tools (especially Instruments.app)

2010-01-13T00:00:00+00:00

For a couple of weeks the Console application on my laptop has been crashing every time I started it up. This was a bit weird, and also very annoying, since I use it from time to time whilst developing, and to check that the system isn’t doing anything particularly heinous.

The stack for the crash log appeared to be in a call to the file system, so I figured that the problem was file related in some way. I tried removing the Console preferences, to no avail.

After a bit of thought, I realised that I could probably use Instruments (part of Apple’s Developer Tools suite) to work it out.

So I launched it, chose the “File Activity” template, and set it up to run Console. This is a great way to see what an application is doing with files.

Lo and behold, the last thing Instruments did was open a file in /var/log/, then repeatedly try to open some file in /.vol/.

Doing a quick ls of /var/log/ revealed a symbolic link to something that I thought that I had removed (some crappy VPN software called SecureClient that I used to rely on but don’t any more as it doesn’t work on 10.6 anyway). Further investigation revealed that the symbolic link pointed to another symbolic link, which pointed… to itself. That would do it!

All in all it took about 5 minutes to find and fix this crash once I realised that using Instruments was more sensible than just sitting around guessing what might be causing Console to crash…

More On Doxygen From XCode

2010-01-13T00:00:00+00:00

I posted last week about running Doxygen from XCode using a custom build step.

I originally found a nice article on the Apple website here which describes how to do this, but it needed a little bit of tweaking, and wasn’t great, so I’ve improved the process a little bit.

First of all, I moved the bulk of the work into a script file, so that I can reuse it in multiple projects. This seems more sensible than cutting & pasting into every project. DRY principle!

I also decided to use a standard layout, relative to the project file, which meant that I could cut down on the number of variables that needed defining per-project.

My custom build step now just has to define a couple of variables and call on to the script.

You can find my version of the script here:

http://github.com/samdeane/code-snippets/blob/master/scripts/build-doxygen.sh

It could be cleaned up a lot more - it’s basically copied from Apple’s script then hacked a bit - I didn’t write it with public review in mind! One thing I fixed in the script was support for projects that have a space in their name or the name of a containing folder. Bloody unix programmers!

Second, I was annoyed by the output of the script, so I piped it into a file.

Finally, I was also annoyed at having to wait for the script to complete. Rebuilding the documentation each time is useful, but it’s not like you actually want to go and use it immediately.

So, I tweaked the build step to run it in the background.

Now my build step looks like this:

${WORKROOT}/scripts/xcode/build-doxygen.sh > “$TEMP_DIR/doxygen.output.log” 2> “$TEMP_DIR/doxygen.log” &

$WORKROOT is a variable that points to the root of my development folder - you can just replace it with the path to the build-doxygen.sh script file.

The script file itself relies on one extra variable - $DOXYGEN_ID - which I define in the project settings. It uses this to name the documentation bundle - so you should set it to something appropriate for the product - like com.yourcompany.yourproduct.

If you want to see the output of the script, you can add a second line:

open “$TEMP_DIR/doxygen.output.log”

This will launch Console and open the log file. Console is smart enough to update as the log file gets filled in, so it works even though the script may not have finished running by the time you open the log.

Update: one more thing…

By default XCode runs the build step every time. You can tell it to be a bit smarter by telling it what the input files and output files are for the step; it will then dependency check them.

I set the inputs to:

$(SRCROOT)/Code
$(WORKROOT)/scripts/xcode/build-doxygen.sh

And the output to:

/Users/$(USER)/Library/Developer/Shared/Documentation/DocSets/$(DOXYGEN_ID).docset/Contents/Info.plist

Now, if you do something that doesn’t change the source code, the documentation shouldn’t rebuild.

Running Doxygen automatically over an XCode project

2010-01-06T00:00:00+00:00

I was wondering whether to use HeaderDoc or Doxygen for Objective-C project, and after a bit of googling I get the impression that HeaderDoc is on its way out.

I found a nice article on the Apple website here which describes how to get XCode to run Doxygen for you as part of your build, and turn the results into a DocSet which XCode knows about.

It works rather well, and the documentation for my test iPhone project now shows up in XCode along with the standard Apple sets.

One slight issue - I had to edit the script a bit to cope with having a space in the path to your project. It also has rather verbose output - adding –silent to the make command helps a bit, as does altering the doxygen.config to run quietly. Unfortunately the commands in the make file that Doxygen generates still seem to spit out a fair amount of crap even if everything is working.

One other problem - XCode can’t seem to find the documentation for a type when I option-click on it, in the way that it would do for an Apple type. This may just be a question of needing to rebuild my indexes though…

Messing with the iPhone - How Many Countdown Apps Can There Be?

2010-01-05T00:00:00+00:00

Over the holidays I’ve been messing about with the iPhone SDK, just for the hell of it.

I always prefer to have a project in mind when learning new tools, but I didn’t have much time, so I fairly randomly decided to do a little countdown application - something that tells you how long you’ve got before Christmas (for some reason this topic was on my mind).

The app itself was pretty simple to get going (less than a day), and doing the UI for the preferences has proved to be far more time consuming than the actual functionality! Once again I’ve been reminded how much fun it is to work with Objective-C / Cocoa.

If this had actually been intended as a commercial prospect, I’d have checked the iPhone store first, but luckily I always viewed it as a made-up project for test purposes only. Having said that I thought that I might conceivably post it to the store, just for the hell of it, so I went to have a look at the competition.

I figured that it was a totally unoriginal idea, so there’d probably be a few up there already. I wasn’t prepared, however, for there to be about forty! What’s more, I’d say that over half of them are paid apps (though costing only a dollar in all cases).

This is quite boggling, and it makes one wonder. Does anyone ever pay for these apps? Some of them have reviews, in some cases quite positive reviews, which suggests that they do. There are supposed to be more iPhones out there than Xbox 360s and PS3s put together. That’s quite a big market, but is it really big enough for 40 applications competing for a tiny and almost pointless niche? I’d love to know if anyone has sold more than 100 copies.

I guess the moral of the story is to check the store before you put any serious time into an application. Although, perhaps not. Perhaps the moral is to check the store, rip off the good features of your competition, then do the app anyway because it’s a big, big, market. Or maybe not!

Documentation - Huh - What Is It Good For?

2009-09-11T00:00:00+00:00

(absolutely quite a bit, actually, say it again)

James at work said yesterday that often he finds, paradoxically, the code that is best documented is also the code that needs documentation the least, since it tends to be well designed and written in the first place.

I know what he meant, and it rang a little bell with me (although whether my code falls into that category is for others to judge :) ).

In lots of the places I’ve worked I’ve tended to be one of the few people who writes documentation, and I often wonder whether anyone else reads it. Is it, perhaps, all a monumental waste of time?

That said (and when I’m not having a bad day), I believe that to worry too much about whether it gets used would be to miss the fundamental point.

The reasons that I write documentation, in order of importance, are:

to clarify what I’m trying to do
to clarify why I’m trying to do it
to offer up my design for peer review
to explain my implementation choices
to note avenues that I’ve tried and rejected, to avoid others doing the same
to help other people use what I’ve done

If anything, the “why?” is even more important that the “what?”. Spend a bit of thought on that one, and the rest may turn out to be irrelevant.

In any case, having other people use the documentation as a reference comes pretty far down the list of priorities.

Which doesn’t mean that I don’t want people to do that (it would really help, frankly, if more people would RTFM!). It does mean though that I’m not going to lose too much sleep if I feel that I’m the only one reading what I write.

At the end of the day, writing documentation, for me, is an essential part of the process of writing software, especially when working with others. How can you code a system that you can’t describe in words or pictures?

Having other people read it is just a bonus.

eee 901 Hackintosh upgrade to 10.5.7

2009-06-12T00:00:00+00:00

Last night, with a bit of trepidation, I let my eee 901 hackintosh upgrade itself to MacOS 10.5.7. I’d been putting it off for fear that I might end up with a non-bootable machine (I could have recovered from it, but it would have been a pain).

As luck would have it, it mostly worked ok.

After reboot, the main problem that I had was that the monitor settings had reverted to 1024 x 768, causing the top & bottom of the screen to be cut off. Since the Mac menubar is at the top, this was a bit of a snag, but luckily I know the layout of everything pretty well, and I also had the advantage of Spotlight. Hitting Cmd-Space brings up the spotlight search box (in the right place, interestingly, even though the menubar was in the wrong place), and typing the name of an application allows you to launch it.

After a bit of experimenting, I ended up just re-installed the kexts from the EEE-Utilities package put together by Gregory Cohen. This seems to have sorted everything out.

Buy a RunCore SSD for your eee901

2009-04-25T00:00:00+00:00

As you may have noticed, I’ve got an eee901, which I’ve turned into a Hackintosh (I’ve installed OS X on it). I’ve previously mentioned that I found it a bit sluggish - but no more!

I finally got round to upgrading the default 16Gb SSD drive to a much faster 64Gb one made by Runcore. By hell does it make a difference! The machine feels substantially quicker, and I very rarely hit the spinning beach ball these days (which happened all the time, previously).

If you’ve got one of these machines, I’d seriously recommend upgrading the SSD, regardless of the operating system you’re using. If my experience is anything to go by, you won’t be dissapointed.

UK Mac Developer Group

2009-04-18T00:00:00+00:00

I’ve just updated the blurb a bit on this group: http://www.elegantchaos.com/uk-mac-dev

After the excellent NSConference, I’m hoping that we can get a few more members on the list, and encourage a few more folks along to our monthly meetings.

NSConference day 2

2009-04-17T00:00:00+00:00

Another excellent day, with some good sessions (core animation looks cool), and a fun finale.

I must confess to being a little bit weary now, and glad to be sitting at home. Tim & Scotty and the others must be knackered - I hope they have the chance to get thoroughly rat-arsed tonight.

Geek Humour

2009-04-17T00:00:00+00:00

Inspired by Andre Pang’s presentation at NSConference, and pointed out to me by Alasdair Allan…

http://www.willamette.edu/~fruehr/haskell/evolution.html

No really, it is funny, honest.

I’ll get my coat.

Are Extreme Programmers A Self-Selecting Group?

2009-04-17T00:00:00+00:00

I had a conversation last night which reminded me of this sneaking suspicion I have about Extreme Programming - which I may have blogged before (but I can’t find my own post if I did!).

Basically it revolves around the fact that I’d be far less likely to want to work with a programmer who had absolutely no knowledge of or interest in XP, unit testing, refactoring etc.

Conversely, I’d be instinctively keen to work with someone who is interested in it, or already doing it. To the extent that I suspect I wouldn’t really care whether we actually DID it at all.

Which leads me to wonder just how much the actual process matters, and how much it’s just a filter to use in order to select the right people to work with.

Why Tying XCode & Distributed Builds To A System Is Dumb

2009-04-16T00:00:00+00:00

In recent years Apple has started doing something that I regard as pretty dumb - tying versions of XCode to a particular system, and/or not supporting older XCode versions on newer systems. XCode 3.0 required Leopard, and I suspect that the next big XCode update will go the same way.

What’s even more annoying is that the distributed build system is tied to your system, architecture and xcode version. I can see why the version of the tool chain used by each node in a distributed build needs to match, but Apple should really spend some time making this stuff work a lot better.

The IDE should really be far more de-coupled from the tool chain, and I really ought to be able to participate in a shared build system running on a number of system versions (for that matter the IDE is a horrible monolithic lump which should really be broken down into smaller components and updated in a far more agile way, but that’s another topic).

The main reason why the Xcode thing is such an issue for me, is that it’s a really big barrier stopping me doing early testing of system updates. One way or another I’ve been getting pre-release systems from Apple for the last 20 years or so. Until fairly recently, I was very happy (keen, even) to live on the edge. Maybe not for alphas, but certainly by the time the system got to beta I’d typically install it and live on it for my main development machine. Crashes and glitches notwithstanding, I wanted to know what was coming, try it out, and give feedback on it whilst there was still time to change stuff.

Now, however, access to pre-release builds is next to useless for me. I simply can’t live on pre-release systems, since I immediately lose access to the other 5-10 macs that are normally happy to help out with my builds (and believe me, I need them to help compile Football Manager, which contains a lot of code!). For the amount of time & effort it takes to install a new system, it’s barely worth it just to click around for half an hour to see what’s changed, before switching back to my “real” system.

I can’t believe I’m the only person who has this problem, and it seems to me that the quality of the feedback that Apple receives must suffer as a consequence.

NSConference day 1

2009-04-16T00:00:00+00:00

First day of the conference proper - and an eclectic mix of excellent sessions.

A lot of stuff wasn’t directly relevant to the day job, but as always with these events it’s fantastic to just get the time & space to think about the issues, and to be pointed in some new directions. What wasn’t relevant today may well become the future.

Plus, of course, the ability to simply meet and hang out with a bunch of like-minded geeks is priceless.

NSConference day 0

2009-04-15T00:00:00+00:00

I’m at the NSConference this week, which is a UK-based get together of Mac and iPhone developers.

Today was workshop day, and I attended an excellent iPhone session given by Bill Dudney - great work Bill.

Hopefully this conference will become an annual event, so if you didn’t make it along this year but are interested in attending in the future, make sure that you let Scotty and Tim (the organisers) know!

Oops (Or What Not To Do On A Friday Night)

2009-02-21T00:00:00+00:00

Come home. Drink two generous vodka & cokes whilst washing up & chatting to Caroline. Decide to “fix” one or two minor issues with the hackintosh.

Use Kext Helper to attempt to re-install some kernel extensions that I suspected hadn’t been installed properly. Reboot said hackintosh.

Experience the “no smoking” / grey boot screen. Experience extreme fear/rage/wish-that-one-had-got-time-machine-working-on-the-eee.

Spend rest of night figuring out what went wrong and how to fix it.

In the end I had to bootstrap off the install cd I used for the original install, then boot into an iDebeb install image I had on a DVD, then use the terminal to copy back an original Extensions folder taken from my MacBookPro. Arse!

On the plus side, I think I now have a better understanding of what’s going on under the hood. On the minus side, it’s 3.38 in the morning, and I’m now back to where I started.

eee 901 Hackintosh Wi-Fi

2009-02-19T00:00:00+00:00

I’m using the original WiFi card in my eee, which does not have native out-of-the-box support from OS X.

Luckily, some third-party “drivers” are available from RALink, makers of the card.

The word drivers was in quotes in that last sentence because although this software allows use of the WiFi card, it doesn’t really supply a Mac OS X driver in the strictest sense of the word. A real driver would just invisibly do its job, allowing you to connect to wireless networks with the normal user interface - you wouldn’t really know that it was there.

The RALink software, in contrast, is rather painful, and a real eyesore. It consists of a custom application which you have to launch when you log in, and which opens a single window which you can’t resize or close.

This window has a tabbed interface with a great deal of random clutter on it that almost nobody will ever care about. The layout is a real dog’s dinner, and looks like it was probably created by a programmer who isn’t used to design visual UI of any kind, let alone for a Mac.

In amongst this is a list of the visible networks in range, and a (confusing) interface allowing you to join one. By default the software does not remember the password for a network, forcing you to re-enter it each time. However, there is a mechanism on a different tab which allows you to save “profiles”, which do save your password to a particular network. You can have multiple profiles, but only one is active at a given time, and you have to choose manually.

All in all, this software is a bit of a mess, but I suppose we should be grateful that it exists and that it has been made freely available. Personally I’d like to see RALink release it as open source - at the very least that would allow someone who knows what they are doing to clean up the UI.

In the meantime, I may upgrade my wireless card, just to get rid of this software. Which is a bit of a shame, since the built in card seems to actually work perfectly well.

eee 901 Hackintosh Performance

2009-02-19T00:00:00+00:00

After a few weeks playing with my eee 901 hackintosh, there’s no denying it, this machine is quite slow, and I seem to spend quite a bit of time watching the dreaded spinning rainbow. I’m running Leopard, and I’ve read various reports suggesting that Tiger is snappier, so I suppose that might be an option if you are prepared to go back to an earlier system (which I’m not).

In some cases the lack of performance seems slightly strange, and I wonder if there’s something a bit wrong at a driver or kernel level. Looking at Activity Monitor, the machine doesn’t often seem to be under major strain. I have two megs of ram, and have set virtual memory up to use the smaller, faster SSD partition, and it doesn’t look like there’s much paging going on. The SSD speed itself may be an issue, and I am planning to upgrade to a larger, faster drive at some point, so it will be interesting to see how that works.

Having said all this, it’s worth emphasizing that the machine is more than useable. A little bit of patience is required when starting or switching applications, and occasionally when a random stall occurs, but for the most part everything is fine - it’s not as if it struggles to keep up with my typing or anything like that.

Apple Mail Feature Request

2009-02-10T00:00:00+00:00

Something occurred to me whilst I was setting up Apple Mail on my new Hackintosh.

It has an option to show plain text messages using a fixed width font. I have this set, but only because of the very occasional plain text message that uses tabs or spaces to line up items in multiple rows.

How about having a “smart” mode that attempts to spot this situation and use fixed width fonts only when necessary?

Off the top of my head I’d say that any mail containing a tab character or a run of two or more spaces (excluding the ends of lines), should trigger this mode.

Alternatively it could attempt to spot plain text formatting and actually convert it to formatted styled text. This would be trickier of course, but might allow it to do nice things like auto-converting common plain text idioms such as emphasis, underlining, bullet lists, and so on (I’m thinking of mark-up languages such as Markdown, or wiki-style formatting). I think Mail may actually do this already for emphasis, but it could be nice if it was expanded to cover other formatting.

Another Hackintosh is born...

2009-02-06T00:00:00+00:00

My eee 901 is now running MacOS X 10.5.6.

Many thanks to the various people out on the web who’ve written up instructions and advice on how to achieve this. The main link I used was this one from Gregory Cohen.

It would be fair to say that the resulting Mac is not flawless - WiFi support is a bit clunky, and there a number of minor things that don’t work right by all accounts (although I’ve yet to encounter any major problems).

That said, this is the cheapest Mac I’ve ever had - and yet I would gladly have paid two or three times as much for a really well put together machine of a similar size, made by Apple. Hint hint.

Hackintoshes Possibly More Popular than Linux?

2009-02-05T00:00:00+00:00

Since I’m in the process of turning my eee 901 into a hackintosh, I was interested in this…

http://lifehacker.com/5146174/hackintoshes-possibly-more-popular-than-linux

I doubt the numbers personally, but it would be funny if it were true :)

Intial eee thoughts

2009-02-03T00:00:00+00:00

Last week I bought an eee 901.

This is a fairly major departure for me - the last time I bought a non-Apple computer was about 1987 (an Amstrad CPC 128).

I already have a 15” MacBookPro supplied by work, but that tends to stay at home, tethered to an external monitor, keyboard, and external usb drive used for Time Machine.

I do occasionally move my MacBook round the house, but I’ve started to realise that increasingly I can’t be bothered (unmounting the usb drive is a particular pain since it often involves waking the machine up and entering a password just so that I can put it to sleep again). I also rarely leave the house with it. It is too large to fit into my normal bag, and quite heavy anyway. All of this is in marked contrast to my old aluminium 12” G4 powerbook, which used to go everywhere with me.

So basically I was in the market for something small, rugged, and not too pricey - in other words a netbook. I am a Mac programmer and fan of course, but Apple don’t make netbooks. Bugger!

So I started looking into going down the Hackintosh route - ie get a generic PC and hack it to run MacOS X (or rather, hack MacOS X to run on it).

I also realised that these days I rarely have time for much in the way of hobby programming, and a lot of what I would do if I had the time would be Python/Django based. Other than that, what I wanted this machine for was note taking, surfing, and that’s about it. So actually the need for a Mac suddenly seemed a bit more remote.

Hence the choice of an eee 901. Its small, I’ve got a solid-state drive so its rugged, it came with Linux installed so I didn’t have to pay anything for a copy of Windows that I’d never used - and a lot of people already have experience hacking it to hell, including running MacOS X.

So far I’ve only had it for a few days, so these are very initial thoughts:

Size

Its small, and pretty light. At first I thought it was feather-light, then I realised that the battery is disconnected when it ships. Lifting up the battery I realised that it weighs about as much as the rest of the machine!

Keyboard

The keyboard is very useable. Yes it is small, but even my fat fingers can cope quite well. I wouldn’t want to write a novel on it, and I’ve made quite a few mistakes typing this.

Once you’ve spent enough time trying to type on an iPhone, the eee feels positively spacious though.

Trackpad

The trackpad sucks. It feels innaccurate, and the buttons require way too much force. The driver situation for them also seems a bit iffy, but that’s my own fault for throwing away the built in OS and installing…

Ubuntu

So I trashed the built in system, but rather than going straight down the Hackintosh route, I thought I’d trying living in Linux first for a while. I develop for Windows and the Mac at work, and sometimes for consoles too, but its a long time since I’ve used a Linux desktop.

So far I’d have to say that it’s “fine”. I haven’t been blown away, but for what I’ve done on it so far, it works. There is a certain geeky pleasure to be had in installing a custom kernel and tweaking file system parameters for optimised SSD performance, but frankly all of that stuff can be done on a Mac too these days (its Unix with nobs on, basically).

Conclusions

Its too early to tell how much use I’m going to get out of this machine, but for £250 its certainly a considerably smaller outlay than a new, smaller Mac would have been, and it’s quite a lot of fun feeling that I can trash the OS whenever I feel like it, or perhaps try risky hardware mods that I wouldn’t attempt on a bigger machine.

I probably will try to get MacOS running on it soon, just to see how the performance compares with Ubuntu. I’ll probably also get some more RAM, and maybe upgrade the SSD. And perhhaps paint it a silly colour or cover it with stickers. Fun :)

Festive Greetings from Chaos Towers

2008-12-23T00:00:00+00:00

Have a fantastic «fill in your festival of choice» everyone, and a Happy New Year if it is a new year for you!

And if you’re not celebrating, then enjoy the peace & quiet at work :)

Live Update Updated

2008-03-13T00:00:00+00:00

I’ve updated the Live Update widget slightly, to fix a bug which caused problems when the station being displayed doesn’t list the platform numbers.

I’m not sure quite why some stations don’t do this, but apparently they do!

Live Update dashboard widget

2008-03-10T00:00:00+00:00

I’ve been fiddling around with Dashcode, and have created my first widget.

It displays live train update information for the UK train network, of the sort that you get on the live indicator boards on the platforms.

Dashcode itself is quite cool, except for its utterly infuriating habit of completely destroying and recreating the entire directory structure of your project all the time, thus rendering it utterly incompatible with Subversion.

sigh.

XCode 3 and the command line

2007-11-08T00:00:00+00:00

One nice little bonus with XCode 3.0 (new with Leopard) is the inclusion of the xed command to invoke the xcode editor from the command line. It’s nice to be able to open an XCode window to edit files, and it’s particularly handy to be able to set P4EDITOR, SVN_EDITOR and other such environment variables to xed.

Something like this was possible before using open -b com.apple.Xcode, but the problem with that way of doing things is that you had to quit XCode afterwards to convince things like Perforce that your edits had been completed (perforce typically invokes the editor for you, then waits until you’ve finished editing a specification file before doing something with it).

The nice thing with the xed command is that it has a -w flag which does just what is needed in most cases - it sits in a loop spinning until XCode tells it that you have closed the file that it opened. This allows you to use XCode as your general text editor without it interfering with your normal work.

Nice touch…

Nasty Recursion In Drupal

2007-10-31T00:00:00+00:00

I think I’ve just found what seems to be a nasty bug in Drupal.

You can specify a page to display if a requested URL isn’t found. However, if you request a URL that starts with “/node/” this page doesn’t seem to be displayed - you just seem to end up back on the referring page, but with a seemingly new URL which is a product of the existing URL plus the requested URL.

Why is this bad? Because if you inadvertantly miss out the “http:” bit from a link and therefore create a local link instead of an external one, you can end up creating an infinitely recursive sequence of “pages” that link to each other. In reality they are all just the same page, but each URL for each one is different.

This isn’t an issue for humans, but it is for you if a dumb crawler stumbles across your site. It will just sit in a loop hammering the same page… forever. Nasty.

Needless to say I discovered this bug by making the mistake myself. And I only spotted it (a long time later) when looking through the logs wondering why on earth a page on my site was continuously being requested that contained the URL www.apple.com in the middle of it!

Theme Mania

2007-09-28T00:00:00+00:00

As you may have noticed, I’ve been messing around with some new themes from Drupal’s site.

At the moment my favourites are:

burnt
blue_zinfandel
kubrick
itheme
barron

The current default theme is a modified version of blue_zinfandel, which I’ve tweaked to use a right nav bar.

Very much work in progress…

Upgrading

2007-07-24T00:00:00+00:00

I’m going to make another attempt to do some upgrading tonight, so the web site will probably be up & down a bit, and I’m temporarily resetting the theme back to a default.

Normal service may be resumed at some point, possibly, if I’m lucky.

Blogged with Flock

Upgrading MySQL on MacOS X whilst running Drupal

2007-07-24T00:00:00+00:00

The short version: it’s a bit bloody hairy! I’d avoid it if I were you.

The long version:

I was running MySQL 4.0, and trying to get Django to work.

Running ./manage.py syncdb, it said “server is too old to set charset”

It seemed to need 4.1 to get some Django stuff working, so thought I’d upgrade. I’m running Drupal 4.7 on the same server, which is a Mac running 10.3.9. I was (fairly) confident that Drupal was backed up ok, so decided to just download the DMG for MySQL 4.1 and give it a whirl.

So, after installing and restarting, I tried to access my website - hello? anyone there? Oh bollocks… no website.

Looking at the /usr/local/mysql/data/ folder I was horrified to discover that all my databases had disappeared. Brown trousers time…

At this point I should point out that I am an idiot. If this happens to you, don’t panic. The installer has not eaten your databases, which is what I thought it had done!

If you look in /usr/local/ you will discover that mysql is aliased to the actual installation, for example mysql-max-4.1.22-apple-darwin7.9.0-powerpc. I didn’t realise this, and I had the directory aliased from somewhere else too, so I thought my databases were gone.

I was remarkably calm. Oh well, I said, time to discover if the backup script actually works.

I am using mysqlhotcopy to back my databases up, and I’d never had to actually do a restore (I can’t believe I’m saying that - I hadn’t tested my backups - but then testing them isn’t that easy when you have no time and no spare server). Anyway, I had no idea how to turn the backup back into a working database.

The answer, in case you are wondering, is remarkably simple - copy it back into the right place. Wow - it’s almost like using a Mac (apologies to the hardened unix geeks out there, who must be wondering what kind of an idiot I am. The answer is, I am a very technical idiot who, due to a career spent programming macs and the occasional pc, has never had to deal with any of this shit before). I love that the Mac is based on unix now, I really do. I just wish unix would catch up sometimes…

Anyway, I copied the backup back into the right place, and as luck would have it, it seemed to work.

Of course, I had to spend a while fixing up all the sql users and passwords and privileges and all of that malarky. I always forget my MySQL root password for some reason, which really doesn’t help. Lucky I wrote it down ;)

So, I had the databases back, and everything should be hunky dory. Hit reload in the web browser, and… Drupal was complaining that it didn’t have access. To be precise, it was saying “Client does not support authentication protocol requested by server; consider upgrading MySQL client”. Eh?

I rummaged around a bit, and started thinking, uh-oh, maybe Drupal 4.7 doesn’t work with MySQL 4.1. After a bit of searching, I got the vague impression that there were some problems, but there seemed no clear solutions.

Right, I figured. In for a penny… lets upgrade Drupal to the very latest 4.7 variant, that’ll fix it.

So I did, and it didn’t.

Oh so it, maybe Drupal 5.1 will fix it.

It didn’t.

Oh, I’ve found a note saying you have to go to 5.1 via 5.0. I’ll install that.

It didn’t help.

At this point I was wishing that I hadn’t started. However, a bit more googling finally gave me this: http://drupal.org/node/13977. Aha! Nothing to do with Drupal at all.

Turns out that the password format has somehow changed and you have to do a bit of mucking around to fix it. Ok, fine, I can do that. So try again, and thank something or other, it worked. The web site came back.

Whoohooo.

This is probably where I should have stopped… except that when I tried to log in to the site to blog all of this, I couldn’t. It just silently failed to log me in, and I ended up back on the log in page. What???

I had a vague notion that a similar passwordy sort of problem was perhaps occuring, this time with Drupal’s own user table, but I couldn’t get it to work and I didn’t know how to change Drupal’s own password entries with SQL. I had gone back to 4.7 by this time, but I still couldn’t log in.

I tried running the various Drupal update scripts for various different versions of Drupal, but no joy. In fact, they didn’t work because of various other permissions problems. Some of them turned out to be the fact that I hadn’t got the permissions for the database files right when I restored them.

Ah well, thought I. I’m in a right pickle now, but at least the site sort of works. What the hell, maybe MySQL 5.0 will fix it. You can tell I was feeling optimistic tonight.

So, another DMG install later, and no further on. Actually, things were worse. Drupal 4.7 wasn’t talking to the database again, at all.

Luckily it was around this point that I realised that the multiple MySQL installations were happily residing in different directories on my server, along with my original databases. This is exactly how I’d expected it to work in the first place, I’d just got a bit confused.

So, realising that I still had my original installs of MySQL, Drupal, and the original data files, I figured that I could probably just get things working as they originally were.

I did. It works.

Interesting evening that.

The only problem is, I still can’t get Django to talk to MySQL. Arse.

Site Upgraded

2007-07-24T00:00:00+00:00

Well I finally managed to figure out some of the problems I’ve been having and upgrade both MySQL and Drupal.

As you will notice, the site theme has changed as a result, and it’ll probably take me a while to get things straight.

The route that I took in the end was to use mysqldump to back up the entire database, upgrade to MySQL 5, and restore the database from the dump. This seemed to work fine and left me with a working database with my existing 4.7.0 install of Drupal.

I then upgraded Drupal in stages from 4.7.0 -> 4.7.6 -> 5.0 -> 5.1, running update.php at each stage. A bit laborious perhaps, but it worked without a hitch.

Thanks to D’Arcy Norman for a handy link, which put me on the right path.

Programming Tests

2007-06-22T00:00:00+00:00

I keep meaning to work on a new one, and was googling around today when I came across this.

I got sufficiently distracted to post a comment:

Link http://climbing-the-hill.blogspot.com/2007/06/on-severely-misguided-nature-of.html

Eclipse And XCode

2006-12-13T00:00:00+00:00

I'm currently writing code for four platforms, using three IDEs (Eclipse, XCode and VisualStudio), and it's driving me bonkers!

I've said it before, but I'll say it again - why oh why doesn't Apple consider freezing XCode development and switching over to using Eclipse. I know that XCode has some nice features but generally I find it pretty clunky and I can't believe that all the good stuff couldn't be reimplemented as Eclipse plug ins.

Eclipse, on the other hand, is really solid but a little bit lumbering and confusing at times - I'm sure it would really benefit from some Apple user interface input.

So once that's happened (I wish), I just need to convince Whole Tomato software to reimplement Visual Assist for Eclipse, and then I'd be laughing...

XCode auto completion is rubbish

2006-11-27T00:00:00+00:00

Every now and then I come back to using XCode for some reason or other.

And every time that this happens, I am once again amazed at just how bad it is in many ways, at least for C++ development.

My current bug-bear is the code auto-completion - when you start typing the name of a function or variable and it tries to jump in there and suggest something for you.

On the whole, the suggestions that it makes seem to have got a bit better since the last time I used it, and it seems to manage to make a suggestion in a reasonable amount of time (maybe that's just because I've got a MacPro now).

The interface for making the choice, however, is awful. The thing that's really annoying me is that there appears to be no way of having it pop up suggestions, but allowing you manual control over whether to accept a suggestion.

It seems that you either have to manually pop up the suggestion box (which I don't want to do - I always want it to suggest things), or you can have the box come up automatically, but as soon as you hit any key other than escape, the currently selected suggestion will get inserted. This is truly awful user interface, since I'd put the success rate for it's suggestions at 50% at best. What this means in practice is that it is endelessly inserting the wrong text, as I type quickly and often the suggestion box has come up, an insertion has been made, and it has gone away again - before I've even registered that it was there.

I really hope that I'm just being stupid, and that there's actually an option to make it put up the suggestion box, but only insert the suggestion if I press a particular key (e.g. Return). This is the way that Visual Assist works, and it's a lot more intuitive (not to mention a lot safer).

Debugging UTF-16 strings in XCode

2006-11-17T00:00:00+00:00

It's been a while since I used XCode, so I was a bit baffled when debugging recently when I couldn't find a way to view a wchar_t* based string.

Surely there must be a way?

As it turns out, there isn't a way to do this that's built in to XCode (amazing!), but XCode can be extended with plugins, and Apple provide a sample plugin that does what I wanted.

To save everyone else going through the same length process that I've just gone through, here's how you do it:

You can get the sample project from http://developer.apple.com/samplecode/WcharDataFormatter/.

Build it with XCode (I built the debug version), and copy the resulting wchardataformatter.bundle into "/Library/Application\ Support/Apple/Developer Tools/CustomDataViews/".

One pitfall to look out for - it appears that the bundle has to be built for the native architecture that you're running on. By default, the project is set up to build for ppc, so if you're on Intel you'll need to change this to i386.

Once you've installed the plugin and restarted Xcode, you should find that when viewing any wchar_t or wchar_t* string, the Summary column in the debugger contains the correct character/text.

Codewarrior Nostalgia with XCode

2006-11-17T00:00:00+00:00

I've just discovered something neat...

After executing the following command in the terminal:

defaults write com.apple.Xcode PBXBuildSuccessSound /Users/sam/Music/Codewarrior\ Clunk.aiff

My XCode now plays the familiar CodeWarrior "metal stamp" sound when my build finishes! Of course, you'll need to replace the path with a sound file of your own. I used Audio Hijack Pro to extract the clunk sound from a Codewarrior, but possibly it's out there on the net somewhere.

Slight Change Of Plan...

2006-09-22T00:00:00+00:00

I've had an interesting time over the last year at Sony, but recently I started to realise that it wasn't really going in the direction that I wanted, and so I started to look around for alternatives.

As it turned out, my old friends at Sports Interactive were on the look out for a programmer. Things have moved on a lot at SI in the last couple of years, and to cut a long story short, I decided that I was very interested in going back there - and so, that's what I'm going to do.

The decision was made a while ago, but I've been keeping quiet about it until I actually left Sony, which happened this Tuesday. So on Monday, I'll be starting back at SI towers.

They say that you should never go back... but then they say all sorts of stupid stuff, so what do they know...

;)

Autolink Drupal Module Updated

2006-05-21T00:00:00+00:00

My Autolink module has now been updated to work with Drupal 4.7.

For the uninitiated, here’s an excerpt from the read me file:

The Autolink module saves users from having to manually enter links for commonly used URLs.

The module scans posts for a set of terms. Any term found is optionally replaced with some other text, and then automatically linked to its corresponding URL.

The module uses Drupalâ€™s built in taxonomy feature, so you can organize your terms in a Drupal vocabulary. Autolink terms are represented with the taxonomy terms in the autolink vocabulary. The descriptions are used to provide the URL that the term should link to.

Drupal upgrade

2006-05-20T00:00:00+00:00

I’ve just upgraded this server to use the latest version of Drupal.

Please let me know if you notice any glitches!

Applescript is a read-only language

2006-04-26T00:00:00+00:00

Tom Smith and I were just discussing Applescript, and we just realised... it's a read-only language!

It's easy to read, but near-impossible to write, due to the weird and shifting syntax caused by the interaction of the various applications you're trying to script.

GDC: Mac Count High

2006-04-12T00:00:00+00:00

One little GDC note in passing - I noticed a surprisingly high number of folks with Apple laptops. I did an ad-hoc survey by walking from one end of the convention centre to the other, counting non-apple laptops and resetting to zero when I saw an Apple one. One average I got to about 6. A ratio of 6:1 isn’t bad - definitely larger than Apple’s alleged market share.

London Cocoa Programmers' Group

2006-04-10T00:00:00+00:00

The uk-mac-dev mailing list, which Richard Buckle and I organise, has affiliated itself with Cocoa Heads, to form a London Cocoa Programmers’ Group.

Effectively we’re still meeting at the same place and same time (second Monday of each month, 19:30, at Lord Moon of the Mall, Whitehall), but the two groups have enough overlap that it seems sensible to combine our efforts.

Next meeting is tonight - feel free to join us!

GDC: Days 4 & 5

2006-03-28T00:00:00+00:00

Bit of a delay in the reporting… caused not so much by the fantastic time that I was having, as by the gradual worsening of my flu :(

The hilight of these days for me was the “Best Practices” session by Grady Booch. I’d like to see more sessions like this at GDC - from mainstream computer science experts rather than just from games specialists.

Other interesting sessions included the next-generation animation panel, “crowds in a polygon soup”, and a session on how the God Of War programmers tried to arrange things so that they could spend the last month of the project on the beach! I’d recommend looking up the slides for these if you missed them.

Overall, I think I’d only give the conference a 3/5 mark. There were a few interesting sessions, but I didn’t feel that it was brilliant from the programmers perspective. The best sessions were about techniques - whether high-level planning, medium-level algorithms, or low-level implementations. These sessions explained how the authors had gone about doing something, and hence how we could do the same thing. Unfortunately, there were a few too many sessions which just told us what someone else had done, without explaining how/why, or in a context which wasn’t very useful for anyone else.

GDC: Day 3

2006-03-23T00:00:00+00:00

GDC: Day 3

Another day, another bagel… actually, I only remember the end of the day vaguely… something about a Sony party… I’m seeing lots of vodka…

This was the first day of the conference proper and the expo - all of a sudden the halls were swamped with people.

I got to a few good sessions - mostly pretty tecchy. C++ on Next Gen Consoles from Pete Isensee was quite useful - there wasn’t much that I hadn’t heard already but it’s good to have one’s assumptions confirmed. Phil Harrison gave a good speech (but then I would say that) - overall I thought that the strategy he described sounded pretty sensible. Some of the PS3 demos looked good, although some were a bit pants. I also went to a good Animation panel presented by some big-wigs in the field. Quite a bit of the detail went over my head, but they showed some great techniques and I think I got the key bits :)

For some bizarre reason, most of the next-gen coding sessions (the ones that everyone wants to attend) seemed to have been located in the smallest rooms, so were absolutely heaving. This is bloody frustrating, and meant that I missed some of the stuff that would have been most useful. Grrr… whoever did the room allocation this year should be shot.

Still battling this bloody cold. Not sure if the excessive vodka consumption has killed it, or just preserved it for a day or two.

Top fives for the day:

spirit measures at the Sony party: 5/5
phil’s keynote and Sony’s strategy: 4/5
conference organisation: 1/5

GDC: Day 2

2006-03-22T00:00:00+00:00

I attended the OpenGL ES tutorial today, which was pretty tedious. To be fair, I’m no graphics programmer so there were parts that I might have got more out of had I understood what they were talking about. That doesn’t really excuse the fact that these guys weren’t great speakers, and that they were mostly giving dry facts that I could read directly from the slides, as opposed to an insightful commentary on the issues around the slides.

I was also suffering badly from a combination of jet-lag and a cold, which probably didn’t help. Unfortunately Sony have plonked us in a hotel which is a 20 minute bus ride away from the conference. This is no problem in the morning, but the ability to nip back to the room for a quick hour of sleep in the middle of the day would have been invaluable today. If you’re coming to GDC from Europe next year, I’d strongly advise that you find yourself a hotel within easy walking distance of the convention centre!

On the positive side, I managed to meet up with Kevin Marks and Maf Vosburgh in the evening (both fellow MMC alumni). We had a few beers and talked over old times, then Maf took me off for a tour of the humungous Google campus where he works. Judging by the size of the canteen, they really are talking over the world ;)

Top fives for the day:

breakfast at the hotel: 4/5 (mmm… pancakes…)
lunch at gdc: 1/5 (a sandwich and a packet of crisps? I long for the halcyon days of WWDC circa 1994!)
dinner at Google: 4/5 (mmm… burritos…)
San Jose weather: 2/5 (feels more like Manchester to me)
OpenGL ES geek factor: 5/5 (nuff said)

GDC: Day 1

2006-03-21T00:00:00+00:00

Hmm, well it seems that the gods are indeed not smiling on me. I got back to the hotel to discover my door ajar, and some stuff missing. Only a book and magazine as far as I can work out, but still somewhat unnerving. Oh yeah, and I’ve got a bloody cold - no boozy parties for me today. Bastard.

Other than that, I had a good day. I went to a day long tutorial on engineering issues for online gaming, which on the face of it isn’t that relevant to what I’m doing. I was worried that it would be a bit dry, and overly technical, but it turned out to be five excellent speakers, all with interesting stuff to say, and focussed on quite high level issues of planning, organisation, management etc. Very useful stuff, and a lot of it was quite applicable to next-gen development even if you’re not making an MMO game.

Marks out of five for the conference so far:

goody bag: 1/5 (two juggling balls? what the hell is the use of that?)
sessions: 5/5 (people who can talk to humans, what a relief)
food: 3/5 (more than one choice for veggies would be nice)
hotel: 1/5 (what’s mine is yours)

Top quotes so far:

“I like a good bitch… erm, no hang on, that came out wrong” - me, after ranting for a while.

“Johnny has a nice pink one” - Dave.

GDC: Day 0

2006-03-20T00:00:00+00:00

This week I’m lucky enough to be attending the GDC Conference in San José. In a wild burst of enthusiasm, I thought that it might be a good idea to post some reports here as it goes along. It remains to be seen how long this lasts…

Day 0:

If one believed in omens, then to start my trip, at 7am in London, by dropping my mobile phone into the bath might have seemed to be a bad one. Luckily I’m made of sterner stuff, and anyway the phone seemed to be ok… until I was on the move and it was too late to do anything about it. Arse!

Hopefully, that was my big bit of bad luck for the week. The rest of the trip was pretty painless - a few minor delays, but we got to the hotel eventually, had a few beers, and hit the sack for some much-needed sleep. Unfortunately Dave wasn’t so lucky - losing his wallet in the cab on the way to the hotel! Double-arse, with knobs on…

It’s 7am again now and I’m about to venture out to the convention centre for the first time. Further reports to follow (they might even have something about GDC in them).

Great Metrowerks Article

2006-03-08T00:00:00+00:00

Here’s a nice article from a guy who worked at Metrowerks, and then at Apple. He gives some interesting perspective, a lot of which confirms the worst fears (at the time) of anyone who was watching Metrowerks from the outside…

Game Developers Conference

2006-03-08T00:00:00+00:00

I’m going to be in San Jose from March 20th - 25th for the Game Developer’s Conference.

If I know you, and you’re going to be there too, give me a yell!

Goodbye To Trackbacks

2006-01-30T00:00:00+00:00

I’ve given up on Trackbacks - they don’t really add much to the site, and the amount of trackback spam around means that they are a pain in the arse to administer.

What I want now is a Technorati module to take their place, which automatically shows me when someone links to one of my blog articles…

EU Software Patent Argument to Reopen?

2006-01-18T00:00:00+00:00

From Slashdot….

New Job

2005-10-17T00:00:00+00:00

Last week I started work at Sony London, as a senior programmer, working on… well I’m not sure if I am allowed to say, so I’d better not risk it.

So far I’m enjoying every minute of it, although getting into the office at 9am every morning is a bit of a shock to my system!

So Long And Thanks For All The Smoothies

2005-10-06T00:00:00+00:00

I’ve just finished a spell of contracting back at Sports Interactive, and I’d just like to say cheers to everyone there for making me very welcome once again.

It was scary how easily I fitted back in (and even scarier that I’m sure there were things in the fridge that have been there since I left last year).

The place is still completely bonkers in many ways, and I could feel myself getting dragged back into the arguments about coding that caused me to resign in the first place, but I still thoroughly enjoyed it - hard not to with such a nice bunch of people.

Football Manager 2006 is looking good too…

Next week I start The New Job(TM). Call me superstitious, but after the Dublin experience I think I’ll wait until I’m actually there before saying anything more…

Haskell and Dylan dominate ICFP programming competition

2005-10-06T00:00:00+00:00

According to Ars, Haskell and Dylan dominate ICFP programming competition

Hurrah for Dylan…

invokedynamic in the JVM

2005-10-05T00:00:00+00:00

Just read an interesting post from Gilad Bracha about adding more support to the JVM for dynamically typed languages.

I’ve been wondering for ages why the guys in the Dylan community haven’t attempted to target the JVM instead of the current approach which involves generating and then compiling C code.

Possibly invokedynamic may help… although it depends a bit on whether it supports generic method dispatch based on the type of any and all arguments. I don’t know enough about the guts of the Dylan implementation to be sure that this will help, but I hope so…

Asserting Oneself

2005-10-05T00:00:00+00:00

Noel Llopis wrote a nice article in Games From Within defending the use of assertions as a programming technique.

I’m right there with him on this one - the idea that asserts are incompatible with unit testing or other good programming practises is just daft.

I Am Still Here, Honest...

2005-09-14T00:00:00+00:00

Things have been a bit quiet at Chaos towers recently.

This is mostly due to the fact that I’ve been back at Sports Interactive doing some contracting - basically helping them to hammer out a few bugs for the next version of Football Manager.

I’ll be there for a few more weeks, and then off on another exciting adventure… more details to follow shortly.

Autolink module version 1.3

2005-08-17T00:00:00+00:00

I’ve updated my Autolink Drupal module to version 1.3.

The only change with this version is that it now works properly with Drupal 4.6. See the Autolink project page for more details.

Site Update

2005-08-06T00:00:00+00:00

Following my site being hacked, I’ve finally been spurred into updating to the latest version of Drupal. Please let me know if you spot any problems.

I suspect that the reason that I was hacked was that I was still running a version of 4.5 which had a security hole in it. This in turn was because I had been putting off the upgrade as it wasn’t reversible and looked like it might require performing some SQL updates. I think that the difficulty of installing and upgrading Drupal are one of its major weaknesses. It often involves backing up, manually downloading items, running scripts, issuing SQL commands, moving files around, all of which can be quite daunting. I’m very capable of doing such things, but the more steps there are, the more it looks like something might go wrong - so you start thinking that you will have to have enough time available to recover from any catastrophic cock-ups, and hence you put it off for another day…

It should be possible to automate most if not all of this process. Installation really ought to be a case of clicking one icon or running one shell script. Creation of databases could and should be automatic, or performed in a wizard on the actual site that it being installed.

New versions of Drupal ought to be detected automatically, and there ought to be an option on the admin page of a site to allow you to perform an update. This update should be performed safely - making a complete backup of all data first - without the administrator having to perform a backup themselves.

Installing updates and custom modules ought to be possible from within Drupal itself, in the same way that Eclipse - for example - has a built in update mechanism. The admin page ought to display a list of all of the items that are available on drupal.org, and there should be support for downloading and installing them from within Drupal. When a new module is enabled, it ought to perform any SQL creation/updating that it needs to do, without the user having to do anything themselves.

All of this wouldn’t be rocket science - the technology mostly exists already, it’s just a question of people taking the time to package things up nicely.

Autolink module updated

2005-07-22T00:00:00+00:00

I’ve updated my Autolink Drupal module to version 1.2.

New with this version is the ability to add text which will be inserted into a rel=”” attribute on the automatically generated link.

You can use this to insert anything relevant into the rel attribute, for example you could use it to add XFN information to links.

Old Stuff

2005-07-12T00:00:00+00:00

Looking at my logs, I noticed that I was getting quite a lot of people looking for old links, particularly for Last Exit to Hypercard.

As a result, I’ve put up most of my old site at http://old.elegantchaos.com.

Autolink Module Update

2005-06-30T00:00:00+00:00

I’ve updated my Drupal module Autolink to version 1.1.

To use Autolink, you define a Drupal taxonomy and add some terms to it.

When any of these terms occurs in text on your site, Autolink automatically adds an html link to whatever URL you have specified for that term. So, for example, you can enter terms for all of your friends, then any time you refer to them in a post, their names will automatically be linked to their blogs.

The main change in this version is that you can specify some replacement text, so that when the link gets created, the replacement text is displayed instead of the original text.

As an example of why replacement text is useful, say we defined a term “Apple”, to be linked to “http://www.apple.com”. The danger with this is that an article containing the word “Apple” in another context might inappropriately be linked to the Apple website. We could use a less ambiguous term, like “Apple Computer Inc.”, but we don’t want that text to have to actually appear in all of our articles, because it is unnecessarily formal. The solution is do match for “Apple Computer Inc.” in the article, but provide a replacement that converts it to “Apple” in the final output. Now it’s unlikely to be matched incorrectly, but the text will still read ok.

For more details, see the Autolink project page.

Endian issues

2005-06-27T00:00:00+00:00

Drunken Blog has been talking about byte-swapping, amongst other things, with regard to the Apple x86 issue.

My comment, having done quite a bit of porting games to the mac from the PC, would be that endian issues accounted for a hell of a lot of my problems, especially the hard-to-track-down bugs where some behaviour quietly changed in some obscure part of the game, without anything actually crashing.

The worst thing tended to be when people cast pointers to (void) or (char) in one place, and then accidentally cast them back to the wrong type somewhere else.

Luckily, incorrect code is actually more likely to work on little endian systems than it is on big endian ones.

This is because, if you have a small value stored in a long, and you take a pointer to the long, then treat it as a pointer to a short (or char), then read the value that it’s pointing to, the value will still be correct on a little endian system, provided that the value actually fits into the short (or char).

On a big endian system, you end up pointing to the other end of the value, which gives totally unexpected results most of the time - if you’re lucky and the value was set as its original type, you’ll get a zero. If you’re unlucky, and the pointer code is doing the setting, whilst some other code is reading the value as the original type, you’ll get uninitialised memory.

Obviously, casting pointers to and from (void) and (char) is a bit of a nasty way to behave generally, but there are legitimate reasons for doing it (well, semi-legitimate, if you’re using old skool C library routines), and it’s easy for mistakes to creep in.

So people with dirty code are going to have some trouble with this. The good news is, over all, getting through the problem will leave their code in a much healthier state.

UI Scripting with Applescript

2005-06-24T00:00:00+00:00

It’s amazing how easily I can get distracted.

Currently my main distraction is playing with Feed Me, my bayesian news filter, which is a thinly veiled excuse to mess about with Python.

However, yesterday I managed to get distracted from that distraction.

I’ve been thinking that I really need to sort out some productivity tools for myself, to deal with the sort of repetitive administrative tasks that I find myself doing again and again.

To that end, I installed FastScripts, and set about writing some trivial scripts to do some simple things.

One of the scripts I wanted was to find all the recent mails received from a particular person, since this is something that I do manually quite often. The script would only save a few keystrokes and mouse clicks, but I figured that if I could link it to a hot key with FastScripts, it would be handy.

Unfortunately, it turns out that Apple Mail’s scripting support doesn’t allow you to perform searches.

Luckily, there is a work around, in the form of the new GUI scripting capabilities of MacOS X.

Essentially, this allows you to ‘fake’ any user interface operation by scripting it in terms of the menus, buttons and or places that you would click, and the keys that you would press.

Presented below, for your delight and delectation, is a little example of how I used this to script a search in Mail.

The hardest part is working out exactly how to specify the interface widget that you’re trying to activate. A valuable helper here is the Prefab UI Browser which allows you to browse another application’s user interface whilst it’s running, so that you can discover the hierarchy of interface widgets.

The bad thing with all of this, of course, is that it’s a very fragile way to program. By relying on the names and layout of particular widgets, the script becomes very likely to break when a new version of Mail comes along with a different user interface. So I wouldn’t recommend it if there is a useable Applescript interface available instead, but it’s bloody handy when there isn’t one!

The key routine here is searchMailboxForMailFrom() which takes the name of a user and a mailbox, and uses them to perform a find in Mail. The assistive devices support that makes all this possible is only available in OS X 10.3 or higher, and the user has to have turned it on in the system preferences, so it’s necessary to check for that if you want to be a well behaved script and fail gracefully when there’s a problem.


on run
	if gotInterfaceScripting() then
		set mailIcon to "/Applications/Mail.app/Contents/Resources/app.icns"
		display dialog "Search for mail from:" with title "Search Inbox" default answer "" with icon POSIX file mailIcon
		if the result's button returned is "OK" then
			searchMailboxForMailFrom("Inbox", the result's text returned)
		end if
	end if
	
end run


on searchMailboxForMailFrom(searchIn, searchFrom)
	tell application "Mail"
		activate
	end tell
	
	tell application "System Events"
		tell process "Mail"
			-- select first mailbox in list - should be the inbox
			select (first row of outline 1 of scroll area 1 of window 1 whose value of static text 1 is searchIn)
			
			-- select the "mailbox search" menu item
			click menu item "Mailbox Search" of menu "Find" of menu item "Find" of menu "Edit" of menu bar item "Edit" of menu bar 1
			
			-- type the search term and hit return
			keystroke searchFrom
			keystroke return
			
			repeat until exists button searchIn of window 1
				-- can get errors if the script runs before the button has been created...
				-- so we wait here
			end repeat
			
			-- click the mailbox button (as opposed to searching in all mailboxes)
			click button searchIn of window 1
			
			-- click the from button to search in the From: field
			click button "From" of window 1
		end tell
	end tell
end searchMailboxForMailFrom


on gotInterfaceScripting()
	if gotPanther() then
		-- check to see if assistive devices is enabled
		tell application "System Events"
			if UI elements enabled then
				return true
			end if
		end tell
		
		tell application "System Preferences"
			activate
			set current pane to ¬
				pane "com.apple.preference.universalaccess"
			set the dialog_message to "This script utilizes " & ¬
				"the built-in Graphic User Interface Scripting " & ¬
				"architecture of Mac OS X " & ¬
				"which is currently disabled." & return & return & ¬
				"You can activate GUI Scripting by selecting the " & ¬
				"checkbox “Enable access for assistive devices” " & ¬
				"in the Universal Access preference pane."
			display dialog dialog_message buttons {"Cancel"} ¬
				default button 1 with icon 1
		end tell
	end if
	return false
end gotInterfaceScripting


on gotPanther()
	-- get the system version
	set the hexData to system attribute "sysv"
	set hexString to {}
	repeat 4 times
		set hexString to ((hexData mod 16) as string) & hexString
		set hexData to hexData div 16
	end repeat
	set the OS_version to the hexString as string
	if the OS_version is less than "1030" then
		display dialog "This script requires the installation of " & ¬
			"Mac OS X 10.3 or higher." buttons {"Cancel"} ¬
			default button 1 with icon 2
		return false
	end if
	
	return true
end gotPanther

New Theme

2005-06-21T00:00:00+00:00

I’ve been tinkering with CSS and Drupal today, trying to get a better understanding of the theme system.

As a result, I’ve got a new theme as the default on the site. It’s called Heatwave (if you live in London, you’ll understand why I chose that name…) and it’s basically a cleaned up and pared down version of the Croxted theme that I was using before.

I’ve been trying to simplify the design as much as possible, removing duplication in the css and trying to make the whole thing a bit lighter. I’ve also tried to reduce the impact of the tabs (which you generally only see when editing pages).

There are probably still some things wrong or missing, so let me know what you think, especially if you see anything particularly hideous. The old Croxted theme is still available, so if you log in you can switch back to it if you prefer.

The Importance Of Syntax

2005-06-16T00:00:00+00:00

During a random session searching for Python html generating tools, I came across an interesting article about language syntax.

I wonder if he’s looked at Dylan?

Searching For Open Source Code

2005-06-15T00:00:00+00:00

Koders is a bit handy. It’s a search engine specifically for open source code.

Open Source Licensing

2005-06-08T00:00:00+00:00

Recently I’ve been working on some odds and ends that I wanted to Open Source, so I had to ponder which license to release them under.

This is a surprisingly gnarly question, and I’m not sure if I’ve got the final answer but right now I’ve gone for the GPL, although I did have some concerns about it.

Universal Binary Guideline Notes

2005-06-07T00:00:00+00:00

Richard Buckle has posted some useful notes on Apple’s newly published Universal Binary Guidelines.

I’d add a couple of things:

Move On Up

Migrating to XCode/Mach-o is all very well, but my gut feeling is that the higher level language you are using, the less pain you are likely to suffer, so I would change the emphasis slightly.

If you are still using Carbon plus C/C++, consider moving to Cocoa plus Objective-C, Java or Python. This is only repeating what Apple have been saying for a while.

Let Someone Else Store Your Data

I concur completely that byte swapping will be the big deal. All of my decade of (often painful) experience working on cross platform projects and porting games tells me that most of the hard work comes in this area (alignment and other things may affect performance, but byte swapping will introduce most of the bugs).

The best way to minimize the pain is to move to some sort of structure storage library. I suspect that people using Core Data, or one of the object persistence APIs in java or python will win big here - always assuming that the maintainers of whatever they are using do a good job!

Short of that, write your own simple API for file operations, add an endian tag to your file formats, and make sure that you stream data in and out of your API in primitive sized chunks (ie pack/unpack structures rather than writing them directly), so that they can be auto-magically swapped later.

Feed Me: Bayesian News Filtering

2005-06-07T00:00:00+00:00

Josh Portway pointed out a post on the FutureOfTheBook blog, talking about a BBC prototype for a system which performs Bayesian filtering on news feeds.

This is something I’ve been thinking about for ages, and funnily enough I started implementing it last week!

The BBC prototype uses email to deliver the news. I decided to write mine as a Python CGI which effectively acts as a filter, sitting between you and a list of feeds. It’s called Feed Me.

The idea is that Feed Me will download and categorise the news items according to your Bayesian “goodness” filter, and then present you with a sorted list. It also appends little voting links to each article allowing you to tweak the filter, telling it you want items more or less like that one.

At the moment the code is very rough, and only works on my Powerbook, but unlike the BBC one I have implemented the actual Bayesian filtering (using crm114), so it is actually quite close to being useful!

I’m going to spend some more time working on this thing, cleaning it up and getting it to a stage where I can use it myself. In the meantime, I’ve made a project page which I’ll try to update as I do things.

Bayesian Filtering With Python

2005-06-07T00:00:00+00:00

I’ve been playing around with Bayesian filtering and Python.

One of the results is this little Python library which provides a very simple wrapper for the CRM114 Discriminator.

I’ve also written a few additional notes on how to build CRM114 for MacOS X.

UK Mac Developer Group

2005-06-06T00:00:00+00:00

Richard Buckle and I moderate a mailing list for Mac, iPhone and iPad developers based in the UK and Europe.

The target audience is a technical one - Cocoa and Carbon programmers writing in Objective-C, C++, Python, Ruby, Java and so on. It isn’t intended as a help group for Mac or iPhone users - there are other resources for that.

Why Another List?

The list isn’t a replacement for Apple’s own lists - and as such it’s generally very low volume. If you have a specific question about Cocoa you’re probably best off posting to Apple’s Cocoa list. Sometimes, though, it’s nice to be able to ask a question of people who you actually know - and of course there are times when the question is specifically relevant to the UK or Europe. We encourage posts about recruitment and the sharing of work - as long as they are relevant and focussed.

For more information, to subscribe to the list via the web, or to read the messages without subscribing, go to http://yahoogroups.com/groups/uk-mac-dev/.

Membership to the list is moderated (in other words, we have to approve your membership). This is a mere formality to try to prevent spammers from joining - so please include a sensible comment when you apply for membership.

If you have any problems, please mail uk-mac-dev-owner@yahoogroups.com.

First Posts

The first post that any member makes to the list is subject to approval by the moderators.

Again, this is just to stop spammers from joining the list just to post spam. The down-side of this is that your first message may take a while to turn up, because we have to notice that it is awaiting approval! Once you’ve posted something that is clearly not spam, we’ll switch your address over to unmoderated and hopefully subsequent posts will be faster.

Job Postings

Job postings are fine as long as they are directly relevant and not repeated unreasonably often (in case of doubt, ask uk-mac-dev-owner@yahoogroups.com). If you don’t want to go to the hassle of joining the list just to post a job message, you are welcome to send it to uk-mac-dev-owner@yahoogroups.com and we will post it for you.

Monthly Meeting

We also have a regular monthly meeting in London, at a pub. This is an informal opportunity to gossip, talk shop about Macs and programming, and to drink beer - all without having our non-technical friends and partners sighing, tutting or leaving us in disgust! These days this meeting is officially a NSCoder night, and is organised via the @nscodernightlon twitter account. It’s also advertised on our mailing list and the Cocoa Heads mailing list, so effectively it’s also the London meeting for a number of groups.

History

We set up the list after meeting at an Apple Script kitchen given by Chris Espinosa in Stockley Park about ten years ago. We realised that we’re a relatively rare breed, we enjoyed meeting other like minded folks, and we really ought to try to keep in touch with each other.

Isn’t The UK In Europe By The Way?

Yes it is :)

I stupidly named the list “UK” at the beginning, despite half the people at the Apple Script kitchen being from mainland Europe. I realised my mistake quickly, but the mailing list name has stuck.

Decoding Crash Logs

2005-06-06T00:00:00+00:00

Richard Buckle has released a new version of decodecrashlog.

decodecrashlog is an open-source, BSD licensed Python program that can apply a CodeWarrior CFM link map to a Mac OS X crash log from a "release build", recovering the function names and offsets.

UI For Complex Boolean Searches

2005-05-11T00:00:00+00:00

John Siracusa’s Tiger review includes a digression about boolean searches, which struck a chord with me.

He says “I think Apple has an insultingly low opinion of users’ ability to understand nested boolean logic. They may not know what it’s called, but the concept is intuitively understood”, and cites a number of Apple’s programs such as Mail and Spotlight which duck this issue by only allowing a single group of tests to be OR-ed or AND-ed, rather than allowing a query that can contain both ANDs and ORs, such as:

(genre is "Anime" or genre is "Game") and my rating is greater than 2 stars

This is something that’s always annoyed me too. Most applications don’t do it, and, I’ve usually found the interface to be pretty poor in the ones that do.

I’ve always thought that the best solution to this problem is to allow the “simple” expressions to be named and saved, and then to allow one simple expression to refer to the results of others. Siracusa actually mentions my approach as a “cumbersome” workaround, but I disagree, I think it’s actually a good pragmatic compromise. In my opinion, the alternative approach of allowing nested expressions and trying to implement an interface which allows them to be edited “in-line” in one big chunk generally ends in tears!

I can see that having to create named sub-clauses is not the fastest interface to use if you want to create a query on the fly, but then I find that any query requiring nested ANDs and ORs is likely to take more than one attempt to get right, and is also likely to be reused, so having an interface that forces you to name the sub-clauses and save them for later is no bad thing.

To illustrate what I mean, imagine a search filter interface which lets the user specify that a rule is true only if either “no test is true”, “any test is true”, or “all tests are true” (in other words, NOT, OR or AND). This is the interface that is provided with programs like Mail, iTunes and Spotlight.

Now imagine that one of the tests that you can specify is that another rule is true. This is the crucial step that is missing from most of Apple’s search interfaces (and lots of others), for no sensible reason that I can see.

The above example would be composed into two sub-clauses:

"Is Anime or Game" if any of the following is true: genre is "Anime" genre is "Game" "Is Good Anime Or Game" if all of the following is true: my rating is greater than 2 stars rule "Is Anime or Game" is true

The advantages of this approach are:

you have all you need to make arbitrarily complex rules, but at each stage of editing, you only have to deal with one kind of comparison - not, or or and - which can be specified by a pop up menu in a clean way
there's no nesting, so all the tests can be listed in a vertical list with no indenting or bracketing required
this approach forces decomposition of a complex problem into simpler steps, which as programmers we know to be a good approach
by naming and subdividing even relatively simple search clauses, and allowing them to be saved for later, we promote re-use, which turns out to be handy too

For me, just the simple addition of the ability to refer to the result of one rule or filter when defining another would vastly improve the utility of many programs. I’m even quite surprised that Apple hasn’t come up with a reusable interface for this sort of thing. It’s clearly something that would benefit from standardisation (like choosing files does, for example), and it ought to be relatively simple to come up with an API which dealt with displaying, editing, and storing rules whilst allowing application-specific code to be plugged in to implement that actual evaluation of results.

Software Page Updated

2005-05-10T00:00:00+00:00

I’ve just noticed an embarrassing error on this site - the software link on the menu bar was doing a search of all blog posts under the software category, instead of linking to the Elegant Chaos Software page as it should have done.

Doh! I’ve fixed it now - apologies for the confusion.

XCode: Am I Missing Something?

2005-04-09T00:00:00+00:00

I’ve recently been spending a bit of time working with Apple’s IDE XCode.

It’s a reasonably mature product now, but I have to say I am very surprised at how limited it is in some ways.

The class browser, for example, seems extremely basic - no folding editor, no support for adding new methods and classes, no factoring, and code completion which is not a patch on VisualAssist.

It also doesn’t seem very modular, so I don’t get the impression that there are lots of third party enhancements available to fill in the gaps.

I have spent the last few years berating Metrowerks for allowing their IDE to become so clunky and old. Now that I’ve seen the major competition, I’m beginning to understand why it might be that way!

XCode is not terrible, but it’s crude enough to make me wonder - am I missing something?

I’d also be very interested to hear whether it’s possible to use Eclipse to do Cocoa development on the mac…

Apple & Third Party Developer Opportunities: The Third Way

2005-04-01T00:00:00+00:00

In the past, Apple has flip-flopped between two positions:

1) We provide the APIs, you (the developers) write the applications.

We don’t want to tread on your toes by writing products that compete with you, so we will provide only trivial applications, and sometimes we’ll even suppress our own internally developed applications, because they are too good.

This is what did a lot of harm to technologies like AppleScript, and PowerTalk. Apple allegedly had a great Mail client for PowerTalk, and a great editor for AppleScript, but didn’t want to release them for fear of offending the competition.

2) We want this platform to kick ass, so we’ll write some kick-ass applications.

We don’t really care about treading on your toes, because frankly we are the only people who have the whole picture. We know the system inside out so our applications can really integrate with it, and each other, resulting in a really nice user experience.

We don’t want to stop you from developing, but you’d be better off not developing anything too good otherwise we’ll have to do our own version and give it away for free.

Position (2) seems to be the default with OS X, and in some ways it’s hard to knock it. Admittedly some people get hurt along the way (Watson? Konfabulator? Come to think of it, anything that Arlo Rose was ever involved in), but all’s fair in love & war, and to be honest, an out-of-the-box Mac running the stock applications is really quite a nice tight ship these days, so it’s hard to argue with it from a user’s point of view.

It occurred to me today that there is a third way! It’s something that Apple is already doing in many ways, but they should definitely be encouraged to do more of it, and perhaps it should be formalised in some level of the Apple echelons of power.

Basically it’s the WebKit approach. WebKit is the rendering code that Safari uses. What Apple did when they developed Safari was to factor out the actual rendering code, put it into a reusable Framework called WebKit (something that other developers can use), and then build Safari on top of it.

Apple still gets to make a nice tight application that is well integrated with the OS and with other applications, but they also make a large chunk of the “heavy lifting” code available to everyone else. So someone like Omni can come along with their competing browser, and recode it to use WebKit. Now Omni can concentrate on adding the features that make OmniWeb better/worse than Safari (depending on your point of view), whilst offloading all of the support headache of the web rendering code onto Apple.

This approach is great for lots of reasons.

It encourages Apple to push the envelope, because they are doing something that benefits all developers. This is good because there are an awful lot of talented engineers at Apple. They might be working on Fireware Hard Disc drivers in the daytime, but they might well also have a great idea for an improved email client. It’s probably a good thing if those ideas can be expressed within Apple.

It also allows Apple to provide well integrated out-of-the-box solutions for whatever they deem to be the core tasks that an average user wants to perform, which, let’s face it, benefits us all if it means that more people buy Macs.

It lowers the entry point for Application development, which gets more talent into the Mac development space. It’s now feasible for me to write a customised web browser with WebKit. I might not do something very polished without a lot of work, but it might allow me to show you my great idea for browsers, which someone else may then run with.

It also encourages (forces even?) developers to focus on the added value (sorry) that their application is bringing to the domain they are working in, and they’ve got more time to focus on that code. The result for the user should ideally be an application that raise the bar another level past Apple’s default one, rather than just doing the same thing in a different way.

As I’ve said, this has already happened with WebKit, and because of the modular way in which a lot of software is developed these days, it is happening in other areas of the operating system - I suspect as a result of code which Apple engineers develop first for their own benefit, then realise that they can generalise for public consumption.

I suggest, though, that Apple make this a policy at some level. Yes, we will develop a mail client, web client, address book, music player, etc, etc, but each time we do so, we will ensure that the core ‘engine’ is provided as a public framework or API which other OS X developers can use. This will be good for them, and great for us.

More thoughts on code-as-a-database

2005-03-28T00:00:00+00:00

My store-code-as-a-database plan is not an original one, and it wasn’t even original years ago when I first started thinking about it - something like Apple Dylan was doing a version of it ten years ago, based no doubt on earlier work by others.

It’s amazing, however, how few people are doing it today. I’m not entirely sure why, but I think it’s a mostly a problem of inertia.

As programmers we invest an awful lot of time in learning to use our tools well. The good programmers do anyway - the lazy ones who just learn enough to enter code and compile it tend to be just as lazy when it comes to designing and implementing code, and about learning more about their craft in general; which I guess is what stops them from being good programmers in the first place.

This investment really bumps up the cost of moving to a new coding environment. It’s hard to justify switching from tool A to tool B just for one or two new features, if it means that you’re going to have to relearn a bunch of stuff that you currently feel very comfortable with. Even when those features are really cool and you can see the massive long term benefits.

And even if we are brave (foolish?) enough to switch to a new IDE/framework/language, most of us have to work with other people, who aren’t so brave, or have their eyes on other short term goals. So most of the “day job” work tends to stay being done with what we know, which of course increases the amount of time and investment in that environment, and reinforces the whole problem.

I don’t think that there are any easy solutions to this, but I guess it does illustrate that any new programming environment which goes down this path needs to do it’s best to support interoperability with existing tools (and by extension with colleagues who are still using them).

I suspect that what this means, perhaps, is that we have to start by creating an environment which acts as a kind of glue to tie together existing stuff, rather than starting from scratch.

Much as I’d like to build something which worked with a nice shiny new dynamic, reflexive, introspective, all-object language, maybe I’d be better off working out a way to wrap up C++ and Java in XML.

Much as I’d like to force everyone to use my system, perhaps I’d better figure out a way to get raw C++ files in and out of the system intact.

Much as I’d like my new tool to act as a it’s own version control system (each code block is an object in the database, so you can do seriously fine-grained version control), perhaps I’ll have more joy if I make something that just works with subversion.

Much as I’d like to make something that allowed truly collaborative programming, maybe I should just make the thing play well with SubEthaEdit!

All of this goes against the grain for me, in the sense that I have some sort of unformed idea in the back of my head of what this new system will be, and whilst I can’t quite figure out the exact shape, I know that it’s supposed to be clean and elegant, not a kludgy mess of glue on top of old tools.

Going down the interoperable route is probably more realistic though, and more achievable. If I start with stuff that already exists, I might even get something working within my lifetime…

Contracting Work

2005-03-28T00:00:00+00:00

Just on the off-chance, I thought I should mention that I’m looking at the moment.

Ideally something that I can do from London, and something fairly short term (3 months or less), but I’m flexible.

If it was Mac based and Cocoa, Python, Java or similar, that would be great. If it was C++ that would be ok. If it was Windows based that would be ok… probably… if the money was good ;)

Behind the Red Shed, with Jonathan 'The Wolf' Rentzsch

2005-03-28T00:00:00+00:00

Nice interview with Jonathan Rentzsch in DrunkenBlog

Extensible Programming for the 21st Century

2005-03-24T00:00:00+00:00

For a long time I’ve been wanting to develop an authoring environment which stores code as some sort of structured database, rather than just a bunch of text files.

Imagine if the program is stored as XML, for example. Perhaps not optimal from an implementation point of view, but it’s a good way to visualise what I’m on about.

Obviously it’s easy to extract the raw source code from the XML if that’s what you want/need to do.

But it’s also possible to extract the actual relationships between program elements, as long as you have some basic knowledge of the object model being used to represent a program.

Suddenly it’s a lot easier to analyse the code. It’s also easier to transform it, so refactoring becomes much simpler.

If you have another language with a compatible representation, you can even automatically translate the code.

Finally, and perhaps best of all, you can add arbitrary meta-data to your code. This could be textual documentation (much more sensible that trying to embed documentation into comments a la JavaDoc), diagrams, optimisation hints, test data for unit tests, whatever.

Anyway, what triggered this particular post was that I came across a reference to an article on the Dylan list, which touches on some of these ideas.

SmartSlab

2005-03-23T00:00:00+00:00

In case anyone’s been wondering, here’s what I’ve been working on recently.

A couple of weeks ago I spent Sunday and Monday in a freezing garage in Bath, playing with some prototype slabs (my role mostly consisted of watching other people do stuff, throwing in the occasional suggestion, and getting really cold).

I’ve not been writing anything that interfaces directly with the hardware, but it’s still a really nice change to be working on something that has a physical side to it.

Now Playing

2005-03-23T00:00:00+00:00

I’ve been having fun over the last couple of days playing with a nice combination of SOAP, Applescript, Amazon, Drupal and iTunes.

The result of this unholy alliance is my Now Playing script, which is yet another of those publish-what-iTunes-is-doing-as-if-anyone-cared tools.

At the moment I’ve got an applescript with an idle loop that gets the current track info from iTunes, looks it up on amazon to get a picture, then writes the information out to a small html file.

I’ve then got a bit of PHP code in a custom Drupal block which reads in the html.

At some point it would be nice to merge these into one script, but I think that there might be issues with a perl script on the server talking to an application running logged in as another user (time for a bit more experimentation).

I’m really impressed at the ease with which one can make SOAP requests from Applescript. This stuff has apparently been around since 2001, but I’ve been so wrapped up with Championship Manager that I had no idea.

Thanks to Tim Jarrett, for some scripts which helped set me off in the right direction.

As it happens, his Amazon Handler script seems to be a bit out of date now, so after a lot of head scratching, I figured out how to do things using the latest Amazon SOAP api.

By way of a thank you, here’s some example code. I haven’t generalised things as much as Tim did, so this function is for a very specific job (look up a track and return urls for the amazon page and an image), but it should give you some idea:


on amazonLookup(trackName, bandName)
	try
		set soapParameters to ¬
			 { ¬
				|SubscriptionId|:"1HT82XR9S43B5V8YDQ02", ¬
				|SearchIndex|:"Music", ¬
				|Title|:trackName, ¬
				|Artist|:bandName, ¬
				|ResponseGroup|:"Small,Images" ¬
			}

		using terms from application "http://www.apple.com/placebo"
			tell application "http://webservices.amazon.com/onca/soap?Service=AWSECommerceService"
				set soapResult to call soap ¬
					{ ¬
						method name: "ItemSearch", ¬
						method namespace uri: "http://webservices.amazon.com/AWSECommerceServices/2005-02-23", ¬
						parameters: soapParameters, ¬
						SOAPAction: "http://soap.amazon.com" ¬
					}	
			end tell
		end using terms from
		
		set soapOutput to |items| of soapResult
		set itemList to |item| of soapOutput
		repeat with i in itemList
			try
				set a to the itemattributes of i
				if the productgroup of a is "Music" then
					if the artist of a is bandName then
						set imageURL to the |url| of the smallimage of i
						set trackURL to the detailpageurl of i
						return {ok:true, track:trackURL, image:imageURL}
					end if
				end if
			end try
		end repeat
	end try
	
	return {ok:false}
end amazonLookup

It took me a long time to figure out exactly what parameters I was supposed to be supplying in the SOAP call (the Apple documentation is a bit sketchy), but once I did, it worked like a dream, and it opens up all sorts of interesting possibilities.

What you get back from the call soap call is a compound record which follows the same structure as the XML SOAP reply. From there it is easy enough to extract the information that you’re after.

As I mentioned, this code just gives a very specific example of one call to Amazon to do one thing. There’s a lot more you can do, and the Amazon documentation seems quite extensive.

Have fun…

The Next Hypercard

2005-03-17T00:00:00+00:00

I’m involved in the early planning stage of a couple of projects which might both describe their aims as “to create the next Hypercard”.

This has been an ambition of mine for the last ten years at least, but unfortunately it’s an ambition shared by an awful lot of other people, and so it has become a bit of a hackneyed phrase. Of course, what we definitely don’t want to do is create a Hypercard rip off. There are already some around, they are probably great, and I wish them all well.

What we want to do is to distill the essence of Hypercard - what it was about it which was so fantastic at the time - and apply those same principles to the environment we find ourselves in today.

One thing I’m surprised about is that I couldn’t find a concise description anywhere on the web of what exactly it was that made Hypercard great. As a result, here’s my first attempt: [What Made Hypercard Great?]. I’d be very interested to hear people’s comments.

Technorati Profile

2005-03-17T00:00:00+00:00

Whatever I do, my Technorati Profile link seems to be producing errors. Not quite sure why.

I blame Kevin Marks (completely arbitrarily you understand). Hopefully he’ll read this post and fix it for me ;)

Update: I’ve deleted all the claims and started again… the JavaScript error was down to Drupal helpfully inserting a <br> tag in the middle of the Technorati script tag.

Some Other Interesting Projects

2005-03-17T00:00:00+00:00

On the topic of Hypercard, there seem to be a few other interesting authoring tool projects around. I’ve listed a few below: anyone got any more I should look at?

Squeak has some pretty big names behind it.

PythonCard seems a way away from prime time, but sounds like an interesting project, with a good motto: “Simple things should be simple and complex things should be possible.”

The Skate project aims to recreate SK8, which was sort of like Hypercard on steroids, written in MCL.

Web Disruption

2005-02-11T00:00:00+00:00

Phase 2 of the great Elegant Chaos relocation is about to get underway.

As a result, the web site may be down for the next week or so. Normal service will be resumed as soon as possible…

My First Drupal Module

2005-01-04T00:00:00+00:00

… by Sam, aged 35 (and a bit).

I’ve written a Drupal filter module called [autolink], which works in a similar way to the glossary module except that it creates links to external sites.

The idea is that every time I enter a particular word or group of words (like Tom Smith for example), the filter module will automatically create a link for me.

This is obviously a bit of a blunt instrument (what if I want to refer to a different Tom Smith?), but it is a handy way of avoiding having to enter common links again and again.

I am thinking about ways of making it a slightly sharper instrument, but for now the basic thing works and will be published here soon!

Drupal Site Now Live

2004-11-26T00:00:00+00:00

I’ve switched things over that my new Drupal based site is live. For the time being you can find the old site at http://old.elegantchaos.com.

There are still quite a few things that I haven’t moved across - some because I haven’t got round to it, others because they are no longer relevant.

Elegant Chaos

Agents, Idealism, And Capitalism

Cynical Idealism

Information Should Be Free

Everything Else Should Be Free, But It Isn’t

Is The Problem AI?

The Revolution Will Not Be Hand-Coded

AI Is The Mother Of All Disruptors

Anyhoo

Fixing The ActionBuilder Plugin

The Deadlock

Unexpected Lack Of Items In The Bagging Area…

Smoke Em If You Got Em

Other Fixes In Passing

Localizable String Catalogues in Swift Packages

Lots Of Small Targets

Using Catalogues In Packages

Which Catalogue Was It Again?

Building Outside Xcode

Other Possible Patterns

SwiftUI Navigation Pain

Actions

SwiftUI Navigation vs Actions

Watching The Path

Not Using NavigationPath

Imperfect Solutions

Tune In Next Time…

Am I Holding It Wrong?

All Quiet In The Western Isles

The Stack

Isn’t This A Solved Problem?

The Stack Isn’t Reminders

The Stack Isn’t A Methodology

The Stack Isn’t An Issue Tracker

So What’s Left?

Triage?

A Big Heap ‘O Stuff

Give It A Go

App Storage

Is That The DRY-Violation Klaxon I Hear?

Shome Mishtake Shurely?

A Better Way™

Caveat Emptor

Caveat Emptor

Release Tools Tagging

Tagging

Archiving And Submitting

Continuous Build Injection

Build Variables

Legacy Cleanup

Development Notes

Asymmetry

Future

Release Tools (Fastlane For Idiots)

Scripting Like An Animal

Basic Design

Build Numbers

Version Tags

Why Are You Telling Me All Of This Again?

Inconsistent Tags

Or…

Dipping A Toe Into The AI Waters

Summarising Notes

On-Device

Adding The Code

So Far, So Meh…

Conclusion

Dual Universe

SwiftUI Action Abstraction

Toolbars

WOT NO ACTION ABSTRACTION?

Lead Us Not Into Temptation…

PS

Matching Strings In Unit Tests

Matchable

Usage

Compound Types

Swift Testing

There’s Gold In Them Thar Hills

More On Workflow Generation

Not Using `NavigationPath`