bou.io

Syntax highlighting in pure CSS

2025-05-04T00:00:00+02:00

Here’s a novel, fun, and (I think) not completely stupid way of highlighting source code on the web:

Transparent text clipping on a background drawn in css,
using the ch and lh units to fit the monospace font grid.

Click to see what I mean:

#include <stdio.h>

int main() {
  printf("Hello, World!");
  return 0;
}

The C code is the raw content of a <pre>...</pre> element. There are no spans or custom classes in the html. Instead, the colors are drawn as the background of the pre, using css linear-gradients. The css units ch (character (width)) and lh (line height) can target specific character positions on the monospace font grid.

background: 
  linear-gradient(orange,orange) 02ch 04lh / 08ch 1lh,
  linear-gradient(blue,blue) 12ch 04lh / 03ch 1lh,
  ... more linear-gradients
background-repeat: no-repeat;

The example above draws an orange rectangle on line 4, starting column 2 and 8 columns wide, and then another blue rectangle for three columns.

Then it’s just a matter of using the text as a clipping mask:

background-clip: text;
color: transparent;

Why?

I find it a bit sad that the contents of the <pre> element is not the actual code. It really bothers me when demonstrating a javascript snippet (in a <script> element) and then, the same sample code, repeated in a <pre>...</pre> element.

Here’s another hack: let’s render the <script> element visible, and use the same code for the demo and for the sample code:

that’s visible just above to reveal the background behind the code that’s being run 🙃.

Random notes and observations

Drawing rectangle with linear gradients inspired by css-tricks over there. I haven’t found a better way to draw rectangles in css, maybe using the new shape() function?
My first idea involved using the :nth-letter pseudo class. It doesn’t exist.
Of course the ch and lh units only make sense with a monospace font. My father used to write code in Comic Sans, you know.
Inspiration for this post came from writing sample code for tuiles.enliberte.fr and this post by lapcatsoftware.
Here’s some self-highlighting CSS. Not sure if it qualifies as a quine or not.
I need to work on the design of this blog.

Next steps

Of course, the CSS linear-gradients for the samples in the page were written by hand. It’s a bit tedious, yes; I suppose the next step would be to write a custom backend for higlight.js to output css instead of spans.

A Twitter Archive

2025-01-26T00:00:00+01:00

I’ve made an archive of all my public activity on twitter, from 2009 to 2022. I’ve fully switched to mastodon more than two years ago, and this archive has little value, but it is my stuff of little value. I’d be sad to see it gone.

Here it is:

twitter-archive.bou.io

I went the easy way and used this tool by Darius Kazemi on tinysubversions.com. I may spend some time to customize it. Or maybe not.

Multitasking User Interface

2020-02-02T00:00:00+01:00

There’s been some discussion around the iPad’s multitasking user interface, i.e., “how to do several things at once on an iPad?”. The general feeling is that the current UI is clumsy, hardly discoverable; that’s what makes the iPad a consumer device, and not a machine to be used for actual work.

I tend to agree, but I don’t think the current state of the Mac user interface is much better. Let’s review the Multitastking UI on Apple platforms.

iOS

The multitasking UI on iOS is a Zooming User Interface. Tapping an icon zooms into it and the app fills all the screen space, then you can pan to neighbouring apps. On iOS, the screen of the device is actually a window through which we interact with a larger space.¹

Another design metaphor had been crucial on iOS, since iOS 7: the interface is made of layers. The Control Center, for example, appears as a layer above the Springboard or the current app. When Apple design documents refer to depth and translucency, this is what it is about.

The iOS user interface is a 2.5D virtual space, made of flat translucent panels, where you can zoom, pan, and move panels in front of others. This is similar to the multiplane camera used in traditional animation.

This design paradigm works really well on iPhone, where only one app is used at a time.

On iPad, several apps are on screen at the same time and things get a little more complicated. Slide Over still makes sense: one apps floats in a layer above the rest.

But in Split Screen, the layers metaphor breaks: both apps are on the frontmost layer.

This brings two major design issues:

Even if two apps appear frontmost, this is a fallacy: only one actually receives keyboard input.
Opening an app on iOS is easy: tap it, and it takes the full screen. But this doesn’t work for multitasking: specific gestures are needed to invoke these modes. A proper iPad OS will require a new interaction design.

macOS

The Mac traditional UI is the Desktop; modern macOS is a lot more. Here’s a partial list of multitasking interactions on macOS:

Click an app window to move it to the front;
Double-click the app icon in the Applications folder;
Click an app icon in the Dock;
Minimize a window into the Dock;
Swipe three fingers up to lay out all the windows side by side (Exposé);
Swipe three fingers left or right to switch to another Space;
Make a window full-screen in its own Space;
Make two windows split-screen in the same Space;
Enter Mission Control to move windows to Spaces and switch to a Space;
Pinch four fingers to enter LaunchPad and click an app;
Open Spotlight and type the app name;
Last but not least, hit Command-Tab to switch to another app and move its windows to the front.

The classic Desktop metaphor is simple: each window is a paper sheet on a physical desktop: you can stack sheets, move them around, or put them side-by-side.

The Dock, that originated in NeXTSTEP, is more abstract; in a way, it’s just an ever-present list of app and file icons.

The more modern features, Mission Control and LaunchPad, use a virtual space model, like in iOS: the screen is a camera that pans laterally.

Unfortunately, when it comes to switching apps, these models each have their own behavior:

In the Command-Tab app switcher, only the launched apps are visible; the active app is first, and then they’re ordered from most recently used to least.
In the Dock, app icons appear in their launch order, and stay in that order unless manually reordered. In that case, app icons remain in place even when the app is quit.
In Mission Control, Spaces with full-screen apps are automatically reordered when each app is activated; this can disabled and Spaces can be manually rearranged.
Finally, the icon order and the folders in LaunchPad don’t match the contents of Applications folder, and are also distinct from the ordering of the pinned apps in the Dock.

When working with several apps, we often switch between three or four apps. The fact that the icons and windows reorder themselves in different ways is a big cognitive load: we can’t build a mental map of the virtual space.

I wish macOS stopped mixing metaphors. Maybe fortunately, switching between native apps is becoming irrelevant.

The future is the web (as usual)

The real multitasking in modern computing is in the tabs of the browser window. I hit Ctrl-Tab even more often than Cmd-Tab. In fact, in Safari, the Tabs Preview is like a small, app-specific Mission Control.

We need a simpler, unified model. I don’t have one: if you listen to me, we’ll go back to a strict Desktop system. Maybe let’s stop thinking in terms of “apps”, and start saying multi-context instead of multitask.

It’s bigger on the inside. ↩

Une brève histoire de l’heure d’été

2018-10-28T00:00:00+02:00

English title, if I ever translate it: “A brief history of french/european daylight saving time policies”.

Ce post reprend un thread twitter d’octobre 2016. Quitte à le recycler régulièrement, autant l’héberger moi-même.

Aujourd’hui on change d’heure - on passe de UTC+2 à UTC+1. Alors que le méridien de Greenwich traverse la France.

🤔

Jusqu’au 19e siècle, il n’y a pas d’heure légale en France. Tout le monde vit à l’heure du soleil. Sauf le chemin de fer, qui base ses horaires sur l’origine des réseaux (Paris).

En 1884 a lieu la Conférence Internationale de Washington, avec pour but de définir le Méridien 0. En lice, Paris et Greenwich. C’est Greenwich qui l’emportera. En échange, les Anglais promettent d’adopter le système métrique.

Les Français boudent un peu et en 1891, définissent l’heure légale au « temps moyen de Paris ». Puis en 1911, à « l’heure de Paris moins 9 minutes et 20 secondes », ce qui correspond à GMT, mais sans référence étrangère. 🐔

En 1916, c’est la guerre, il faut économiser l’énergie : on passe pour la première fois à l’heure d’été. Ça se fait à 23h. L’heure en France alterne alors entre GMT+0 en Hiver et GMT+1 en été.

En 1942, c’est derechef la guerre. L’Allemagne impose son heure à mesure qu’elle avance. Le 14 juin 1940 à 23h, il est minuit, et Paris est officiellement en GMT+2. La zone libre reste en GMT+1. C’est le bazar, en particulier pour les horaires de trains. En février 1941, Vichy s’aligne sur l’heure allemande, soit GMT+1 pour tout le monde (c’est l’hiver).

En 1945, c’est compliqué: on est en été, GMT+2. Le gouvernement provisoire veut repasser en GMT+0 et abandonner l’heure d’été au passage. Le plan est de repasser en GMT+1 en septembre 1945, puis GMT+0 en novembre, deux mois plus tard. Patatras, le 5 novembre, ce deuxième changement est annulé. On reste à l’« heure allemande d’hiver », GMT+1 toute l’année.

En 1972, le Bureau international des poids et mesures établit le Temps Atomique International. On ne parle plus de GMT, mais d’« UTC » pour le franglais “Universal Time Coordinated”.

Puis en 1976, après le choc pétrolier, le gouvernement réintroduit l’heure d’été. La France alterne depuis entre UTC+1 l’hiver et UTC+2 l’été. Fin, pour le moment.

Ajout - octobre 2018

La Commission Européenne a organisé cet été une consultation publique sur les “Summertime Arrangements” qui a obtenu plus de 4 millions de réponses, ce qui est probablement un record. Les résultats sont sans appel :

Il ne reste plus qu’à choisir entre l’heure d’été et l’heure d’hiver… dans chaque pays membre.

NSBundle.current

2018-03-07T00:00:00+01:00

When accessing resources from my code, I sometimes need to specify the current bundle, but the existing API isn’t great. Here’s an alternative.

Apps typically load resources via the Main Bundle (Bundle.main), but sometimes this is not what we want. For example, in plugins or frameworks, the code of the framework will need the bundle for that framework. Another example is with unit tests: I often load mock data from json files, packaged in the .xctest bundle. In the context of the unit tests, the Main Bundle is the xcunit binary (or the app, for UI tests).

The existings solutions to find the current bundle are:

Bundle(identifier: String), which is OK, but a bit tedious;
Bundle(for: AnyClass), which is mostly what we want. It uses the Objective-C internal tables to find which loaded bundle contains the code for the passed class.

The problem with Bundle(for: AnyClass) is that sometimes, there is no class; in Objective-C, this was somewhat of an exception, but a Swift framework could really be just structs and enums.

The workaround, that I had been using for years, is to create a dummy class; in Swift, it can even be a local type:

func foo() {
    class _k {}
    let bundle = Bundle(for: _k.self)
    ...
}

It’s smart, and it works, but it’s a bit too smart. Really, it’s just a workaround to please the API.

I just want to write let bundle = Bundle.current.

Luckily, Nicolas Bachschmidt just gave me the right idea last week:

Use backtrace() to find the function pointer of the caller;
Use dladdr() to find the executable image path containing this function;
Find the loaded bundle with this executable path.

I was afraid Step 1 was impossible in Swift, as the backtrace() function is not available, but Thread.callStackReturnAddresses() works just as well.

Step 3 is more difficult. Given the executable path, we could rewind the directories and find the containing Bundle. However, the internal structure of mac apps, iOS apps, plugins, frameworks, and unit tests is complex. Frameworks, for example, use symlinks internally to reference the executable.

Instead, we’re going to use Bundle.allBundles and Bundle.allFrameworks, which will give the full list of loaded Bundles, and find the one matching the .executablePath.

Finally, since resolving the executablePath of a Bundle is a bit expensive, we’re going to cache the result.

Here’s the code, as a Bundle extension:

extension Bundle {
    class var current: Bundle {
        let caller = Thread.callStackReturnAddresses[1]

        if let bundle = _cache.object(forKey: caller) {
            return bundle
        }

        var info = Dl_info(dli_fname: "", dli_fbase: nil, dli_sname: "", dli_saddr: nil)
        dladdr(caller.pointerValue, &info)
        let imagePath = String(cString: info.dli_fname)

        for bundle in Bundle.allBundles + Bundle.allFrameworks {
            if let execPath = bundle.executableURL?.resolvingSymlinksInPath().path,
                imagePath == execPath {
                _cache.setObject(bundle, forKey: caller)
                return bundle
            }
        }
        fatalError("Bundle not found for caller \"\(String(cString: info.dli_sname))\"")
    }

    private static let _cache = NSCache<NSNumber, Bundle>()
}

I was surprised how easy it is to call low-level C functions from Swift; in fact, I wrote a first version in Objective-C, but it turned out to be more more awkward.

With this, I can finally write Bundle.current everywhere, instead of Bundle.main or Bundle(for: AnyClass).

🎉

Let’s just use the NS prefix in the title, for reference. And I can’t let it go. Yet. ↩

Type-Safe UserDefaults

2017-10-14T00:00:00+02:00

If you’re familiar with Cocoa developement, you probably know what UserDefaults are. Otherwise, a quick recap: (NS)UserDefaults is the macOS/iOS/tvOS name for “user preferences”. The NSUserDefaults API, while very powerful, has all the traits of Objective-C: it’s dynamic, weakly-typed, string-based, and quite verbose.

Recently, two very good implementations of Type-Safe User Defaults have made the news:

Mike Ash on Friday Q&A, protocol-based, using Codable;
Macmade’s version (xs-labs) using Mirror and KVO.

Because the world probably needs yet another one, here’s mine¹:

The Problem

An entry in UserDefaults is: a value, of a given type, with a given key. Naively, a small layer above UserDefaults could look like this:

struct Prefs {
    var username: String? {
        get {
            return UserDefaults.standard.object(forKey: "username") as? String
        }
        set {
            UserDefaults.standard.set(newValue, forKey: "username")
        }
    }
    var firstLaunchDate: Date? {
        get {
            return UserDefaults.standard.object(forKey: "firstLaunchDate") as? Date
        }
        set {
            UserDefaults.standard.set(newValue, forKey: "firstLaunchDate")
        }
    }
    // ...
}

You can see where this is going: getters and setters have to be written for each entry, the key is copy-pasted, and we have to cast to the expected type manually. It’s tedious and error-prone.

The Solution My Own Quick Hack

Let’s define a TSUD struct, with a T generic type, a key attribute, and a value accessor to UserDefaults:

struct TSUD<T> {
    let key: String
    init(_ k: String = #function) { key = k }
    var value: T? {
        get { return UserDefaults.standard.object(forKey: key) as? T }
        set { UserDefaults.standard.set(newValue, forKey: key) }
    }
}

It can then be used like this:

struct Prefs_ {
    lazy var foo = TSUD<Date>()
    lazy var bar = TSUD<[String]>()
    lazy var baz = TSUD<TimeZone>()
}
var Prefs = Prefs_()

That’s all there is to it! We can then write:

Prefs.foo.value = Date()
Prefs.bar.value = ["nineteen", "eighty-four"]

Prefs.foo.value // "14 oct. 2017 à 16:51"
Prefs.bar.value // ["nineteen", "eighty-four"]

As you can see in the TSUD.init() method, the #function literal is used to define the key with the property name. It’s passed as the default argument value so it’s only written once, as is often the case.

The hardest part is actually to define the UserDefaults key from the property name². Mike Ash’s solution is to define a new type for each UserDefaults entry, and then use String(describing: self); Macmade’s variant is to use the Mirror API to list the properties at runtime. Using #function is just another solution that seems simpler, but it comes with caveats.

Why `lazy`?

Also, what’s that weird global var Prefs = Prefs_() ?

#function, as per the documentation, “inside a property getter or setter it is the name of that property”. Unfortunately, that means we can’t write:

struct Prefs_ {
    var foo = TSUD<Date>()
}

because in this context, #function would be Prefs_. Using lazy, I guess, implicitely creates a closure, which is called as a getter the first time the property is accessed, and #function is foo.

For the same reasons, we can’t write:

struct Prefs {
    static var foo = TSUD<Date>()
}

In this context, #function evaluates to -, which doesn’t look like a good key for a UserDefaults entry. This is the reason for the global var Prefs, which is just a singleton trying to pass for a namespace.

–

That’s all for today! Here’s the playground as a snippet.

Apparently, I’m writing Swift now, and against all odds, I actually enjoy (parts of) it. Hopefully, next post before 2019.

On the other hand, I have never written a JSON parsing library. ↩
At least Objective-C has proper introspection! Also, Macros! ↩

NSObject+PerformIfResponds

2016-03-10T00:00:00+01:00

¹Raise your hand if you’re an Objective-C developer and you’re sick of writing this:

if([obj respondsToSelector:@selector(foo)])
{
    id foo = [obj foo];
}

Unless you’ve switched to full-time Swift, in which case you can do this:

let foo = obj.foo?()

I want that, but for Objective-C.

id foo = [[obj please] foo];

I had been thinking about doing this for years, but only actually tried recently.² It turns out it’s not very difficult, but we’ll have to cover a lot of topics first:

Objective-C Message Dispatch
Forwarding
NSInvocations
Boxing

ObjC Message Dispatch — A (largely incomplete) summary

Messages are the fundamental feature of Objective-C, far more than objects and class inheritance; classes are just the basic mechanism for dispatching messages. It probably shoud have been called “Messaging-C”.

Sending a message to an object goes like this:³ ⁴

Messages are compiled to objc_msgSend().
objc_msgSend() looks for an implementation in the Class’s Methods
No implementation? Is there another object that can handle it?
Nobody? Maybe we can just create a new Method out of nowhere?
Well then, doesNotRespondToSelector:. ☠️.

Let’s examine it step by step.

objc_msgSend

At compile time, clang replaces all messages with objc_msgSend() calls.

[obj foo]
➡️ objc_msgSend(obj, "foo")

There are a lot of gory details regarding how parameters and return values are passed. In particular, methods that return structs are compiled to a different version of objc_msgSend():

[view frame]
➡️ objc_msgSend_stret(view, "frame")

The reason for this is that functions that return objects (id) put their return value in a register. This isn’t possible for structs, for which we must allocate space on the stack. ⁵

Inside objc_msgSend(), the regular method lookup works like one would expect: it uses the object’s isa (its class pointer) and the SEL (the selector, or method name⁶) to find the IMP (the method implementation) which is just a function pointer of the form:

*(id self, SEL _cmd, ...)

The two first args are the implicit arguments available in all Objective-C methods: self, and _cmd. With the types of the other arguments and the return value, they form the Method Signature, which is expressed as an Objective-C Type encoding. For example:

- (void)foo:(int)bar; // encodes as "v@:i"

v means the return type is void,
@ the first argument is an id (self),
: the second argument is a SEL (_cmd),
i the third argument (the first explicit arguemnt) is an int.

objc_msgSend then jumps to the address of the IMP function, and program execution continues from here⁷.

Forwarding

That was just the regular case. But what if there’s no IMP? Enter Message Forwarding.⁸

In the forwarding mechanism, messages are objects, too. An NSInvocation is an object that represents an actual message (a selector) send to an object (its target, or receiver), with its arguments.

NSInvocations are strange and beautiful beasts. They can be seen are blocks, or closures: they are Function Objects. But NSInvocations can also be worked with entirely at runtime. As clang emits the correct call to obj_sendMsg, NSInvocation setups the call, but at runtime.⁹

Using NSInvocation goes like this:

NSInvocation * invocation = [NSInvocation invocationWithMethodSignature:/*...*/];
invocation.target = (id) ;
invocation.selector = (SEL) ;
[invocation setArgument:(void*) atIndex:(NSInteger)];
...
[invocation invoke];

This iss how you setup an NSInvocation yourself in order to send a message.
When you receive an invocation, that is how you set the return value for the caller:

...
id result = ...
[invocation setReturnValue:&result]; // a void*

NSInvocation copies the result, and uses the method signature to return it properly for objc_msgSend or objc_msgSend_stret.

Boxing

Let’s look at our proposed solution again:

id foo = [[obj please] foo];

It looks like -foo returns an id. What about those?

CGRect r = [[obj please] frame];
NSInteger i = [[obj please] count];

Moreover, I’ve purposedly left out an important detail:

What do we return if the method isn’t implemented?

When the method returns an object, nil is the easy answer, but what about other types? We need a way to specify a fallback value. Something like:

[[obj pleaseOtherwiseReturn:42] count];

So. Boxing.

Boxing is what we use whenever we need an object, but really only have a scalar or a struct, or anything non-object.

In Objective-C, Boxing is “putting stuff in NSValue” — NSNumbers are NSValues, by the way. It’s used to store ints in NSArrays.

Key-Value Coding uses it all the time: the KVC primitives return objects, but sometimes the underlying values are structs. KVC automatically boxes them:

CGRect r = [view frame];
NSValue * v = [view valueForKey:@"frame"]; // Get the CGRect with -CGRectValue

Unsurprisingly, NSValue is based on Objective-C Type Encodings. Any C or Obj-C value can be boxed in an NSValue:

typedef struct { int x, y } MyStruct;
MyStruct myStruct = ...;
[NSValue valueWithBytes:&myStruct objCType:@encode(MyStruct)]

That’s what the @( <thing> ) syntax does at compile time. It just creates an NSValue, or directly an NSNumber with the contents.¹⁰

@implementation details

Here’s what I’d like to write:

[[obj performIfResponds] doSomething];
[[obj performOrReturn:@"foo"] foo];
[[obj performOrReturnValue:@YES] bar];

The “perform…” methods return a proxy object, that forwards messages to the original target, only if it responds to them. Otherwise, the proxy grabs the NSInvocation and sets a fallback return value.

The public methods are this NSObject category:

@interface NSObject (PerformProxy)
- (instancetype)performIfResponds;
- (instancetype)performOrReturn:(id)object;
- (instancetype)performOrReturnValue:(NSValue*)value;
@end

instancetype, of course, is a lie, to make clang and Xcode happy: the returned object is a proxy. Additionally, we need two performOrReturn- methods to deal with the boxing issues we’ve seen earlier: the underlying objc_msgSend variant is different, the NSInvocation has to match it.

Internally, the proxy object looks like that:

@interface PerformProxy : NSProxy
{
    id _target;
    const char * _returnType;
    void(^_returnValueHandler)(NSInvocation* invocation_);
}

We’ll initialize a PerformProxy using the original receiver as the target and set the correct return type; let’s keep the NSInvocation handler aside for now..

In PerformProxy itself, NSForwarding works in two steps: the first is the “fast path” that’s used to redirect the message to another object — that’s what we want, if the target actually responds to the selector — and the “slow path”, where we can actually play with the NSInvocation.

@implementation PerformProxy

...
// Try to forward to the real target
- (id)forwardingTargetForSelector:(SEL)sel_
{
    if ([_target respondsToSelector:sel_]) {
        return _target;
    } else {
        return self;
    }
}

// Otherwise, handle the message ourselves
- (NSMethodSignature *)methodSignatureForSelector:(SEL)sel_ 
{
    NSString * types = [NSString stringWithFormat:@"%s%s%s",_returnType,@encode(id),@encode(SEL)];
    return [NSMethodSignature signatureWithObjCTypes:types.UTF8String];
}

- (void)forwardInvocation:(NSInvocation *)invocation_ 
{
    _returnValueHandler(invocation_); 
}

@end

What about _returnType and _returnValueHandler? They were passed to the PerformProxy at initialization, by the -performIfResponds methods of the NSObject category. Again, it’s only useful in cases where the real target doesn’t respond to the message.

@implementation NSObject (PerformProxy)

// Messages returning `void`: no need to set a fallback return value.
- (instancetype)performIfResponds
{
    return [[PerformProxy alloc] initWithTarget:self returnType:@encode(void)
                             returnValueHandler:^(NSInvocation* invocation_){}];
}

// Messages returning an object: simply set the fallback as the invocation return value.
- (instancetype)performOrReturn:(id)object_
{
    return [[PerformProxy alloc] initWithTarget:self returnType:@encode(id)
                             returnValueHandler:^(NSInvocation* invocation_){
                                 id obj = object_;
                                 [invocation_ setReturnValue:&obj];
                             }];
}

// Messages returning a “value”: use the objcType of the passed NSValue, and “unbox” it into a local buffer to return it.
- (instancetype)performOrReturnValue:(NSValue*)value_
{
    return [[PerformProxy alloc] initWithTarget:self returnType:value_.objCType
                             returnValueHandler:^(NSInvocation* invocation_){
                                 char buf[invocation_.methodSignature.methodReturnLength];
                                 [value_ getValue:&buf];
                                 [invocation_ setReturnValue:&buf];
                             }];
}

@end

That’s all! Easy, right.

Well, OK, That wasn’t exactly trivial. That’s not much code, either. So what does it give us?

The good stuff

Existing patterns become easier to write, and easier to read.

We never use respondsToSelector: anymore: all our delegates, and protocols with @optional methods are sent through performIfResponds, and we don’t worry about it anymore.

Another great usecase is for using recent API while maintaining compatibility with older systems. For example, here’s how we’re dealing with 3D Touch Quick Actions, while maintaining compatibility with iOS8:

UIApplication.sharedApplication.performIfResponds.shortcutItems = /* array of items */;

The -setShortcutItems: message is simply ignored is UIApplication doesn’t handle it. Before that, we used to write abstraction layers for compatibility.
It’s completely useless now.

We’re starting to use new coding patterns.

It’s OK not to check anything: just send the message, and if the receiver can handle it, that’ll work.

In the latest version of Captain Train for iOS, we’ve added a new feature called “Contextual Help”. The context is inferred from tags, and tags come from all kinds of objects: ViewControllers, because the the help isn’t the same in the search form or in the cart; and model objects, because there are specific help articles for e.g. Eurostar or Deutsche Bahn train tickets.

In the end, the easiest method was to simply “ask” objects for their help tags:

for(id obj in contextObjects) {
	tags = [obj performOrReturn:nil].helpTags;
}

Instead of defining categories and superclasses, we just have abstract protocols.

PerformIfResponds allows to specify a default implementation, and this is starting to look very similar to the Protocol-Oriented-Programming from that famous “Grumpy” talk at WWDC 2015.

The not-that-good stuff

Well, there are few drawbacks.

Forwarding is slow. In the good scenario, when the target object does respond to the message, it’s 20 times slower than regular method dispatch. The slow path, with NSInvocation, is 100x slower.
There’s no type safety. At all. It’s very easy to use the wrong -performOrReturn: variant, and then it’s Game Over. ¹¹

What if we made a new language that made this kind of things easy, fast, and safe?
Oh, wait.

I’ve made the most “Objective-C” thing that I could, using dynamic message resolution, invocations, and other runtime features, and the end result a fundamental feature of Swift.

Mmm.

–

Source code is on github for your amusement, and I’m on Twitter for comments.

This is the blogpost version of a talk at Cocoaheads Paris in March 2016. Also, it may be my last post ever on Objective-C. Let’s make it count. ↩
The solution is very “Objective-C” in spirit. I’m disappointed to only find it now. ↩
See also the official documentation ↩
… and Bill Bumgarner’s tour of objc_msgSend, from the last decade, which is still largely relevant ↩
This is the dirty little secret of Objective-C: although we don’t know which method will be actually called until runtime, its return “variant” has to match the version of objc_msgSend that was chosen at compile time. Not so dynamic. ↩
Selectors are (almost) C string pointers. They’re uniqued at compile time, so they can be compared with ==. ↩
it doesn’t call the implementation function, it jumps to it. That’s “tail call optimization”: the called method can return directly to the caller, without passing through objc_msgSend again. That’s also why objc_msgSend doesn’t appear in call stacks. . ↩
There’s this terrific blogpost reverse-engineering how __forwarding__ exactly works internally. ↩
By the way, blocks are objects too. And you can send -invoke to an NSBlock. ↩
Xcode 7.3 brings objc_boxable, which lets you declare boxing support for custom structures. ↩
Foundation logs a rather nice error message for this: NSForwarding: warning: method signature and compiler disagree on struct-return-edness of 'someMethod'. Signature thinks it does not return a struct, and compiler thinks it does. ↩

Revisiting POI Clustering, Deciding Against It

2016-01-21T00:00:00+01:00

This is the blogpost version of a talk at Cocoaheads Paris in December 2015¹.

TL;DR: 17-second video at the end.

–

A few months ago, Captain Train opened a dataset of train stations in Europe. Here are the train stations in Paris:

Let’s zoom out to get a better view: Whoops. The UI stalls for a dozen of seconds, which isn’t entirely a surprise: the whole csv is 23,000 POIs².

Once we’re done waiting, this appears:

This is Very Bad™.

There are three probems here:

Performance is abysmal, even on a powerful device.³
It’s hard to use: with points so close to each other, it’s impossible to accurately pick one.
There’s no information hierarchy: this looks like a forest. There’s no structure at all.

The usual solution to this problem is to group nearby POIs together, using a POI Clustering algorithm⁴:

Here we have grouped pins together, making sure there aren’t too many pins on the map. Performance is smooth again; and users can select each pin easily. Problem solved!

Except it’s not.

The cluster points do not exist in the real world.⁵ They are just flat, arbitrary groups: they don’t add structure to our map.

In fact, even with that noisy forest of pins, we could see that our dataset had way fewer stations in eastern Europe. Now everything looks the same everywhere. We’ve actually made things worse.

This is 💩

On top of that, most POI clustering algorithms introduce weird artifacts. For example, cluster groups tend to be aligned on a grid:

Again, this grid does not exist. Why does it appear on my map?

Let’s rethink this.

What are we trying to do exactly?

We’re showing the location of train stations on a map. But a train station isn’t just a latitude/longitude coordinate. It also has an address, in a City, a Region, a Country, etc. These are the groups we should use instead of making them up.

Let’s try with just the countries:

Each country is overlaid by a green layer (in MapKit, this is an MKOverlay). The opacity of the layer is relative to the number of stations in that country.

We don’t see individual items, but suddenly we’ve learned something new about our dataset from this highest-level view.

Let’s try something else, zoom in just a little bit, and display the station again:

Each green dot is a train station.

This reveals the shape of the train network. This is something that we couldn’t see in the first visualization.

The dots are not interactive. They would be too small to be tapped and selected individually anyway, but this also helps performance. In fact, the dots are, once again, drawn as a single layer. In MapKit, this is done using an MKOverlay again, instead of full-fledged MKAnnotations.

Different zoom levels

A map view is actually a Zooming User Interface. In a ZUI, “folders” are simply zoomed into instead of being opened. POI groups are really the same thing as folders, and POIs are individual documents. When a “folder” is closed, you can’t see its contents, but you can have a preview.⁶

It’s the same with maps: when you zoom out, you can’t see individual features, but the big picture is revealed.

In our example, we have three levels of detail:

Europe countries, or “Folders”
Train network, or “Preview”
Individual stations, or “Files”

And then it’s just a matter of switching the level of detail dynamically. We simply reveal more details as we zoom in:

That’s better

Of course, the naïve “Let’s dump thousands of POIs on the map” approach is easier at first. But when you run into performance issues, don’t just head to stackoverflow and pod install the first thing you find. Find what makes sense for your dataset, and display relevant, usable information at different scales.

–

The demo app I made for the screenshots in this post is available on Github.

The working title for this was POI Clustering considered harmful. The slides are over here. ↩
POI stands for Point of Interest: the location of “something” on a map.
It is also the meaning of the ⌘ symbol on your keyboard. ↩
MapKit isn’t to blame here. MKMapView can handle thousands of POIs just right: it’s smart enought to filter out what’s not currently visible. The problem is to actually display so many translucent views. ↩
Most opensource libs use a quadtree-based algorithm which is faster, but less accurate than K-means clustering. ↩
Here’s a hint: there is no “Cluster” entity in our data model. ↩
Jef Raskin, one of the Macintosh’s fathers, later advocated Zooming UIs in The Humane Interface. ↩

Functional Programming in Objective-C

2015-02-09T00:00:00+01:00

This is Objective-C:

@interface NSArray (Map)
DefineMethod(map, id (^)(id), NSArray*);
DefineMethod(reduce, id (^(^)(id))(id), NSArray*);
@end

@interface NSString (Funcs)
DefineProperty(uppercaseString, NSString*);
DefineMethod(plus, NSString*, NSString*);
@end

int main()
{
  Print(@[@"foo", @"bar", @"baz"].map(NSString.uppercaseString));
  // -> ( FOO, BAR, BAZ )
  
  Print(@"Hello".plus(@" I love you"));
  // -> Hello I love you
  
  Print(@[@"foo", @"bar", @"baz"].reduce(NSString.plus));
  // -> foobarbaz
  
  InstanceProperty massUppercaser = NSArray.map(NSString.uppercaseString);
  Print(massUppercaser(@[@"abc",@"def"]));
  // -> ( ABC, DEF )
  
  InstanceProperty exclamate = NSString.plus(@"!!");
  Print(@[@"quick", @" to the batmobile"].map(NSString.uppercaseString).map(exclamate).reduce(NSString.plus));
  // -> QUICK!! TO THE BATMOBILE!!
}

Yes, it compiles and runs.

The NSArray.map, NSArray.reduce, and NSString.plus methods are implemented as regular objective-C methods:

@implementation NSString (Plus)
- (NSString*) plus:(NSString*) string
{
    return [self stringByAppendingString:string];
}
@end

The main idea is simply to define additional, parameter-less methods (e.g. -[NSString plus] instead of -[NSString plus:]) that return a block, and that blocks takes a parameter that’s used as the parameter of the actual method. It’s okay if you need to reread that sentence, but really, it’s not that complicated.

Of course, writing those block wrapper implementations for each and every method would be painful. Lucky for us, the objective-C runtime lets us add methods on demand with resolveInstanceMethod. This was almost surprisingly straightforward.

The whole gist is here. The usual warnings apply: this is a hack and probably full of bugs, don’t start doing anything serious with it.

A few random notes/things I learned/thoughts I had along the way:

On Functional Programming:

exclamate = NSString.plus(@"!!"); is effectively a partial application of the NSString.plus function. It produces a new function, with one parameter less. (Is it Currying or Partial Application? I never know, my understanding is that it’s basically the same thing for functions with 2 or less arguments.)
I’m a functional programming noob (or rather I forgot what I learned in school.). These block properties are probably called monoids or something.
I first saw this technique in OCMock 2, where it enables a simpler, shorter syntax.

On the ObjC runtime:

To add class methods at runtime in Objective-C, one actually adds instance methods to the metaclass. The class methods of a class are the instance methods of its metaclass.
Similarly, resolveClassMethod could largely be ignored. When resolveClassMethod fails, the runtime calls resolveInstanceMethod on the metaclass.
I only coded support for object (id) argument and return types, because I care about my mental health. I had started experimenting with methods with >1 arguments, using NSInvocation an va_args, but this was getting out of hand.
I’m swizzling [NSObject resolveInstanceMethod], because YOLO. This is incredibly powerful: this is just like patching the dispatch mechanism of the language.
Clang supports the dot notation for properties as well as argument-less methods, including Class methods. [Class method] can be written Class.method. However, no autocomplete for these.

On Objective-C and language design:

I’ve said it before, Objective-C is really malleable. All in all, it doesn’t complain too much: dynamic languages FTW. The less enjoyable part of it all is, as usual, the fucking block syntax.
It’s fun to use a language with a coding style that is completely different from what it’s been designed for. However, if you take a look at mid-90s Objective-C source files and compare it to recent UIKit, you’ll see that it doesn’t really seem like the same language. In old-style ObjC, the id type was implied for arguments and return types (it still works), and methods often returned self instead of void.
Higer Order Messaging isn’t really a new concept in the Objective-C community. Of course, there’s the KVC collection operators such as in [items valueForKeyPath:@"@sum.price"]; the very existence of NSInvocation is also proof that this had been a field of interest for the language maintainers a long time ago.
Before UIKit, before stackoverflow, there was this page on cocoadev.com: http://cocoadev.com/HigherOrderMessaging
I wonder what the Target-Action paradigm of Cocoa would look like if it was implemented with Functional Programming in mind. I guess we’ll see in a few years. Also, why are you all excited about this UXKit thing in Photos.app?
One final thought: I implemented block methods as readonly properties. What if methods were readwrite properties? We could write NSString.plus = <something else> at runtime, either for the whole class or for a specific instance.

Anyway, it looks I’m ready for Swift. Can we have more dynamism please?

Git Clones of opensource.apple.com

2014-12-02T00:00:00+01:00

Apple isn’t really famous for its contributions to open source projects: its latest major effort was Mac OS forge. As the name implies, it dates from when sourceforge was cool.

On the other hand, Apple publishes the code ot the opensource components it uses at opensource.apple.com. I wrote a small script that scrapes the projects metadata, downloads all the tarballs and recreates git repositories at github.com/unofficial-opensource-apple.

Apple releases opensource code for four Products: Developer Tools, OS X, OS X Server and iOS. For every Release of these products, they (in theory) also publish the sourcecode of the Version of the opensource Project.

There are 509 projects in total. Some are never actually referenced in a Product. (Like DarwinInstaller, of course, or zfs, unfortunately.) The source code is made available as tarballs for each version, 5334 in total.

tatin.py attempts to recreate a more usable git repository for each of these projects:

It downloads all the version tarballs of this project,
- Creates a new commit for each version
- Adds tags for each Product Release referencing this version.
- The Modified-At header returned by opensource.apple.com when requesting the tarballs is relatively coherent after 2010. (The epoch of opensource.apple.com seems to be the 5th of February 2009.) This date is used as the GIT_AUTHOR_DATE and GIT_COMMITTER_DATE.
- Additionally, the git user is set to opensource.apple.com. ¹
Creates a repository at github.com/unofficial-opensource-apple/<project>

Warning: it’s written in Python, a language in which I am a total noob. Basically, it worked once on my machine. On the other hand, there’s probably something useful to be made of it. Pull requests are welcome.

GitX does a decent job to quicky look at the history of a repo. If you’d rather use the CLI, this pretty format is the best I’ve found so far:

objc4 $ git log --pretty=format:"%C(bold)%s %C(dim)%ci%Creset%d"
646 2014-10-30 20:55:50 +0000 (HEAD, tag: OS_X-10.10, origin/master, origin/HEAD, master)
551.1 2013-10-29 00:21:36 +0000 (tag: OS_X-10.9.5, tag: OS_X-10.9.4, tag: OS_X-10.9.3, tag: OS_X-10.9.2, tag: OS_X-10.9.1, tag: OS_X-10.9)
532.2 2012-09-28 15:34:15 +0000 (tag: OS_X-10.8.5, tag: OS_X-10.8.4, tag: OS_X-10.8.3, tag: OS_X-10.8.2)
[...]

Enjoy, and don’t hesitate to send comments and suggestions via twitter.com/_nb or submit an issue on github.

A nice side effect of overriding the git commiter/author and the commit/author dates is that the commit hashes stay identical when a repository is recreated from scratch. ↩

Cross-Platform Development

2014-11-05T00:00:00+01:00

When I started working as a software engineer ten years ago, I was tasked to write the Mac version of an existing project, and I was to do it using Visual Studio, on Windows. As a matter of fact, the first decision I made was to pick a cross-platform development framework.

Over the following years, I spent countless days trying to make cross-platform code look decent both on Windows and Mac OS.¹ This might explain the strong opinions I have on cross-platform development.

Cross-platform development lets you write common features just once for several platforms.

But there’s something inherently wrong with that. Say you want to write an app for iOS and Android? Each platform has its own story, its own set of metaphors users are familiar with. You can write a common cross-platform UI, but be prepared to spend a lot of time tweaking on each platform. Otherwise, it will just look weird.

But maybe you want the UI to behave the same on all platforms, because “the experience is at the core of your product”? Guess what, that’s bullshit. Exactly 0% of your users care about “cross-platform brand consistency” and no, your product manager does not count. An iOS (or Android, or Mac, Windows, or any OS) user is accustomed to the specific behavior of this platform. How buttons react. How contents scrolls. How text is rendered. There’s a whole lot of untold expectations, and if your app starts to behave differently, it’s just crap.²

Don’t forget your users are not just your users. They are primarily users of a device, and happen to use your app on that device,³ and it coexists with other apps on this platform.

Your app isn’t a platform.

OK, but can we still use a common backend for the app?

You know what? Probably not. There are a lot of non-UI tasks that are specific to the platform. Data storage. Network requests. Background tasks. Push notifications. Password storage.⁴ Encryption. Either you’re going to reinvent the wheel (several times), or you’ll build an abstraction layer with just the common features. With a common backend, you won’t be using platform-specific technlogies, for example, CoreData on iOS and OS X. Maybe CoreData isn’t the right solution for your app, but “I want the same code to run on Android” is not the right reason to rule it out. The same goes for NSURLSession. Do you really need to reimplement a full network stack?⁵

OK, maybe just the business-specific logic then?

If you really want. But is it really that much code? Now your business logic is just a layer between a native UI and a native backend. Is it still worth the effort to integrate alien code in each platform-specific project? Is it worth writing Java ↔︎ C++ ↔︎ Scripting language of the day ↔︎ Objective-C bridges, maintaining and debugging them? And then there’s the project management aspect: do you really want the iOS and Android teams to depend on each other? Whose job is it to maintain the cross-platform code? Neither team is going to be happy with it.

If you‘re going to write apps, make them native. Otherwise it’s just not worth the effort. There’s that amazing cross-platform thing called the web, spend those resources on making a great website instead.

Don’t get me wrong, Qt is (was?) a great developement framework otherwise. ↩
That’s also the reason why websites shouldn’t try to look native. Not only is it impossibly hard for a website to behave like a native app, but “native”, of course doesn’t mean the same thing on different platforms. ↩
On the other hand, they may be users of your service across different platforms. ↩
And now you know why I don’t use Firefox. ↩
And honestly, if you have the resources for this kind of project, writing cross-platform code to save money just doesn’t make sense. ↩

RunLoopRunUntil

2014-10-02T00:00:00+02:00

In the previous post I mentioned a code pattern frequently used in tests:

Run the RunLoop until something specific happens.

I’ve written and rewritten this method, making the same mistakes a few times. Here are two of the latest implementations I had ended up with:

Variant A:

- (void) runUntilBlock:(BOOL(^)())block timeout:(NSTimeInterval)timeout
{
    NSDate * endDate = [NSDate dateWithTimeIntervalSinceNow:timeout];
    do {
        CFTimeInterval quantum = 0.0001;
        CFRunLoopRunInMode(kCFRunLoopDefaultMode, quantum, false);
    } while( [timeoutDate timeIntervalSinceNow]>0.0 && !block() );
}

(Don’t do that.)

Variant B:

- (void) runUntilBlock:(BOOL(^)())block timeout:(NSTimeInterval)timeout
{
    NSDate * endDate = [NSDate dateWithTimeIntervalSinceNow:timeout];
    while( [timeoutDate timeIntervalSinceNow]>0.0 && !block() ) {
        CFRunLoopRunInMode(kCFRunLoopDefaultMode, DBL_MAX, true);
    }
}

(Don’t do that, either.)

Neither felt particularly satisfying:

Variant A runs the RunLoop for 0.0001 second no matter what, then checks for completion and timeout, then runs the RunLoop again, etc. This is me doing polling, but with a small delay, because I was told that polling is bad.
Variant B is a bit better: it runs the RunLoop, blocking until one event is processed (that’s what the third parameter set to true is for), then checks for completion and timeout. However, there again, why am I handling a timeout myself? And why do I need a while loop? Isn’t that precisely the job of CFRunLoop?

The biggest difference between the two variants if the polling mechanism. Typically, the stop condition is expected to be fulfilled after some code runs on the same thread as the RunLoop, which means this code would be called out by the RunLoop. In variant B, that’s the “processed event” that makes CFRunLoopRunInMode return: as soon as the code that fulfills the condition has run, CFRunLoop returns and we check the condition.

On the other hand, if the condition we’re testing only becomes true as the result of code running on other threads, we need to check for completion regularly, and there may be no solution but polling.¹

XCTestExpectation

In Xcode 5, Apple added Asynchronous unit testing, with the new class XCTestExpectation and a category on XCTestCase. It’s really great, and it comes with a few very nice helpers for KVO or NSNotification-based tests. However, it has a few limitations²:

I can only be used for asynchronous tasks with an explicit callback. The test code has to send the -[XCTestExpectation fulfill] message for the test to pass. This is simply impossible in some cases: for example, if the asynchronous task being tested simply changes the value of a flag, there’s no client callback to do this. Arguably, that’s a sign of a bad API, or that you’re trying to test the wrong things. Unfortunately, some of the things I want to do are precisely this kind of “Black Box” tests. (More on that below.)
It can only fail by timing out. If the task finishes, but with an error, should it really send the fulfill message? In my opinion, running the app asynchronously should be not be a feature of the test framework.

CTT

To write the Capitaine Train iOS app, we’ve setup a large suite of tests including high-level integration tests. For example, here’s the simulator running the -test_login scenario.

The early versions of these tests were written using KIF. KIF is really powerful, but after a while, its syntax became too heavy to write dozens of scenarios.

KIF:

[tester enterText:@"testuser@capitainetrain.com" intoViewWithAccessibilityLabel:@"Email Address"];
[tester enterText:@"testpassword" intoViewWithAccessibilityLabel:@"Password"];
[tester tapViewWithAccessibilityLabel:@"Sign In"];

[tester waitForTappableViewWithAccessibilityLabel:@"Search"];

So after a while, we wrote our own system, dubbed CTT. In CTT, the same test looks like:

CTT:

[[CTT match:@"Email Address"] enterText:@"testuser@capitainetrain.com"];
[[CTT match:@"Password"] enterText:@"testpassword"];
[CTT tap:@"Sign In"];

[CTT match:@"Search" options:CTTMatchOptionsTappable];

The whole point of this post (and hopefully, the next ones) is to make me extract CTT, document it, and open-source it.³

By I digress. The hardest thing with Integration testing is to run the app normally, while waiting for specific views to appear. We need to run the RunLoop, until an arbitrary condition becomes true.

CTTRunLoopRunUntil

Here’s what I believe is a correct implementation of CTTRunLoopRunUntil:⁴

Boolean CTTRunLoopRunUntil(Boolean(^fulfilled_)(), Boolean polling_, CFTimeInterval timeout_)
{
    // Loop Observer Callback
    __block Boolean fulfilled = NO;
    void (^beforeWaiting) (CFRunLoopObserverRef observer, CFRunLoopActivity activity) =
    ^(CFRunLoopObserverRef observer, CFRunLoopActivity activity) {
        assert(!fulfilled); //RunLoop should be stopped after condition is fulfilled.
        // Check Condition
        fulfilled = fulfilled_();
        if(fulfilled) {
            // Condition fulfilled: stop RunLoop now.
            CFRunLoopStop(CFRunLoopGetCurrent());
        } else if(polling_) {
            // Condition not fulfilled, and we are polling: prevent RunLoop from waiting and continue looping.
            CFRunLoopWakeUp(CFRunLoopGetCurrent());
        }
    };
    
    CFRunLoopObserverRef observer = CFRunLoopObserverCreateWithHandler(NULL, kCFRunLoopBeforeWaiting, true, 0, beforeWaiting);
    CFRunLoopAddObserver(CFRunLoopGetCurrent(), observer, kCFRunLoopDefaultMode);
    
    // Run!
    CFRunLoopRunInMode(kCFRunLoopDefaultMode, timeout_, false);

    CFRunLoopRemoveObserver(CFRunLoopGetCurrent(), observer, kCFRunLoopDefaultMode);
    CFRelease(observer);

    return fulfilled;
}

The idea is to use of CFRunLoopObservers. As mentioned in the previous post, CFRunLoopObservers are used to setup callbacks at different checkpoints of the RunLoop’s loop:

/* Run Loop Observer Activities */
typedef CF_OPTIONS(CFOptionFlags, CFRunLoopActivity) {
    kCFRunLoopEntry = (1UL << 0),
    kCFRunLoopBeforeTimers = (1UL << 1),
    kCFRunLoopBeforeSources = (1UL << 2),
    kCFRunLoopBeforeWaiting = (1UL << 5),
    kCFRunLoopAfterWaiting = (1UL << 6),
    kCFRunLoopExit = (1UL << 7),
    kCFRunLoopAllActivities = 0x0FFFFFFFU
};

By observing kCFRunLoopBeforeWaiting, we can test for completion on every loop of the RunLoop. Before sleeping (i.e. waiting for an event), the RunLoop has called everything there was to call. That’s the right time to test for completion. This variant also solves the “active polling” scenarios: if the polling_ flag is set, the RunLoop actually never sleeps and run continuously; fulfilled_ is checked on every pass. And of course, contrary to most implementations, including my own, there’s no “minimal delay”, and no additional code to handle the loop or the timeout. That should do the trick.

If you see an error or a possible improvement, please do let me know. Here’s the code for CTTRunLoopRunUntil on github; the repository is pretty bare right now; hopefully, I’ll gradually add all the functionality of CTT into it. It’ll involve poking at UIView internals, accessibility and keyboard private APIs, general UIKit hackery, and some smarty-pants C macros. Stay tuned.

To be exact: if an asynchronous task calls back client code on a secondary thread, the correct way to make the main thread check for completion is by waking its RunLoop using CFRunLoopWakeUp(), not to use polling. Polling is only necessary if the asynchronous task doesn’t call back client code at all. ↩
Additionally, as there’s no XCAssert macro, it looks completely different from the rest of the XCUnit API. I wonder how it prints the error at the correct line without using __FILE__ and __LINE__. ↩
By the way, if you want to help me, we’re hiring. ↩
Until I realize there’s another significant flaw. ↩

Run, RunLoop, Run!

2014-09-17T00:00:00+02:00

Here’s a subject that’s rarely discussed amond developers, even though it’s one of the most important component of all apps: Run Loops. Run loops are like the beating heart of apps; they are what make your code actually run.

The basic principle of a run loop, in fact, is quite simple. On iOS and OS X ¹, CFRunLoop implements the core mechanism used by all the higher-level messaging and dispatching APIs.

What’s a Run Loop Anyway?

Put simply, a run loop is a messaging mechanism, used for asynchronous or interthread communication. It can be seen as a post box that waits for messages and delivers them to recipients.

A run loop does two things:

wait until something happens (e.g., a message arrives),
dispatch that message to its receiver.

On other platforms², this mechanism is called the “Message Pump”.

Run loops are what separates interactive apps from command-line tools. Command-line tools are launched with parameters, execute their command, then exit. Interactive apps wait for user input, react, then resume waiting. In fact, this basic mechanism is found in pretty much every long-lived process. A good old while(1){select();}³ in a server is a run loop.

The job of a run loop is to wait for things to happen. Those things can be external events, caused by the user or the system (e.g. a network request.) or internal app messages, like inter-thread notifications, asynchronous code execution, timers… Once an event (or a “message”) is received, the run loop finds a relevant listener and pass it the message.

A basic run loop is actually fairly simple to implement. Here’s a trivial pseudocode version:

func postMessage(runloop, message)
{
    runloop.queue.pushBack(message)
    runloop.signal()
}

func run(runloop)
{
    do {
        runloop.wait()
        message = runloop.queue.popFront()
        dispatch(message)
    } while(true)
}

With this simple mechanism, each thread will run() its own run loop, and exchange messages with other threads’ run loops asynchronously using postMessage(). My colleague Cyril Mottier pointed me to the Android implementation which isn’t much more complex that that.

What about iOS and OS X

On Apple systems, that’s the job of CFRunLoop, in a slightly more advanced variant ⁴. All the code you write is called out by CFRunLoop at some point, except for early initialization, or if you spawn threads by yourself. (As far as I know, threads created automatically for Grand Central Dispatch queues don’t need a CFRunLoop, but certainly have a messaging system to allow reuse.)

The most important feature of CFRunLoop is the CFRunLoopModes. CFRunLoop works with a system of “Run Loop Sources”. Sources are registered on a run loop for one or several modes, and the run loop itself is made to run in a given mode. When an event arrives on a source, it is only handled by the run loop if the source mode matches the run loop current mode.

In addition, CFRunLoop can be reentered from the application code, either from your own code or within frameworks. As there is exactly one CFRunLoop for each thread, when a component wants to run it in a special mode, it simply calls CFRunLoopRunInMode(). All the run loop Sources that aren’t registered for this mode simply stop being served. Usually, that component eventually returns control to the previous mode.

CFRunLoop defines a pseudo-mode called the “common modes” (kCFRunLoopCommonModes), which is actually a set of modes containing the “normal” run loop modes for your app. The main run loop, at first, runs in kCFRunLoopCommonModes.

UIKit, on the other hand, defines a special run loop mode called UITrackingRunLoopMode. It uses this mode “while tracking in controls takes place” i.e. during touch events. This is very important, because this is what makes tableview scrolling smooth. While the main thread’s run loop is in UITrackingRunLoopMode, most background events, like network callbacks, aren’t delivered. This way, no extra processing is done, and there’s no lag in scrolling. (Well at least, now it‘s your fault.) ⁵

CFRunLoop demystified

If you’ve already debugged an iOS on OS X stack trace, you’ve probably noticed, down in stack trace, an all caps message starting with CFRUNLOOP_IS_CALLING_OUT. When CFRunLoop calls out to application code, it likes to make a show about it. There are six such functions, defined in CFRunLoop.c:

static void __CFRUNLOOP_IS_CALLING_OUT_TO_AN_OBSERVER_CALLBACK_FUNCTION__();
static void __CFRUNLOOP_IS_CALLING_OUT_TO_A_BLOCK__();
static void __CFRUNLOOP_IS_SERVICING_THE_MAIN_DISPATCH_QUEUE__();
static void __CFRUNLOOP_IS_CALLING_OUT_TO_A_TIMER_CALLBACK_FUNCTION__();
static void __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__();
static void __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE1_PERFORM_FUNCTION__();

As you can guess, these functions have no purpose except helping debugging in the stack trace. CFRunLoop makes sure that all application code is called through one of these functions.

Let’s take a look at them one by one.

static void __CFRUNLOOP_IS_CALLING_OUT_TO_AN_OBSERVER_CALLBACK_FUNCTION__(
    CFRunLoopObserverCallBack func,
    CFRunLoopObserverRef observer,
    CFRunLoopActivity activity,
    void *info);

Observers are a bit special. The CFRunLoopObserver API lets you observe the behavior of CFRunLoop and be notified of its activity: when it processes events, when it goes to sleep, etc. It’s mostly useful for debugging, and you probably won’t need it in your app, but it’s there if you want to experiment with CFRunLoop’s features. [Update 2014-10-02: In fact, there are certain purposes where it will be useful: for example, CoreAnimation runs from an observer callout. It makes sense: by making sure all the UI code has run, it executes all the animations at once.]

static void __CFRUNLOOP_IS_CALLING_OUT_TO_A_BLOCK__(
    void (^block)(void));

Blocks are the flipside of the CFRunLoopPerformBlock() API, which is useful when you want to run code “on the next loop”.

static void __CFRUNLOOP_IS_SERVICING_THE_MAIN_DISPATCH_QUEUE__(
    void *msg);

That Main Dispatch Queue tag is of course CFRunLoop speaking to Grand Central Dispatch. Obviously, on the main thread at least, GCD and CFRunLoop work hand in hand. Even though GCD can (and will) create threads without a CFRunLoop, when there is one, it will plug itself in.⁶

static void __CFRUNLOOP_IS_CALLING_OUT_TO_A_TIMER_CALLBACK_FUNCTION__(
    CFRunLoopTimerCallBack func,
    CFRunLoopTimerRef timer,
    void *info);

Timers are relatively self-explained. On iOS and OS X, high-lever “timers” like NSTimer or performSelector:afterDelay: are implementd using CFRunLoop timers. Since iOS 7 and Mavericks, Timers can now have a tolerance on their fire date; this feature, too, is handled by CFRunLoop.

static void __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__(
    void (*perform)(void *),
    void *info);

CFRunLoopSources “Version 0” and “Version 1” are actually very different beasts, even if they have a common API. Version 0 Sources are simply an in-app messaging mechanism and must be handled manually by the application code. After signaling a Version 0 Source (with CFRunLoopSourceSignal()), the CFRunLoop must be awaken (with CFRunLoopWakeUp()) for the source to be handled.

static void __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE1_PERFORM_FUNCTION__(
    void *(*perform)(void *msg, CFIndex size, CFAllocatorRef allocator, void *info),
    mach_msg_header_t *msg, CFIndex size, mach_msg_header_t **reply,
    void (*perform)(void *),
    void *info);

Version 1 Sources, on the other hand, handle kernel events using mach_ports. This is in fact the very heart of CFRunLoop: most of the time, when your app is standing there, doing nothing, it’s blocked in this single mach_msg(…,MACH_RCV_MSG,…) call. If you take a sample of any app with Activity Monitor, chances are you’ll something like this:

CFRunLoopRunSpecific  (in CoreFoundation) + 296  [0x7fff98bb7cb8]
__CFRunLoopRun  (in CoreFoundation) + 1371  [0x7fff98bb845b]
__CFRunLoopServiceMachPort  (in CoreFoundation) + 212  [0x7fff98bb8f94]
mach_msg  (in libsystem_kernel.dylib) + 55  [0x7fff99cf469f]
        2718 mach_msg_trap  (in libsystem_kernel.dylib) + 10  [0x7fff99cf552e]

This is right here in CFRunLoop.c. Just a few lines above, Apple engineers commented with this very relevant quote from the Hamlet soliloquy:

    /* In that sleep of death what nightmares may come ... */

A sneak peek at CFRunLoop.c

Whenever your apps run, the core of CFRunLoop is the __CFRunLoopRun() function, called through the public APIs CFRunLoopRun() and CFRunLoopRunInMode(mode, seconds, returnAfterSourceHandled).

__CFRunLoopRun() will exit for four reasons:

kCFRunLoopRunTimedOut: on timeout, if an interval was specified,
kCFRunLoopRunFinished: if it becomes “empty”, e.g. all the Sources have been removed,
kCFRunLoopRunHandledSource: with the returnAfterSourceHandled flag, as soon as an event have been dispatched,
kCFRunLoopRunStopped: it has been stopped manually with CFRunLoopStop().

Until one of these reasons occur, it will keep on waiting and delivering events. Here’s a single pass looks like, handling the various event types we discussed above:

Call “blocks”. (The CFRunLoopPerformBlock() API.)
Check Version 0 Sources, and call out their “perform” function if necessary.
Poll and internal dispatch queues and mach_ports, and
Go to sleep if there’s nothing awaiting. The kernel will wake us up if something comes up. It’s actually a lot more complex in the code, because a/ the Win32 compatibility code adds a lot of #ifdef #elif noise and b/ there’s a goto right in the middle. The main idea is that mach_msg() can be configured to wait on several queues and ports. CFRunLoop does this to wait for timers, GCD dispatches, manual wakeups, or Version 1 Sources, all at the same time.
Wake up, and try to see why:
1. A manual wakeup. Just continue running the loop, maybe there’s a block or a Version 0 Source to serve.
2. One or several Timers fired. Call their function.
3. GCD needs to work. Call it via a specific ”4CF” dispatch_queue API.
4. A Version 1 Source has been signaled by the kernel. Find it and serve it.
Call “blocks”, again.
Check for exit conditions. (Finished, Stopped, TimedOut, HandledSource)
Start all over again.

Whew. Easy, right? As you may know, CoreFoundation is implemented in C, and frankly doesn’t look too modern. Reading this, my first reaction was “Wow. This is begging for a refactor”. On the other hand, this code is more than field-tested, so I don’t expect a full rewrite in Swift anytime soon.

There’s a code pattern that I’ve been using a lot in recent years, especially in testing. It’s “Run the run loop until this condition becomes true”, the basis for any kind of asynchronous unit tests. Over time I’ve probably written a dozen of variants of this, using NSRunLoop or CFRunLoop directly, doing polling, using timeouts, etc. Now I may be able to write a decent version of it; let’s find out in the next post.

We need a name for the “family of operating systems made by Apple“. “Darwin” wouldn’t work, as it is the name of the underlying system. OS? ↩
In a previous life, I wrote my fair share of Win32 code. ↩
man 2 select, if you’re lucky enough not to know. ↩
CFRunLoop.c is a 3909 lines long, while Looper.java is just 309 lines. ↩
You may remember from 2011 this post attributing the scrolling performance on iOS to a “dedicated UI thread with real-time priority”. It’s been corrected and clarifed later; of course, that was just UITrackingRunLoopMode doing its job. ↩
To be honest, I’m not really familiar with the internals of GCD. I’m trying to guess how this can work, please correct me if I’m wrong. ↩

Espaces insécables fines

2014-07-15T00:00:00+02:00

English/TL;DR The rules of french punctuation require a thin space before some marks. However, there’s no easy way to insert a thin space, so most people just go with regular spaces. That’s a shame.

Vous êtes sans doute déjà tombé sur ce genre de citation :

Les « pains au chocolat » et les « chocolatines ». Qui sont-ils ? Quels sont leurs réseaux ?

Un point pour l’effort : il y a des espaces avant les points d’interrogation et à l’intérieur des guillemets. Par contre, ce point d’interrogation tout seul, c’est n’importe quoi. ¹ Les gens biens utilisent des espaces insécables, qui, comme leur nom l’indique, empêchent de revenir à la ligne n’importe quand :

Les « pains au chocolat » et les « chocolatines ». Qui sont-ils ? Quels sont leurs réseaux ?

C’est déjà mieux. Mais comme le disent Wikipedia et le Dictionnaire des règles typographiques, il faut des espaces « si possible fines »². Comme ceci :

Les « pains au chocolat » et les « chocolatines ». Qui sont-ils ? Quels sont leurs réseaux ?

Sur Mac, avec les dispositions de clavier du système, il est tout simplement impossible de taper une espace fine insécable. Heureusement, on peut les personnaliser ! Il y même une app pour ça : Ukelele permet d’éditer les fichiers .keylayout, en modifiant sur les fichiers existants.

En voici donc deux, basés respectivement sur « Français » et « Français numérique ». Ils permettent d’insérer une insécable fine avec ⌥-espace, l’insécable normale restant disponible sur ⌥-⇧-espace.

Français – insécables.bundle

Il suffit de placer le .bundle dans le répertoire ~/Library/Keyboard Layouts/ puis de faire votre choix dans Préférences Systèmes > Clavier > Méthodes de saisie.

Sauf sur Chrome, qui dessine ce texte un peu plus serré et évite le point d’interrogation orphelin. Comme je n’ai ni la patience ni le talent d’un vrai développeur web, voilà ce que vous devriez voir. ↩
Devant la plupart des signes de ponctuation haute, sauf, en théorie, le deux-points. ↩

Extended Type Info in Objective-C

2014-05-19T00:00:00+02:00

A few weeks ago, I stumbled upon these private methods on NSObject:

@interface NSObject (Private)
-(id)_ivarDescription;
-(id)_shortMethodDescription;
-(id)_methodDescription;
@end

They can be used to obtain debug descriptions of the passed objects:

_shortMethodDescription lists all the instance and class methods of the receiver,
_methodDescription does the same, including the superclasses’ methods,
_ivarDescription lists all the instance variables of the receiver, their type, and their value.

The underlying mechanism is runtime introspection, similar to what Nick Lockwood described in this post on iosdevelopertips.com. The good news is Apple already did the hard work for us. The bad news, of course, is that these methods are “private”.

Here’s an example¹ of what it does:

@implementation MyClass : NSObject
{
    NSString* myIVar;
}

- initWithString:string
{
    self = [super init];
    if(self) {
        myIVar = string;
    }
    return self;
}
@end

void main()
{
    id obj = [[MyClass alloc] initWithString:@"Hello"];
    NSLog(@"%@",[obj _ivarDescription])
}

ivars: <MyClass: 0x8d3d130>:
in MyClass:
    myIVar (NSString*): @"Hello"
in NSObject:
    isa (Class): MyClass
methods: <MyClass: 0x8d3d130>:
in MyClass:
    Instance Methods:
        - (void) .cxx_destruct; (0x2a30)
        - (id) initWithString:(id)arg1; (0x2930)
(NSObject ...)

These are really great debugging tools, and I’ve started to use them instead of -description or -debugDescription. In fact, I’d love the default implementation of -debugDescription to use -_ivarDescription.

(A word of warning, however: these methods are in fact implemented in UIKit on iOS7, and not available at all on Mac OS. Yet.)

The class of the nil object

Here’s the really cool trick: _ivarDescription knows the class of nil object ivars. If we change our example to this:

id obj = [[MyClass alloc] initWithString:nil];

-_ivarDescription still prints out the myIVar iVar as type "NSString*", with a nil value:

ivars: <MyClass: 0x8e40960>:
in MyClass:
    myIVar (NSString*): nil
in NSObject:
    isa (Class): MyClass

What? Nil objects don’t have a class; they’re just nil pointers. _ivarDescription prints the expected class of the instance variable, as specified in the source code.

In fact, the compiler stores extended type info in the binary. That’s really weird: Objective-C treats all objects equals, as ids, and will not complain when assigning an object of a class to a pointer of a different type. I always assumed that class info was lost at runtime; it turns out that’s not true.

Import the Runtime!

Let’s try to reimplement _ivarDescription and see what the runtime has to offer. First, class_getInstanceVariable() lets us get a object’s Ivar given its name, then ivar_getTypeEncoding() will return its type encoding:

> p (char*) ivar_getTypeEncoding(class_getInstanceVariable([MyClass class], "myIVar"))

@"NSString"

Now this is big news to me. According the documentation, the type encoding for objects is simply @. In reality, it’s followed by the actual expected class.

Let’s try it again, with a declared property:

@property NSString * myproperty;

This time we’ll print out the complete attributes for the property:

> p (char*) property_getAttributes(class_getProperty([MyClass class], "myproperty"))

T@"NSString",&,V_myproperty

Again, we find the expected class, in addition to the property-specifc attributes: the storage (weak or strong) and the ivar name.

opensource.apple.com

Digging a little further in the objc4 runtime source code, here’s another surprise: Protocol methods extended type info.

As with ivars, methods arguments expect objects of a specific class, and this information is embedded in the binary. For some reason, it’s only available for methods declared in Protocols, and the public runtime API won’t let us access it, but the private function _protocol_getMethodTypeEncoding() will do.

Here it is in action:

@protocol FooProtocol
- (NSString *) barWithArray:(NSArray*)array; 
@end

> p (char*) _protocol_getMethodTypeEncoding(@protocol(FooProtocol), @selector(barWithArray:), YES, YES)

@"NSString"12@0:4@"NSArray"8

We get extended type info for the parameter and the return type of the method. This is much more than what the public function method_getTypeEncoding() returns:

@implementation FooClass
- (NSString *) bazWithArray:(NSArray*)array_; 
@end

> p (char*) method_getTypeEncoding(class_getInstanceMethod([FooClass class], @selector(bazWithArray:))

@12@0:4@8

In fact, according to the docs, the encoding for this method should be @@:@:

'@' for “object” return type
'@' for “object” receiver (self)
':' for “selector” first parameter (the _cmd hidden parameter)
'@' for the NSArray parameter.

(As far as I know, the numeric values in @12@0:4@8 are not documented; I can only assume these are hints for byte alignment.)

To sum up:
Clang and the objc runtime have partial, undocumented support for Extended Type Info, both for the instance variables and the protocol methods. For some reason, it’s not implemented for regular (non-protocol) methods, what is implemented is private API, and all of it is completely undocumented.

More of this please

It’s a bit complicated to understand the changes in progress in the objective-C runtime, given the lack of regular opensource releases. Extended Type Info in Objective-C currently looks like a work in progress, a partially implemented experiment.

On the other hand, and this is pure speculation, but if this announces a new, more powerful introspection mechanisms, I’m all for it.

Following the minimalist codestyle of the latest post ↩

Objective-C Minimalism

2014-04-13T00:00:00+02:00

I often write small tests programs in Objective-C, to experiment stuff or play with something. Most of the time, I put everything in the main.m file, and get rid of everything else:

#!/usr/bin/env objc-run
@import Foundation;

@implementation Hello : NSObject
- (void) sayHelloTo:name
{
    printf("Hello %s, my address is %p\n", [name UTF8String], self);
}
@end

int main ()
{
    id hello = [Hello new];
    [hello sayHelloTo:@"sunshine"];
}

Everything goes into main.m. Or “Test.m”, or whatever. The main() function can be in any file, you know.
sha-bang objc-run. If you’re not using it yet, objc-run is Ilja A. Iwas’ excellent utility that “makes it easy to use Objective-C files for shell script-like tasks”. Install it with brew install objc-run and chmod u+x your source file, you’re ready to go.
Modules, no precompiled headers. Maybe there are legitimate reasons to keep using #imports and PCH in large projects, but for quick test programs? No way.
No @interface declaration. I inadvertently found out the other day that ObjC let you specify the superclass in the @implementation directive; I’m not sure why this is allowed, and of course you can’t change your mind between the @interface and the @implementation. However, you can omit the @interface block entirely.¹
Implicit parameter types. ObjC methods return types and parameters are implicitely (id), which means -(id)doSomethingWith:(id)param; is exactly the same thing as
-doSomethingWith:param; which I also find clearer.
No arguments to main(). Although it’s considered bad practice, it’s actually legal to write void main () instead of int main (int argc, char**argv). Why bother if you’re not using the arguments?
No return from main(). Strangely, -Wreturn-type doesn’t fire a warning when omitting the return from main(). Anyway, this was pointless as I would return zero². [Update 2014-09-19: Graham Lee tells me that “Since C99, if you return control from main() without a return statement then “return 0;” is assumed.” Cool.]
printf, not NSLog. NSLog is for logging errors, not for text output: I don’t need, and I don’t want to printout the executable path and thread ID with each line.

I’m not sure the Radar team will appreciate this kind of “Sample Project” over full-fledged Xcode templates, but I’ll certainly try it out for the next bugreports.

Clang fires a warning, and you can’t mute that one. In the llvm source, that’s the warn_undef_interface warning, and it’s on the list of warnings that have no associated -W flag. Bummer. ↩
On the other hand, declaring int void main() would fire a 'main' must return 'int' error. ↩

Uploading Screenshots with iTMSTransporter - a Tutorial

2014-03-14T00:00:00+01:00

Great news Mac and iOS developers! You don’t have to upload screenshots manually to iTunes Connect anymore. There’s an Apple-provided tool to help you! It kinda sucks, but that’s OK. At least we can script it.

iTMSTransporter stands for iTunes Music Store Transporter, which means it wasn’t made for you. Actually, I don’t think it was made for anyone; my best guess is that it’s a byproduct of the original iTunes Store development ten years ago, and at some point a manager found out about it and realized it could be used by the Content Providers, where “Content” really means “thousands of hours of HD Video”.

It’s really a weird piece of software: it’s a java command-line tool, which is in fact a WebObjects app running locally. It might be able to run on windows somehow, which partially explains this odd choice of technology.

iTMSTransporter is probably already installed on your mac. It’s embedded inside the Xcode bundle: Xcode App submission and Application Loader both use it under the hood. You can also download it directly from Apple’s servers. At 71MB, it’s a rather large download for what it does, but keep in mind it’s a full-blown WebObjects app. Another weird fact: it comes as a .pkg installer that is not signed. For an App Store management tool, I find this particarly ironic.

Anyway, here it is, right inside Xcode:

/Applications/Xcode.app/Contents/Applications/\
    Application Loader.app/Contents/MacOS/\
    itms/bin/iTMSTransporter

You’ll probably want to alias iTMSTransporter= that thing above.

iTMSTransporter: the CLI

The official documentation for the iTMSTransporter tool can be found in the “Manage Your Apps” section of iTunes Connect, under the “Transporter User Guide” link. It’s just a 9 pages PDF, and it very briefly describes the few options relevant to app developers.

We’ll just run iTMSTransporter with no argument, and after a second or two of WebObjects hard thinking, we’ll get the usage info:

usage: iTMSTransporter [-help <arg> | -info | -m <arg> | -version]   [-o <arg>] [-v
    <arg>]  [-WONoPause <arg>] [-Xmx1024m]
iTMSTransporter : iTunes Store Transporter 1.7.9
 -m <arg>         The -m option specifies the tool's mode.  The valid values are:
                  verify, upload, provider, diagnostic, lookupMetadata, status,
                  statusAll, queryTickets, generateSchema, transferTest,
                  downloadMetadataGuides, iTunesExtraQCDownload, iTunesLPQCDownload,
                  iTunesExtraTemplate, iTunesLPTemplate, listReports, requestReport
 -o <arg>         The -o option specifies the directory and filename you want to use
                  to log output information.  By default, Transporter logs output
                  information to standard out. If you specify a filename,
                  Transporter logs the output to the specified file, as well as to
                  standard out.
 -v <arg>         The -v option specifies the level of logging.  The five values
                  are: off, detailed, informational, critical, eXtreme.
 -version         The -version option should be used by itself and returns the
                  version of the tool.
 -WONoPause <arg> The -WONoPause option is only valid on Windows and its value can
                  be 'true' or 'false'.  If an error occurs during script execution,
                  the process idles because the message 'Press any key...' is
                  displayed on the console and the system awaits a keypress. To
                  avoid this behavior, set this property to true
 -Xmx1024m        Specifies that you want to change the Java Virtual Machine's (JVM)
                  allocated memory by increasing the JVM heap size.  By default,
                  Transporter uses a 512MB heap size. You can use the -Xmx1024m
                  option to specify a 1-gigabyte (GB) heap size. Apple recommends,
                  if needed, increasing the heap size to 1024MB by specifying the
                  -Xmx1024 option and adjusting as needed.

(Just look at the -WONoPause and -Xmx1024m options for a minute. What year is this again?)

The only option we care about is -m for Mode. And the only modes we need are:

lookupMetadata: download the .itmsp stub package of an app;
verify: check if the contents of the modified package if valid before uploading it;
upload: upload it back to the iTunes servers.

That’s how iTMSTransporter works: download the package metadata, tweak it and add our screenshots, then upload it back to iTunes.

The .itmsp package

An .itmsp package is simply a folder containing a metadata.xml configuration file, and a number of associated resources (in our case, App Store screenshots.).

File.itmsp/
     * metadata.xml
     [additional resource files]

Although the doc for iTMSTransporter is very short, the package format and the metadata xml is described at length in the Metadata specification, which is once again available in the “Manage Your Apps” section, under the “App Metadata Specification” link.

Let’s start by downloading the metadata for our app:

$ iTMSTransporter -m lookupMetadata\
    -u [iTunes Connect user name]\
    -p [iTunes Connect password]\
    -vendor_id [App SKU]\
    -destination [local download path]

(The App SKU is whatever we set in iTunes Connect. Also, you don’t really have to pass the password as an argument, iTMSTransporter will prompt you if it is omitted.)

As its names implies, lookupMetadata only fetches the metadata.xml file: the current App Store screenshots are not in the downloaded package.

Let’s take a look inside this metadata.xml file.

<package xmlns="http://apple.com/itunes/importer" version="software5.1">

 [snip] Header info

 <software>
  <software_metadata>
   <versions>
    <version string="8">
     <locales>
      <locale name="fr-FR">
       <title>Very Nice App</title>

       [snip] French description and release notes

       <software_screenshots>
        <software_screenshot display_target="iOS-3.5-in" position="1">
         <file_name>fr-01_SignupScreen-2x.png</file_name>
         <size>60846</size>
         <checksum type="md5">f48951ba3076af6022c9e7dba70a20d9</checksum>
        </software_screenshot>

        [snip] More screenshots

       </software_screenshots>
      </locale>
      <locale name="de-DE">

      [snip] German texts and screenshots 

      </locale>
      <locale name="en-US">
      
      [snip] English texts and screenshots 

      </locale>
     </locales>
    </version>

    [snip] Next "Prepare for upload" version, if any.

   </versions>
   <products>
    <product>

    [snip] Product pricing and availability

    </product>
   </products>
  </software_metadata>
 </software>
</package>

This is essentially the same thing that can be set via the iTunes Connect web UI:

Information for all the “current” versions is available. That means the live version on the appstore, as well as the next version, being prepared on iTunes Connect.
Inside both versions, we find another level of hierarchy, for each locale that’s enabled on iTunes Connect. As you can see, this app is localized in French, German and English.
For each locale, the various texts (title, keywords, description, and release notes) are available, as well as the full list of screenshots.

After the versions, the metadata also contains the iTunes Connect products, beginning with the app itself and including any in-app products. Here you can add products, setup pricing and availability. This would deserve another tutorial; in a free app, there’s not much to see here. Just be aware that you can configure pretty much everything related to IAP right from this xml file.

It’s important to note that unmodified fields can be omitted when uploading an .itmsp package. This means, in our case, that we’ll remove the “Ready for Sale” version info, as we don’t want to mistakenly change its description or release notes. We’ll also get rid of the keywords, description, and releasenotes xml nodes, as we just want to deal with the screenshots.

Here’s the data for a single screenshot again:

<software_screenshot display_target="iOS-3.5-in" position="1">
 <file_name>fr-01_SignupScreen-2x.png</file_name>
 <size>60846</size>
 <checksum type="md5">f48951ba3076af6022c9e7dba70a20d9</checksum>
</software_screenshot>

display_target, obviously, refers to iPhone4/iPhone5/iPad screen size. If you mess up with the image size, iTMSTransporter will stop you.
position lets you control the ordering of the various screenshots. Again, you can’t include more than five images, and you have to be coherent, otherwise iTMSTransporter will complain.
file_name, obviously, refers to the name of the screenshot file. This is really a file name, not a path: the files must actally be located at the root of the itmsp package, not in a subfolder, and certainly not outside the package.
size and checksum. iTMSTransporter tries to be nice with you: if the size and checksum are identical to the previous version of the file, it won’t be transferred.
However, you have to compute the size and the checksum yourself, and you can’t put phony values. iTMSTransporter actually checks the values in the xml against the actual files sizes and hash, and will throw an error if they don’t match. I suppose that it makes sense for Video and Audio Content Providers, who want to make sure they upload the correct files, but for screenshots, it’s more of a nuisance, and we’ll have to automate it.

Scripting it

Now that we know how to fetch the .itmsp package and how to set it up, let’s make it all work!

First, we’ll cleanup the metadata.xml file as much as possible, and remove all the entries we don’t want to change. As I mentioned earlier, that means everything but the screenshots.

Next, we have to match the filenames in the xml with the actual screenshot files. Whether the screenshots were automatically generated by test scenarios, or if they were hand-crafted, remember the files must be placed right inside the package:

VeryNiceApp.itmsp/
    * metadata.xml
    * fr-01_SignupScreen-2x.png
    * fr-01_SignupScreen-568h-2x.png
    * fr-02_SearchForm-2x.png
    * fr-02_SearchForm-568h-2x.png
    [...] all FR, EN and DE screenshot files...

And in metadata.xml:

<locale name="fr-FR">
 <software_screenshots>
  <software_screenshot display_target="iOS-3.5-in" position="1">
   <file_name>fr-01_SignupScreen-2x.png</file_name>
    [...]
  </software_screenshot>
  <software_screenshot display_target="iOS-4-in" position="1">
   <file_name>fr-01_SignupScreen-568h-2x.png</file_name>
    [...]
   </software_screenshot>
   [...]
 </software_screenshots>
</locale>
[...other locales...]

We have to update the size and checksum values to match the values of the files. Here comes the script, which probably can be improved: ¹

ITCPACKAGE=[path to your .itmsp package]
ITCMETADATA=$ITCPACKAGE/metadata.xml

for FILENAME in `ls $ITCPACKAGE/*.png | xargs basename`
do
    FILEPATH=$ITCPACKAGE/$FILENAME
    
    # Remove all non-essential data from the PNGs
    pngcrush -rem alla -rem text $FILEPATH $FILEPATH-crushed &> /dev/null
    mv $FILEPATH-crushed $FILEPATH

    # Compute the md5 of the file and update its value in the xml file
    xml ed -L -u //_:version//_:software_screenshot[_:file_name=\"$FILENAME\"]/_:checksum -v `md5 -q $FILEPATH` $ITCMETADATA
    # Do the same for the file size
    xml ed -L -u //_:version//_:software_screenshot[_:file_name=\"$FILENAME\"]/_:size -v `stat -f%z $FILEPATH` $ITCMETADATA
done

The pngcrush phase makes sure that pixel-identical files are actuall the same on disk. This will be helpful if you add the itmsp packages to your source repository. Additionally, if you need to overlay some text over the screenshots, or embed the screenshot in an iPhone shape, you’ll probably need to add some imagemagick compose steps.

All that’s left is uploading it back to Apple:

$ iTMSTransporter -m verify\
    -f [path to App Store Package]\
    -u [iTunes Connect user name]\
    -p [iTunes Connect password]

$ iTMSTransporter -m upload\
    -f [path to App Store Package]\
    -u [iTunes Connect user name]\
    -p [iTunes Connect password]

Voilà! Look Ma, no iTunes Connect!

Actually, you’ll want to check the screenshots were successfully uploaded – don’t panic if they don’t show up immediately, there’s a small delay before the new screenshots appear on the Web UI.

This proved very convenient for us at Capitaine Train: we do this at every app update, just before uploading the binary. That way, we’re always sure that the App Store screenshots are always up-to-date with the UI tweaks we’ve added in the latest update. In fact, it has turned the very tedious task of uploading screenshots into a minor step of the Release Checklist.

Going Further

As mentioned, the official documentation is available in iTunes Connect, under the links “Transporter User Guide” and “Metadata Specification”. iTMSTransporter is also mentioned in the 2013 WWDC session 306.
Nik Fletcher of Realmac software wrote a little Mac App over iTMSTransporter: Connecter.
There’s a also Ruby wrapper with a somewhat nicer CLI, made by Skye Shaw.

I haven’t (yet) tried to analyze the actual HTTP traffic. I have a feeling it would be easier to reverse engineer the webservices rather than interfacing and parsing the command line. Let me know if you ever look into that :)

You’ll need xmlstarlet: brew install xmlstarlet ↩

Several Approaches to Higher-Order Functions with Objective-C Collections

2014-02-10T00:00:00+01:00

The Objective-C collections classes from Foundation, NSArray, NSSet and NSOrderedSet, already support various forms of functional-style programming.

First, and probably oldest, is:

[NSArray makeObjectPerformSelector:]

Key-Value Coding has:

[NSArray valueForKey:];

and of course, there’s the block-based:

[NSArray enumerateObjectsUsingBlock:]

Similar methods are available for NSSet and NSOrderedSet.

However, I feel there’s room for improvement. I spent a few hours last weekend experimenting with different techniques and API styles for Collections Functional Programming.

Here’s what I want:

A consistent API for NSArray, NSSet and NSOrderedSet,
The basic higher-order functions:
- Map (a.k.a. collect)
- Filter (a.k.a. select)
- Fold/reduce could be added easily, but I never see the need for it.¹

I’ll implement four different approaches, using various Objective-C coding styles:

Style #1 - Block-based enumeration

That’s the most modern and flexible style, but not the most clear or concise. It’s also the one with the most existing support in Foundation, with the base methods available on NSArray, NSSet and NSOrderedSet.

- (void)enumerateObjectsUsingBlock:(void (^)(id obj, NSUInteger idx, BOOL *stop));
- (NSIndexSet *)indexesOfObjectsPassingTest:(BOOL (^)(id obj, NSUInteger idx, BOOL *stop));

This isn’t exactly the API I want, so I’ll add two other methods:

- (instancetype) map:(id(^)(id obj))block;
- (instancetype) filteredCollectionWithTest:(BOOL(^)(id obj))test;

map: and filteredCollectionWithTest: do exactly what they say, and return a collection of the same type as the original receiver: an NSArray for an NSArray, and so on.

Here’s what it looks like in a real world example²:

id heroes = @[batman, catwoman];

id (^getIndentity) = ^id(id hero) {
    return hero.identity; 
};
id identities = [heroes map:getIndentity];
// -> @[@"Bruce Wayne", @"Selina Kyle"];

BOOL (^isBruce)(id) = ^BOOL(id hero) {
    return [hero.identity hasPrefix:@"Bruce Wayne"];
}
id batmen = [heroes filteredCollectionWithTest:isBruce];
// -> @[batman]

Easy, right?

Style #2 - Key-value coding

Like with blocks, there’s already good support in the frameworks for KVC on collections. KVC has a lot o very cool features, including Collections Operators. ³

Map is trivially done using valueForKey::

id identities = [heroes valueForKey:@"identity"];
// -> @[@"Bruce Wayne", @"Selina Kyle"];

Filter requires some work, but it’s also very easy to write:

- (instancetype) filteredCollectionWithValue:(id)value
                                  forKeyPath:(NSString*)key;

Then, back to our real-world example:

batmen = [heroes filteredCollectionWithValue:@"Bruce Wayne"
                                      forKey:@"identity"]; 
// -> @[batman]

Wait a minute! I just cheated. There’s no easy way to do custom algorithms using KVC like the hasPrefix: match we’ve done with blocks.

Until we can write custom collection operators, the workaround is to add the test method directly to the items’ class:

- (BOOL) isBruce {
    return [self.identity hasPrefix:@"Bruce Wayne"];
}

and use it like this:

id batmen = [heroes filteredCollectionWithValue:@YES
                                         forKey:@"isBruce"]; 
// -> @[batman]

It kinda works, but it’s way less flexible that blocks.

On the other hand, it’s completely type-unsafe, and Apple doesn’t seem to care a lot about it: the KVO APIs could use some love, and NSOrderedSet doesn’t support Collection Operators at all.

So: mixed feelings about KVC.

Style #3 - NSInvocation-based enumeration

Moving on, our next coding style is an old lady of Objective-C development, NSInvocation. Usually, when you see NSInvocation, <objc/runtime.h> isn’t very far.

An NSInvocation is conceptually very similar to a block: it records a specific code invocation. But where a block can contain abitrary code, NSInvocation represents the invocation of a message and its parameters on a target.

NSInvocations are rather tedious to create and manipulate. They also have special memory management rules, so handle with care.

Anyway, here’s what the map/filter API looks like with invocations:

- (instancetype) map:(NSInvocation*)invocation;
- (instancetype) filteredCollectionWithTest:(NSInvocation*)invocation;

This is very similar from the block-based API, except we’re now passing NSInvocations instead of blocks.

NSMethodSignature * signature =
    [batman methodSignatureForSelector:@selector(identity)];
NSInvocation * getIdentity =
    [NSInvocation invocationWithMethodSignature:signature];
getIdentity.selector = @selector(identity);

id identities = [heroes map:getIdentity];
// -> @[@"Bruce Wayne", @"Selina Kyle"];

Not that bad. Things get a little more complex when using methods with arguments:

NSMethodSignature * signature =
    [batman methodSignatureForSelector:@selector(identityHasPrefix:)];
NSInvocation * isBruce =
    [NSInvocation invocationWithMethodSignature:signature];
isBruce.selector = @selector(identityHasPrefix:);
NSString * searchedName = @"Bruce";
[isBruce setArgument:&searchedName atIndex:2];

id batmen = [heroes filteredCollectionWithTest:isBruce];
// -> @[batman]

There again, I had to cheat a little, and add a identityHasPrefix: method to batman’s class.

It’s a bit less flexible than blocks, of course, but it’s actually a good thing: identityHasPrefix: can be unit-tested, while the (anonymous) block cannot.

One last thing about invocations: with the blocks, I was able to specify the return type of the blocks directly in the method declarations. Map blocks return id, while Filter blocks return a BOOL. As far as the API is concerned, NSInvocations are just objects, there’s no way to indicate what their result type should be.

Of course, NSInvocations are almost never created by hand, because that’s the job of the runtime. This leads to…

Style #4 - Higher-Order Messaging

NSObject (and NSProxy) have a method called forwardInvocation:. It’s called when the receiver doesn’t know what to do with a message.

We’re going to use it to redirect a single invocation to each of the collection’s items.

To do this, we’ll use⁴ a trampoline object that will receive an arbitrary message, and forward it to the items of the collection, using the NSInvocation API described above.

id identities = [[heroes map] identity];
// -> @[@"Bruce Wayne", @"Selina Kyle"];

Easy, right? map returns a proxy that receives the identity message, forwards it to each object of the collection, and returns the results in a new collection.

There’s a small catch: for the compiler, identity returns an NSString*, but our trampoline actually returns a collection (NSArray, NSSet or NSOrderedSet). We can workaround it by simply casting to id, but this is going to be a little more complex for filter.

The problem is that identityHasPrefix: returns a BOOL:

id batmen = [[heroes filter] identityHasPrefix:@"Bruce"];
// implicit conversion of 'BOOL' to 'id' is disallowed with ARC

This isn’t going to work. Alternatively, we can pass a return pointer as a parameter:

id result;
[[heroes filteredCollectionInto:&result] identityHasPrefix:@"Bruce"];

But in fact, the best solution is to simply mutate the collection itself:

heroes = [heroes mutableCopy];
[[heroes filter] identityHasPrefix:@"Bruce"];
// -> heroes is now @[batman]

Beautiful.

So what

I’ll confess, the whole goal of this post was to write a Higher-Order Messaging API for the Foundation collections. While blocks are clearly more powerful, message-passing and invocations feel more natural to me, more native to the language. I usually avoid language-changing hacks, but I may make an exception here.

There’s a small repo on github with the various implementations for NSArray, NSSet and NSOrderedSet.

Feel free to fork and hack. Comments are welcome, either on github or via Twitter

Bonus: How to Implement a Method Once For Several Classes

As I said at the beginning of this post, I want a consistent API for all the Collection classes. To achieve this, I declared the various methods in several protocols:

@protocol BlockCollecting
- (instancetype) block_map:(id(^)(id obj))block;
- (instancetype) block_filteredCollectionWithTest:(BOOL(^)(id obj))test;
- (id) block_oneObjectPassingTest:(BOOL(^)(id obj))block; // If the collection is ordered, returns the first matching object
@end
[...]

I later state that these protocols are adopted by all the Collection classes:

@interface NSArray      (Collecting) <BlockCollecting, KVCCollecting, InvocationCollecting, HOMCollecting> @end
@interface NSSet        (Collecting) <BlockCollecting, KVCCollecting, InvocationCollecting, HOMCollecting> @end
@interface NSOrderedSet (Collecting) <BlockCollecting, KVCCollecting, InvocationCollecting, HOMCollecting> @end

However, the actual implementation doesn’t follow that pattern. Instead, the methods are implemented on NSObject, which is tweaked just enough to make the compiler believe it’s working.

@implementation NSObject (BlockCollecting)
- (instancetype) block_map:(id(^)(id obj))block
{
    id values = [self col_emptyMutableContainer];
    for (id object in self) {
        id value = block(object);
        if (value) {
            [values addObject:value];
        }
    }
    return [values col_immutableCopy];
}
[...]
@end

This way, each method has to be implemented only once, for three unrelated classes.⁵

I fail to find real-world code patterns that would benefit from reduce. If you ever used it beside the obvious “sum of numbers” and “string concatenation” samples, please do tell me. I’ll update the post. ↩
My sample data may not be accurate. ↩
I already mentioned KVC in previous posts. ↩
Once again. ↩
Yes, I prefix methods of categories of foundation classes. ↩

Number Ranges in Objective-C

2013-08-28T00:00:00+02:00

Yet another little objective-C hack, for when you just want to count:

for (NSNumber * num in [@2 to:@4]) {
    // do something with num
}

Here, [@2 to:@4] is actually the same thing as @[@2, @3, @4].

Probably of little use, but very easy to write: all it takes is a method on NSNumber and a very small NSArray subclass. Here’s the one public method:

@interface NSNumber (ObjcRange)
// returns an array of the n + 1 NSIntegers between the receiver and `end_`, inclusive.
- (NSArray*) to:(NSNumber*)end_;
@end

And all it does is creating an small subclass of NSArray.

@implementation NSNumber (ObjcRange)
- (NSArray*) to:(NSNumber*)end_
{
    return [[ObjcRange alloc] initWithNumbersFrom:[self integerValue] to:[end_ integerValue]];
}
@end

The trick is that NSArray doesn’t care where its objects are stored. They can actually be organized in a C array, a linked list, a tree, etc. The actual implementation even swaps its internal storage sometimes.

In fact, all that’s required to subclass NSArray is to override -count and -objectAtIndex:, both of which can trivially be implemented just by knowing the start and end of our range:

@implementation ObjcRange
{
    NSInteger _start,  _end;
}

- (id)initWithNumbersFrom:(NSInteger)start_ to:(NSInteger)end_
{
    self = [super init];
    if(start_ < end_) {
        _start = start_;
        _end = end_;
    } else {
        _start = end_;
        _end = start_;
    }
    return self;
}

- (NSUInteger)count
{
    return _end - _start + 1;
}

- (id)objectAtIndex:(NSUInteger)index
{
    return @(_start + (NSInteger)index);
}

@end

Here’s a github repo with the source. There’s probably much more to be done with it, feel free to hack away and let me know.

Fan Deck View Controller

2013-07-22T00:00:00+02:00

A few months ago, I made this cool looking View Controller animation for my app Bicyclette.

I was tired of the “Horizontal Flip” animation, and I wanted to clarify the distinction between the main (foreground) and accessory (background)¹ views.

Here is an open-source, cleaned-up version of this Fan Deck View Controller. It is a Container View Controller, which makes it easy to use:

Step 1: Configure the Front and Back View Controllers:

fanDeckVC.frontViewController = [[UIViewController alloc] initWithNibName:@"Main" bundle:nil];
fanDeckVC.backViewController = [[UIViewController alloc] initWithNibName:@"Settings" bundle:nil];

Step 2: Bind a control to switchVisibleViewController, or use one of the completion-block variants.

Step 3: Change the rotation center to whatever you like:

fanDeckVC.rotationCenter = infoButton.center;

Here it is again, with the rotation center in the bottom right corner:

Please take a look at the (minimal) sample project, feel free hack around. Comments and patches are welcome, either on github or via Twitter.

e.g. the “Prefs”, “Info”, “About”, etc. view. ↩

An alternative syntax for KVC

2013-05-28T00:00:00+02:00

Here’s a little experiment I made last week. It’s an alternative syntax for Key-Value Coding, which is, as you may know, one of my favorite features of Objective-C.

My quick hack lets you write stuff like this:

// labels is an array of UILabels
labels.kvc[@"font"] = aNiceFont;

instead of the usual syntax:

[labels setValue:aNiceFont forKey:@"font"];

The idea is to use a Trampoline Object and make use of Keyed Subscripting.

(Keyed Subscripting is the feature that enables C-array-style access to the members of NSArrays and NSDictionaries.)

We’ll first add a method to NSObject to creates and return a trampoline:

@interface KVCTrampoline ()
@property id object;
@end

@implementation NSObject (KVCTrampoline)
- (KVCTrampoline*) kvc
{
    KVCTrampoline * kvc = [KVCTrampoline new];
    kvc.object = self;
    return kvc;
}
@end

The trampoline object implements the methods for Keyed Subscripting, and simply translates them to Key-Value Coding messages to the original receiver:

@implementation KVCTrampoline
- (id)objectForKeyedSubscript:(id)key
{
    return [self.object valueForKeyPath:key];
}
- (void)setObject:(id)obj forKeyedSubscript:(id <NSCopying>)key
{
    [self.object setValue:obj forKeyPath:(NSString*)key];
}
@end

So simple and obvious.

Wait! Why don’t we directly implement Keyed Subscripting on NSObject?

Actually, I have been thinking of mixing KVC and Keyed Subscripting since the latter was unveiled. We could then write:

labels[@"font"] = aNiceFont;

Well, that’s completely feasible but horribly confusing: how do you know that labels is an NSArray, not an NSDictionary? You can’t tell objects from collections anymore, and you probably don’t want that. The whole point of the “kvc” trampoline is to make our intentions clear.

Hopefully.

The code is available as a small repo on gitlab. If you have comments, you can find me on Twitter.

Responder Chain Debugging

2013-04-23T00:00:00+02:00

I recently had to debug a complex Responder Chain-related bug:

Did you know that when you send a nil-targeted action message, but there’s no first responder, the chain is started from the sender of the message? It seems to make sense, but I can’t find mention of it in the documentation.¹

Anyway, I ended up writing a few utility methods to help me track this bug. Here they are:

@interface UIView (NBResponderChainUtilities)
- (UIView*) nb_firstResponder; // Recurse into subviews to find one that responds YES to -isFirstResponder
@end
 
@interface UIApplication (NBResponderChainUtilities)
- (UIView*) nb_firstResponder; // in the -keyWindow
@end
 
@interface UIResponder (NBResponderChainUtilities)
- (NSArray*) nb_responderChain; // List the -nextResponder starting at the receiver
@end

UIView * NBFirstResponder(void); // in the app key window
NSArray * NBResponderChain(void);  // Starting at the first responder

Armed with this, you can simply po the responder chain in the debugger:

(lldb) po NBResponderChain()
$13 = 0x213dda70 <__NSArrayI 0x213dda70>(
<UIActionSheet: 0x11915160; frame = (0 231; 320 337); opaque = NO; ...
<UIView: 0x222c73b0; frame = (0 0; 320 568); opaque = NO; layer ...
<UIView: 0x2229d340; frame = (0 0; 320 568); opaque = NO; layer ...
<UIView: 0x11913dc0; frame = (0 0; 320 568); clipsToBounds = YES; ...
<_UIAlertOverlayWindow: 0x2226d4e0; frame = (0 0; 320 568); layer ...
<UIApplication: 0xaa66b30>,
<BicycletteApplicationDelegate: 0xab78c60>
)

It is available as a gist.

Enjoy and share; comments are welcome, via nico@bou.io or on twitter.

If I lost you at “nil-targeted”, go read the docs. The Responder Chain is one of the most powerful features of Cocoa(Touch), but it’s largely being ignored on iOS. ↩

IBOutlets in 2013

2013-04-20T00:00:00+02:00

TL;DR: Don’t declare IBOutlets using @property (weak) anymore.
Just use private ivars: ARC will take care of the rest.

IBOutlets are, as the name implies, what links your Objective-C code to your nib files (“IB” stands for Interface Builder¹).

It’s not a language keyword (IBOutlet is actually defined as “nothing” for the compiler) Interface Builder uses them to find the connections that can be made from your UI objects. Then at runtime, the Nib-Loading mechanism will assign the deserialized view to the corresponding outlet in your code.

Pretty simple actually. However, this ancient feature has evolved over time, mostly on (1.) how they are declared in code and (2.) their memory management.

A long time ago, in the old-style Cocoa world, outlets were simply declared as instance variables:

@interface MyController : NSObject {
    IBOutlet id outlet;
}
 @end

@implementation MyController

- (void) dealloc
{
    [outlet release];
    outlet = nil;
    [super dealloc];
}

@end

Nothing fancy here: the Nib-Loading simply alloc-initialized new objects and assigned them to the outlets, without any additional retain or release. Of course, they had to be released in dealloc.

Objective-C 2.0 introduced named properties, which could be used to declare IBOutlets using @property instead of accessing ivars directly:

@interface MyController : NSObject
@property (nonatomic,retain) IBOutlet NSView * anOutlet;
@end

@implementation MyController
@synthesize anOutlet=_anOutlet;

- (void) dealloc
{
    [anOutlet release];
    anOutlet = nil;
    [super dealloc];
}

@end

We had a slightly more precise semantic of the memory management, at the cost of some boilerplate in the @property declaration and its @synthesize. Of course, we could get rid of the nonatomic qualifier: we don’t need atomic properties, but performance is rarely an issue when loading a nib file.

With the introduction of iOS, everything changed: Views could be unloaded without their View Controller being deleted. Enter viewDidUnload:

@interface MyController : UIViewController
@end

@interface MyController()
@property (retain) IBOutlet UIView * anOutlet;
@end

@implementation MyController
@synthesize anOutlet=_anOutlet;

- (void) viewDidUnload
{
    self.anOutlet = nil;
    [super viewDidUnload];
}

- (void) dealloc
{
    [anOutlet release];
    anOutlet = nil;
    [super dealloc];
}

@end

For a single outlet, we had to write five statements, and an additional method just for IBOutlet management. Now that’s boilerplate!

(At least, once auto-nil weak properties became available, we were able to ignore a lot of it, at least for outlets connected to subviews.)

Thankfully, ARC came and allowed us to completely remove the dealloc method. In the meantime, properties were upgraded with auto-synthesize, and became strong by default:

@interface MyController : UIViewController
@end

@interface MyController()
@property IBOutlet UIView * anOutlet;
@end

@implementation MyController

- (void) viewDidUnload
{
    self.anOutlet = nil;
    [super viewDidUnload];
}

@end

The final blow came with iOS 6, which deprecated viewDidUnload. Since ARC makes ivars strong by default, we can now simply write this:

@interface MyController : UIViewController
@end

@implementation MyController {
    IBOutlet UIView * _anOutlet;
}

...

@end

Let’s discuss it a little bit:

With ARC, and because viewDidUnload is never called, there’s little point in using weak outlets, even if it is still what the documentation recommends². A strong ivar will retain it once more, and release it in the dealloc, but the lifespan of the object will just be the same.
An outlet is (almost always) an implementation detail of its ViewController or custom View. It’s not an API, and should not appear in the .h. Private ivars are the place for this.
Finally, @properties should be reserved for actual state properties, like the model object being presented, the display mode, etc. An outlet is certainly not a “property” with this definition, so again, private ivars make sense.

To sum up, we can now declare IBOutlets as ivars and forget about it. It’s actually the shortest and the most logical way to do it.

Comments are welcome, via nico@bou.io or on twitter.

##

This post is adapted from a Small Talk (in french) I gave at Cocoaheads Paris in april 2013.

I still refer to the UI Editor in Xcode 4 as “Interface Builder”. ↩
The doc reads: “Outlets should generally be weak, except for those to top-level objects.”. I respectfully disagree. ↩

Filtering NSArray With Key-Value Coding

2013-03-22T00:00:00+01:00

Here’s an NSArray Category I’ve used in every single Cocoa project in the last seven years.¹ What it does is quite simple, and you can probably guess just by the method names: it lets you filter NSArrays with Key-Value Coding.

@interface NSArray (KeyValueFiltering)
- (id) firstObjectWithValue:(id)value forKeyPath:(NSString*)keypath;
- (NSArray*) filteredArrayWithValue:(id)value forKeyPath:(NSString*)keypath;
@end

It addresse the very, very common task of searching items in list, for example:

Given a list of train trips, show me the ones that start in Paris.

In a list of RSS feeds, only display the ones with the label “News”.

Of course, Objective-C developers already have several tools² for this task:

regular loops,
- C-style (for(int i=0;i<count;i++))
- or with fast enumeration (for(id obj in array)),
NSEnumerators,
filtering with NSPredicates, (with -filteredArrayUsingPredicate:)
filtering with blocks. (with -enumerateObjectsUsingBlock:)

NSPredicate and block-based filtering are very powerful, but for simple cases, they just make the code hard to read. You may also have noticed that I have a thing for Key-Value Coding.

In a nutshell, filtering NSArrays with Key-Value Coding lets you write this:

NSArray * employees = [...]
Person * ceo = [employees firstObjectWithValue:@"timcook@apple.com" forKeyPath:@"email"];

Of course, you can use keypaths:

Person * steve = [employees firstObjectWithValue:[NSNull null] forKeyPath:@"car.licenseplates"]

(³)

And you can filter with KVC Collection Operators:

NSArray * boxes = [...]
NSArray * emptyBoxes = [boxes filteredArrayWithValue:@0 forKey:@"items.@count"];

That’s it! Easier to read than NSPredicates, shorter than block-based filtering.

You’ll probably have noticed that it only works for simple cases: it can’t execute arbitrary code on the items, and it can only return the objects equal to the passed value. As I said, I use it for the simple cases, which are actually the most common.

That’s if for today! Thanks for reading. You can checkout the code on Github, and criticize it over here on Twitter.

The very first version dates from before ARC, before blocks, before Objective-C 2.0, and before fast enumeration. NSEnumerators FTW! ↩
Just yesterday, Jim Dalrymple made a great blog post on the Big Nerd Ranch blog about all these options. ↩
http://osxdaily.com/2010/03/28/steve-jobs-car-has-no-license-plates/ ↩

How To Write a Self-Compiling Objective‑C File

2013-03-14T00:00:00+01:00

I really, really suck at shell scripting. I don’t do it quite often, hopefully, but I’d like to write scripts in a more modern language. Of course, in the real world, I’m going to use Python or Ruby or anything that’s less insane than shell.

However, the language I’m most comfortable with is Objective-C. Here’s a hack (don’t do this at home) to make your .m files directly runnable. ¹

The idea is to play with shell and C comments so that the file is both a valid shell script and an Objective-C program. The shell script calls clang to compile itself, and then runs the output executable.

/*/../bin/ls > /dev/null
COMPILED=${0%.*}
clang $0 -o $COMPILED -framework Foundation;
$COMPILED; rm $COMPILED; exit;
*/
 
 
#import <Foundation/Foundation.h>
  
int main(int argc, char *argv[])
{
    @ autoreleasepool
    {
        NSLog(@"Hello %@",NSProcessInfo.processInfo.environment[@"USER"]);
    }
    return 0;
}

The first 5 lines are the shell script, inside of /* */ comments. The first line is essentially a noop: we only need a valid shell command that starts with “/*”. The script proceeds to compile itself ($0) into a file with the same name, but without the extension, runs it, and deletes it. Finally, it has to exit before the end of the C comment. Voilà!

Obviously, this as serious drawbacks:

it’s a clever hack, and you shouldn’t use clever hacks. Noone else will understand what you’re doing; don’t be “clever”.
Objective-C, Foundation, Cocoa and the various frameworks are not meant to be used like that : it’s going to break anytime the compiler introduces some change, or when an API becomes deprecated… Anything could happen.

As I said, in the real world, I’ll use a proper scripting language. However, with LLVM, it should be possible to build an interpreter/JIT compiler for Objective-C. It might be fun.

Here’s a gist of it. If you’re really curious, I used a more complex version of it a while ago for a program that compiled and ran itself several times, in various configurations.

As usual, comments are welcome, on twitter and by email.

I certainly didn’t come up with this idea. I can’t seem to find a source right now, but this hack has been around for decades. ↩

The Design of Spinning Indicators

2013-03-04T00:00:00+01:00

This week’s post is about a UI widget you’ve already seen dozens of time today:

Please wait, I'm loading stuff.

Everyone who’s used a computer in the last 20 years knows what this means. This is a “spinnnig loading indicator”, technically known as “Indeterminate Progress Indicator”. That’s opposed to a “Determinate Progress Indicator” (or a “Progress Bar”) : this UI item informs you there’s something going on and you need to wait, but there’s no telling how long it’s going to take.

Back in the 1980s and 1990s, we used to have some other ways to indicate “loading times” :

The original Apple wirstwatch

Windows 95's hypnotizing hourglass

But I digress. These days, every website, app or device uses a variant of the spinning progress indicator. At the top of this post is Apple’s version, introduced in Mac OS X a decade ago, and used in iOS and on the web. Let’s take a look at it again:

Still loading

Here’s one from Google Images¹:

Google Images, loading.

They are visually very similar: they turn clockwise and, on a white/clear background, draw in black, which fades out to white again. This is important : Clockwise, and draw in black.

Now this one is from Kiwi, a new App.net client for Mac OS X with a slick UI:

Kiwi.app's animated gif

Can you see the difference? It’s turning clockwise, like the others. However, it’s not drawing black on white; it’s erasing black dots in gray, and the dots fade to black again.

Maybe it’s just me, but I think it looks wrong. It’s not really showing progress: I call these “Regress Indicators”.

Here’s what I think is the “correct” version of the Kiwi progress indicator:

Kiwi, actually showing progress

Looks better to me.

This is really one of these things you can’t stop noticing everywhere once you’ve started, just like bad kerning of misaligned pixels. I’m surprised to see this so often: I’ve spotted it in many other apps, including apps with an (otherwise) gorgeous user interface. This is case of Tweetbot for iPad, for example.

Well, I like my progress indicators to actually go forward, not backward. Your app design is amazing. Just get your progress indicators right.

Please :)

Comments are welcome, on twitter (twitter.com/_nb) or by email.

It’s actually the Google Images “loading” indicator, it might be pretty hard to find it using Google Images. ↩

Native SDK Buttons on iOS

2013-02-15T00:00:00+01:00

Even with the current trend of gesture-based interfaces, every app needs a button here and there. The UIKit class for buttons is called UIButton and it’s easy to customize with a background or an icon. In fact, nearly all iOS apps use customized buttons instead of using the native look.

It’s not a surprise, given that the native style of UIButton, isn’t very appealing:

This is UIButtonTypeRoundedRect.
It probably doesn't look good with your design.

This post is not about UIButtons at all

Here’s UIToolbar. You probably already met.

Hi! My name is UIToolbar. Look at my **curves**.

Let’s build a subclass of UIToolbar with a transparent background. That’s easy with UIAppearance¹:

@interface TransparentToolbar
@end

@implementation TransparentToolbar

+ (void) initialize
{
    [[self appearance] setBackgroundImage:[UIImage new] forToolbarPosition:UIToolbarPositionAny barMetrics:UIBarMetricsDefault];
    [[self appearance] setShadowImage:[UIImage new] forToolbarPosition:UIToolbarPositionAny];
}

@end

That’s all.

Now instantiate one TransparentToolbar, from a nib file or from code:

Tadaaa! Here’s a good-looking, 100% SDK, iOS button.

Let’s add some color²

Individual buttons in UIToolbars are UIBarButtonItems, and you can tune their color with tintColor.

Here is it in action: those two buttons were entirely designed in Interface Builder, with zero drawing code, and no image resources.

Look Ma! No Photoshop!

Great background texture, right? Seriously, the buttons look just right. Apple made them.

Well, there you have it. Basically for free, great-looking, native buttons for iOS, with adjustable tint.

But that’s a dirty hack!

That was @ndfred’s reaction to this idea. Indeed there are a few limitations to this method, but frankly, it’s not as bad as it seems.

Don’t use UIToolbar for UIButtons! That’s heavy/gross/wrong!

The biggest issue is that you end up with UIToolbars everywhere in your nibs instead of UIButtons. It’s not a problem per se, but since the toolbars are not transparent in Interface Builder, it adds some noise.

On the other hand, it prevents you from misaligning your buttons: it’s possible to misalign UIButtons, much harder with UIToolbar items.

It’s a hack!

It’s only using official APIs. As seen above, it’s exactly two lines of UIAppearance.

The Programmer Dilemma:

Do you want a two-liner hack that works, or a 200-lines “philosophically correct” solution ? Which one is easier to maintain?

OK, it’s not really using the API as Apple thought of it³, but I’m pretty sure it wouldn’t lead to a rejection.⁴ All the buttons in my app Bicyclette are actually made this way, and it’s been on the store for 6 months.

You can’t customize UIToolbars Buttom Items!

Well, that’s the point actually. Apple made those toolbar buttons with a decent look and a correct size.

Using standards is good in general. Only create something else if the standard doesn’t work for you.

In this case, you know UIToolbar buttons work for your users: they 1) look good and 2) don’t require your users to learn.

Couldn’t you use the underlying images in a UIButton instead?

UIBarButtonItem is not a subclass of UIView. Of course, there’s a real UIView somewhere in the hierarchy, and a “drawRect” method responsible for drawing the gradients and the corners, but it’s really private stuff, internal to UIToolbar.

BONUS: UINavigationBar

Here’s another hack that can be achieved with a transparent toolbar.

UINavigationBar, is UIToolbar’s cousin. It works very similarly, and uses two UIBarButtonItems, one on each side of the title. It’s theorically impossible to add more items.

The trick is that UIBarButtonItem can use a custom UIView as its contents, and we can just fit a transparent toolbar in there!

Take a look at this view hierarchy in Interface Builder:

Voilà! A clean-looking duo of buttons inside our UINavigationBar. Even with randomly-chosen colors, it doesn’t look completely ugly.

Two BarButtonItems walk into a Bar.
Into a view.
Into an BarButtonItem.
Into a NavigationItem.
Into a NavBar.

(Of course, on iPhone, there’s often no room for a second button. On iPad however, there’s plenty of space in the Navigation Bar.)

Conclusion

That’s it for today! Now you have exactly zero excuse to use Rounded Rect UIButtons, even if you have no designer in the team.

I added the test code for this post as a small github repo. Comments and suggestions are welcome, on Twitter or by email.

Edit, 2013-02-17: Added some thoughts on why it’s not such a bad idea as it seems.

Before UIAppearance, we would have made the backgroundColor clear, and implemented an empty drawRect: method. It’s easier now. ↩
A word of warning: I’m a developer. I may correspond to some definitions of a “designer”, but if I can stay away from Photoshop, it’s better for both of us. ↩
Did you ever write [UIImage new]? ↩
Of course, you never know. This is the App Store Review Process. ↩

Key-Value Coding: Custom Operators

2013-02-08T00:00:00+01:00

Key-Value Coding is an Objective-C mechanism for introspection and (limited) Higher Order Messaging. If you don’t already know about it, you should read

the documentation,
NSHipster’s post about KVC Collection Operators,
and “Let’s build Key-Value Coding” by Mike Ash.

While KVC is already really powerful, its implementation hides some really nice (and undocumented) details.

The Basics — KVC 101

There are two basic methods: valueForKey: and setValue:forKey:.

[foo valueForKey:@"bar"]

Provided foo implements the -bar method, this is the same as:

[foo bar]

It will fallback to other methods like getBar and countOfBar/objectInBarAtIndex:, ultimately accessing the instance variable directly.¹

When used on NSArrays and NSSets², it calls valueForKey: on each item, and returns a new Collection with the results.

[@[steveJobs, billGates] valueForKey:@"coolness"]
    // -> @[@"A LOT.",@"Not that much."]

(According to Bill Gates. But I think Jobs never did this.)

The extended version of valueForKey: is valueForKeyPath:. It uses KVC Paths, sequence of keys separated by periods and evaluated from left to right.

[object valueForKeyPath:@"a.b.c"]

It’s the same as:

[[[object valueForKey:@"a"] valueForKey:@"b"] valueForKey:@"c"]

Of course, KVC Paths can also be used on collections:

[@[steveJobs, billGates] valueForKeyPath:@"car.brand"]
    // -> @[@"Mercedes", @"Porsche"]

Calling All Methods

It’s important to realize that KVC predates Objective-C 2.0. This means that it doesn’t work just with @properties : you can “get the value” of any method with no parameter.

[@"hello" uppercaseString]
    // -> @"HELLO"
[@"hello" valueForKey:@"uppercaseString"]
    // -> @"HELLO"

Again, this works with collections too:

[@[@"hello", @"goodbye"] valueForKey:@"uppercaseString"]
    // -> @[@"HELLO", @"GOODBYE"]

It’s really any method. For example, -[NSObject self] is a method like any other.

[@"hello" valueForKey:@"self"]
    // -> @"hello".

(This looks useless? See below.)

Collections Operators

Collection operators are special components of KVC paths, that start with an “@”. Officially, there are 10 of them:

The simple ones:

@"@max"
@"@min"
@"@avg"
@"@sum"

The slightly less trivial ones:

@"@distinctUnionOfObjects"
@"@unionOfObjects"

The ones for which I have to read the documentation everytime:

@"@unionOfArrays"
@"@distinctUnionOfArrays"
@"@distinctUnionOfSets"

And finally, the one that does not really belong here:

@"@count"

Let’s begin with this one. It just calls -[NSArray count].

[@[@"a",@"b"] count]
    // -> 2
[@[@"a",@"b"] valueForKey:@"@count"]
    // -> @(2)

It’s different from the others, because it really works on the Collection itself, and not its items. I don’t consider it to be a collection operator, really.

Additionally, it doesn’t need any special implementation:

In fact, any keypath component starting with an “@” will use a property of the collection object, instead of collecting properties of the collection’s items.

(Of course, this is undocumented behavior.³)

This means we can call any method of the Collection object:

[@[@"a",@"b"] valueForKey:@"@lastObject"]
    // -> @"b"

Let’s add a -reverse method to NSArray⁴:

@implementation NSArray (Reverse)
- (instancetype) reverse
{
    return [[self reverseObjectEnumerator] allObjects];
}
@end

Behold! The custom @reverse “operator”!

[array valueForKey:@"@reverse"]
    // -> @[@"b",@"a"]

(By the way, note that this operator returns a Collection.)

Again, this is undocumented behavior. Don’t use it.

Collection Operators (for real)

OK, let’s look a little deeper at the other guys. How do they work? Here is the example in the official documentation:

[object valueForKeyPath:
@"<keypathToCollection>.<@collectionOperator>.<keypathToProperty>"]

This is the same as calling:

[[object valueForKeyPath:@"<keyPathToCollection>"]
     valueForKeyPath:@"<@collectionOperator>.<keypathToProperty>"]

Step by step, that would be:

[[[object valueForKeyPath:@"<keyPathToCollection>"] 
  valueForKeyPath:@"<keypathToProperty>"]
 <performTheCollectionOperator>]
     // -> Fetch the objects for the right part of the keypath, 
    // then perform the operation

Let’s take a look for a moment at the first group: @max, @min, @avg and @sum. Apple calls them “Simple Collection Operators”.

[company valueForKeyPath:@"employees.@avg.salary"]

This is equivalent to:

[[company valueForKeyPath:@"employees.salary"] average]

(Except that the average method doesn’t exist. Just pretend it exists, or write it.)

Here’s what the implementation for @avg does:

Call [self valueForKeyPath:@"salary"]
The result must be a collection of NSNumbers.
Computes the average and return it as an NSNumber.

It’s important to note that the part to the right of @collectionOperator is a parameter for the operator’s implementation.

The second group, @unionOfObjects and @distinctUnionOfObjects, work on a collection of objects. The documentation refers to these as “Object Operators”.

The first one, @unionOfObjects, is basically useless.

NSArray* pencils = @[@{@"color": @"blue"},
               @{@"color": @"red"},
               @{@"color": @"blue"},
               @{@"color": @"green"}];

[pencils valueForKeyPath:@"@unionOfObjects.color"]
    // -> @[@"blue", @"red", @"blue", @"green"]

It’s exactly the same as calling [pencils valueForKey:@"color"]. I’m not sure why this operator even exists.

@distinctUnionOfObjects works the same, but gets rid of duplicates:

[pencils valueForKeyPath:@"@distinctUnionOfObjects.color"]
    // -> @[@"blue", @"red", @"green"]

Nice and easy.

The last group, @distinctUnionOfArrays, @unionOfArrays, @distinctUnionOfSets work on a Collection of Collections. (In the docs: “Array and Set Operators”)

@unionOfArrays does what it says:

NSArray* pencils = @[@{@"color": @"blue"},
                     @{@"color": @"red"},
                     @{@"color": @"blue"}];
NSArray* markers = @[@{@"color": @"purple"},
                     @{@"color": @"blue"},
                     @{@"color": @"green"}];
               
[@[pencils, markers] valueForKeyPath:@"@unionOfArrays.color"]
    // ->  @[@"blue", @"red", @"blue", @"purple", @"blue", @"green"]

and @distinctUnionOfArrays gets rid of duplicates⁵:

NSArray* pencils = @[@{@"color": @"blue"},
                     @{@"color": @"red"},
                     @{@"color": @"blue"}];
NSArray* markers = @[@{@"color": @"purple"},
                     @{@"color": @"blue"},
                     @{@"color": @"green"}];
               
[@[pencils, markers] valueForKeyPath:@"@distinctUnionOfArrays.color"]
    // ->  @[@"green", @"red", @"blue", @"purple"]

@distinctUnionOfSets works exactly like @distinctUnionOfArrays, and there’s, of course, no @unionOfSets, because, well, there can’t be duplicates in NSSets.

KVC + self = Love

As I said above, -[NSObject self] is a method like any other. You can of course use it in KVC Paths:

[numbers valueForKeyPath:@"@avg.self"]
    // same as [[numbers valueForKeyPath:@"self"] average]

This calls “self” on each item, creates a new array with the results, and computes the average.

This is the easiest way to get the average (or sum, maximum, minimum) of an NSArray of NSNumbers.

If you were in the mood, you could even use @self (Remember! Undocumented!):

[numbers valueForKeyPath:@"@avg.@self"]
    // same as [[numbers valueForKeyPath:@"@self"] average]

This calls “self” on the array and computes the average.

Hidden Treasure: Custom Collections Operators

(Hat tip goes to Nicolas Bachschmidt for this discovery.)

Of course, this too is completely undocumented.

The valueForKeyPath: method looks for the implementation of operators in method named like this:

- (id) _<operator>ForKeyPath:(NSString*)keyPath

NSArray implements _avgForKeyPath:, _sumForKeyPath, etc⁶.

You can create new methods on this template, and valueForKeyPath: will pick them up.

For example, let’s add a @standardDeviation operator:

- (NSNumber*) _standardDeviationForKeyPath:(NSString*)keyPath
{
    [...]
}

[...]

[@[@0, @5, @5, @5, @5] valueForKeyPath:@"@standardDeviation.self"]
    // -> @2

And the @differential operator:

- (NSArray*) _differentialForKeyPath:(NSString*)keyPath
{
    [...]
}

[...]

[@[@0, @1, @3, @0] valueForKeyPath:@"@differential.self"]
    // -> @[@1, @2, @(-3)]

The return type is up to the operator: it can return an simple object or a collection.

Let’s build a better @distinctUnionOfObjects

There are two annoying limitations with the implementation of @distinctUnionOfObjects and his friends. First, an important detail in the docs:

Important: This operator raises an exception if any of the leaf objects is nil.

Additionally, each of the items in the collection must be KVC-compliant for the key being collected.

Here’s the pencils example again:

id pencils = @[@{@"color": @"blue"},
               @{@"color": @"red"},
               @{@"color": @"green"},
               @"notacolor"];          // <- !
               
[pencils valueForKeyPath:@"@distinctUnionOfObjects.color"];
    // -> NSUnknownKeyException: "This class is not key value coding-compliant for the key color"

It crashes. Let’s fix this.

We want a new operator similar to @unionOfObjects, but with these additional properties:

it ignores incompatible items,
it ignores nil values,
it ignores NSNull values.

Let’s name it @distinctUnionOfPresentObjects. Here’s a quick implementation:

- (id) _distinctUnionOfPresentObjectsForKeyPath:(NSString*)keyPath
{
    NSMutableArray * values = [NSMutableArray new];
    for (id obj in self) {
        @try {
            id value = [obj valueForKeyPath:keyPath];
            if(value && value!=[NSNull null] &&
               ![values containsObject:value])
            {
                [values addObject:value];
            }
        }
        @catch (id) {}
    }
    return [NSArray arrayWithArray:values];
}

Then, in use:

id objects = @[@{@"color": @"blue"},
               @{@"color": @"red"},
               @{@"color": @"green"},
               @"notacolor"];

[objects valueForKeyPath:@"@distinctUnionOfPresentObjects.color"] // -> @[@"blue", @"red", @"green"].    

No crash!

Operators Galore

Let’s push it a little further. As said above, the part of the keypath to the right of an @-operator is a parameter to this operator’s implementation. We can use it for what we want.

For example, an image-resizing operator:

[images valueForKeyPath:@"@resizedTo.{128,128}"]

A regexp filter:

[useragents valueForKeyPath:@"@filterWithRegexp.[Mm]ozilla"]

Or a unit-conversion operator:

[arrayOfMiles valueForKeyPath:@"@convertFromTo.miles.KM"]

The possibilities are endless!

Wrap Up

KVC is already very powerful, and with Custom Operators, it would really be fantastic. Unfortunately, it’s all undocumented implementation details right now, and KVC is a little forgotten by Apple these days. As far as I can tell, it hasn’t changed at all since at least Mac OS X 10.4. I filed a bug requesting an API, I encourage you to duplicate it.

Comments are welcome, via nico@bou.io or on twitter.

##

Most of this post comes from a talk (in french) I gave at Cocoaheads Paris in december 2012.

In other words, there are virtually no private ivars in Objective-C. Here are the Apple docs for the implementation details. The default implementation of +accessInstanceVariablesDirectly returns YES (for compatibility reasons), and nobody ever overrides it. ↩
Using valueForKey: with an NSDictionary is basically the same as using objectForKey:. As for NSOrderedSet, as of now, it doesn’t support KVC operators at all. ↩
It’s been like for a long time, actually. I think I read about it on a cocoadev discussion a few years ago. ↩
By the way, many people are doing it wrong. ↩
For some reason, @distinctUnionOfArrays loses the ordering. The documentation does not mention it, but in my testing, its sibling @distinctUnionOfObjects did maintain relative ordering in the filtered keys. ↩
Actually, probably not NSArray : it’s a Class Cluster. Its concrete subclasses do. ↩

Bicyclette 2 Beta

2013-01-29T00:00:00+01:00

The best app for Vélib is growing bigger! Bicyclette 2 is almost ready, and it needs your help!

It now works with 50+ Bike-Sharing Systems in the world, including Paris, Lyon, Bordeaux, Strasbourg, Nice, Lille, Barcelona, Dublin, Brussels, Wien, Montréal, Toronto, Washington, Melbourne…

Also new in 2.0 : the favorites system has been totally reworked. Instead of “Radar Zones”, you simply star your favorite stations. Bicyclette automatically creates a geolocalized alert in the 200m zone around it. Easy.

And of course, a bunch of refinements, optimizations, and bugfixes.

Get it now !

UIView+NibLoading

2013-01-25T00:00:00+01:00

In iOS apps, we write Custom Views all the time. For example, CocoaControls has a nice collection of custom views.

However, it’s sometimes a tedious task: you typically have to implement drawRect: or build your view contents by hand, using -addSubview: repeatedly and setting UI properties in code. Not beautiful.

label = [[UILabel alloc] init];
        
label.font = [UIFont boldSystemFontOfSize:15.0];
label.textColor = [UIColor whiteColor];
label.shadowColor = [UIColor blackColor];
label.shadowOffset = CGSizeMake(0, 1);
label.baselineAdjustment = UIBaselineAdjustmentAlignCenters;
label.backgroundColor = [UIColor clearColor];
label.frame = CGRectMake(10,10,200,28);
label.autoresizingMask = UIViewAutoresizingFlexibleTopMargin | UIViewAutoresizingFlexibleBottomMargin | UIViewAutoresizingFlexibleLeftMargin | UIViewAutoresizingFlexibleRightMargin;
        
[self addSubview:label];

And that’s just a simple, single-line label.

Same thing exactly. That’s better, right?

Now, there’s a handy tool that can quickly layout views, it’s Interface Builder. Problem is, while dynamically loading views stored in a nib file is not difficult per se, it still requires a few lines of plumbing, and it’s something I’ve rarely seen, even by experienced developers.

What if you could just write your custom view class (say, MyCustomView) and design its contents subviews in MyCustomView.nib? Even better, what if it loaded the subviews automatically?

Enter UIView+NibLoading

UIView+NibLoading is a UIView category that does just that. There’s also a concrete UIView subclass called NibLoadedView. Basically, all you have to do is subclass it, create you xib file, you’re done. NibLoadedView, in its initializers, loads its subviews from the nib file.

If you need to subclass another class (say, UIControl), you can use the UIView+NibLoading methods directly, and simply call [self loadContentsFromNib]. If you want even more control for example if you want several design variants, with as many nib files, you can specify the nib name and use [self loadContentsFromNibNamed:].

How does it work?

There’s something subtle going on: our custom view isn’t actually loaded from the nib file, only its subviews are. The custom view is being created elsewhere, from code (via a variant of [[UIView alloc] initWithFrame:]) or loaded from another nib (for example, the nib file of a UIViewController.)

However, we want to position the subviews properly, so we need a container view. Here’s the thing:

In the nib file, there can only be one root object, and it’s only used as a container,
the subviews are reparented from the container to the actual custom view,
this container is discarded after loading,
for this reason, it should be a UIView, not a member of your custom subclass.

Basically, -loadContentsFromNib loads a container view, and reparents the suviews to the receiver.

Moreover, the actual custom view is the file’s owner of the nib, so that you can define outlets from the custom view to its subviews.

Finally, here’s a small bonus: if the custom view being created has been created without a frame (more exactly, its frame is CGRectZero), it is resized to the size of the container view (the root view of the nib file). Otherwise, the subviews are properly resized.

Quick Demo

Let’s say I’m writing a “ColorView”, that shows a small color swatch, and its RGB value:

Now all I have to do is write a small subclass of NibLoadedView:

@interface ColorView : NibLoadedView
@property (nonatomic) UIColor * color;
@end
	
@implementation ColorView
{
    IBOutlet UILabel * _codeLabel;
    IBOutlet UIView * _swatch;
}

- (void)setColor:(UIColor *)color_
{
    _color = color_;
    
    _swatch.backgroundColor = color_;

    CGFloat red, green, blue, alpha;
    [_color getRed:&red green:&green blue:&blue alpha:&alpha];
    _codeLabel.text = [NSString stringWithFormat:@"#%2X%2X%2X",(int)(red*0xFF), (int)(green*0xFF), (int)(blue*0xFF)];
}

@end

and design its contents subviews in ColorView.xib:

That’s all! I can now use instances of ColorView as usual, include them in other nib files, create them from code:

Wrap up

That’s all there is!

UIView+Nibloading is available on github, under a MIT licence.

Discussion

Comments are welcome, via nico@bou.io or on twitter.

Constant Strings in Objective-C

2013-01-18T00:00:00+01:00

When coming from another language, strings in Objective-C are surprising on one aspect : objects of the string class, NSString are immutable : their value cannot be changed.

Objective-C also has Constant Strings. As C does with string literals, Objective-C embeds, in the binary, objects of the NSString class. Of course, Constant Strings are Immutable.

Let’s look at some of the surprising details of these objects.

In all this post, whenever I display more info about an expression, I use a little macro thap prints :

the class of the result object,

its adress

and its retain count. ¹

This macro, as well as test code for this post, is available as a gist.

A closer look at NSString

So what does a Constant NSString look like ?

NSString * a = @"a";
a :
    __NSCFConstantString 0x10c2fe0d8      -1

Constant strings are of the special subclass __NSCFConstantString and have a retain count of “-1” which means “infinite retain count”. These objects are actually part of the executable code : they can never be deallocated.

The compiler is your friend, and gives you the same object when you specify the same string in code :

NSString * a_too = @"a";
a_too :
    __NSCFConstantString 0x10c2fe0d8      -1

In C parlance, the two pointers are equal. In other words :

assert(@"a"==@"a"); // always.

It’s not just about saving a few bytes of memory : when comparing two strings, the first step is to compare the pointers; thanks to this behaviour, it actually happens a lot.

Similarly, -copying a Constant string returns the original object :

[@"a" copy] :
    __NSCFConstantString 0x10c2fe0d8      -1 //same address

Let's see what happens when we make a mutable copy :

[@"a" mutableCopy] :
    __NSCFString         0x7fdf1ac10630   1

A real String object is created, with its proper address on the heap, and a nice, brand-new-object retain count of 1.

Now what if we copy this object :

[[@"a" mutableCopy] copy] :
    __NSCFConstantString 0x7fff768183d0   -1

NSString just returned a Constant String out of nowhere !

Is there something special with @"a"? What if we build the "a" string in a more subtle manner :

[NSString stringWithFormat:@"%s","a"] :
    __NSCFConstantString 0x7fff768183d0   -1
[[[@"path/a" lastPathComponent] mutableCopy] copy] :
    __NSCFConstantString 0x7fff768183d0   -1

We still get the Constant String object !

Let’s try some other :

[[@"b" mutableCopy] copy] :
    __NSCFConstantString 0x7fff7682b9f0   -1
[[@"$" mutableCopy] copy] :
    __NSCFConstantString 0x7fff7682b770   -1
[[@"/" mutableCopy] copy] :
    __NSCFConstantString 0x7fff76822310   -1
[[@"€" mutableCopy] copy] :
    __NSCFString         0x1001076b0      1
[[@"§" mutableCopy] copy] :
    __NSCFString         0x100106e30      1
[[@"ab" mutableCopy] copy] :
    __NSCFString         0x100103230      1

Well, apparently, single-ASCII-character NSStrings are very special :

When you build a “single-ASCII-character” NSString in your code, what you get is the constant from the framework.

Let’s dig a little dipper.

Yummy, memory

Here’s what the memory around 0x7fff768183d0 looks like :

0x7fff768183d0:383E8176FF7F0000 C807000000000000 E4F7688CFF7F0000 0100000000000000 
0x7fff768183f0:383E8176FF7F0000 C807000000000000 E6F7688CFF7F0000 0200000000000000 
0x7fff76818410:383E8176FF7F0000 C807000000000000 E9F7688CFF7F0000 0300000000000000 
0x7fff76818430:383E8176FF7F0000 C807000000000000 EDF7688CFF7F0000 1700000000000000 

What do we have here ? Fist thing to notice, it’s the same structure every 32 bytes. What we see is a bunch of Objective-C objects, each 32 byte in size. The first group of 8 bytes is a pointer (always the same), as well as the third group.

In Objective-C, the first ivar in an object is the isa pointer, it points to the Class of the object. ²

(lldb) po 0x7FFF76813E38
$0 = 140735181569592 __NSCFConstantString
(lldb) po [__NSCFConstantString class]
$5 = 0x00007fff76813e38 __NSCFConstantString

As expected, this is a pointer to our __NSCFConstantString class.

What about the other pointer in our object ?

(lldb) p (char*) 0x7FFF8C68F7E4
(char *) $9 = 0x00007fff8c68f7e4 "a"

Well, here’s our data : a C string. These are the symbols for Foundation’s constant strings.

OK, to sum up

As we’ve seen, NSString can do some tricky optimizations with constant strings. This is actually possible because NSString is immutable. All of this would be much harder if all string objects were mutable.

Since NSString is a Class Cluster, none of the objects we’ve seen is just an NSString or an NSMutableString. ³ Rather than having a big string class that can handle every use, we have a number of specific implementations, tailored for every corner case.

Of course, you should never use -retainCount. ↩
It’s a little-endian machine : pointers in memory are “backwards”. ↩
Actually, it’s very hard, if not pointless, to create an object of the NSString class. ↩

bou.io

2013-01-16T00:00:00+01:00

I like that name, don’t you?

bou.io

There’s a ton of subjects I want to write about, so hopefully this will not stay empty too long.

This blog is entirely static, (except for google analytics). It’s written in markdown, using Jekyll hosted on GitHub Pages.

bou.io

Syntax highlighting in pure CSS

Why?

Random notes and observations

Next steps

A Twitter Archive

Multitasking User Interface

iOS

macOS

The future is the web (as usual)

Une brève histoire de l’heure d’été

NSBundle.current

Type-Safe UserDefaults

The Problem

The Solution My Own Quick Hack

Why lazy?

NSObject+PerformIfResponds

ObjC Message Dispatch — A (largely incomplete) summary

objc_msgSend

Forwarding

Boxing

@implementation details

The good stuff

The not-that-good stuff

Revisiting POI Clustering, Deciding Against It

This is 💩

What are we trying to do exactly?

Different zoom levels

That’s better

Functional Programming in Objective-C

On Functional Programming:

On the ObjC runtime:

On Objective-C and language design:

Git Clones of opensource.apple.com

Cross-Platform Development

RunLoopRunUntil

XCTestExpectation

CTT

CTTRunLoopRunUntil

Run, RunLoop, Run!

What’s a Run Loop Anyway?

What about iOS and OS X

CFRunLoop demystified

A sneak peek at CFRunLoop.c

Espaces insécables fines

Extended Type Info in Objective-C

The class of the nil object

Import the Runtime!

opensource.apple.com

More of this please

Objective-C Minimalism

Uploading Screenshots with iTMSTransporter - a Tutorial

iTMSTransporter: the CLI

The .itmsp package

Scripting it

Going Further

Several Approaches to Higher-Order Functions with Objective-C Collections

Style #1 - Block-based enumeration

Style #2 - Key-value coding

Style #3 - NSInvocation-based enumeration

Style #4 - Higher-Order Messaging

So what

Bonus: How to Implement a Method Once For Several Classes

Number Ranges in Objective-C

Fan Deck View Controller

An alternative syntax for KVC

Responder Chain Debugging

IBOutlets in 2013

##

Filtering NSArray With Key-Value Coding

How To Write a Self-Compiling Objective‑C File

The Design of Spinning Indicators

Native SDK Buttons on iOS

This post is not about UIButtons at all

Let’s add some color2

But that’s a dirty hack!

Don’t use UIToolbar for UIButtons! That’s heavy/gross/wrong!

It’s a hack!

You can’t customize UIToolbars Buttom Items!

Couldn’t you use the underlying images in a UIButton instead?

Why `lazy`?

What about iOS and OS X

Let’s add some color²