You can't discover anything if you can't see what you're doing.

The above presentation by Bret Victor is amazing, and I couldn’t stop tweeting about it the other day. My mind was blown at:

4:20. cool! I mean, I’ve seen live editors before, but I never thought of actually using sliders on numbers. Clever! I remember now though, that this has been done by Codea (and probably others) before.

5:25. What. That changes the way I think about programming visual things. I keep thinking of all the times I’ve tweaked an animation or gradient, recompiled, redeployed to device, and get to experience my change maybe a full minute later. How many solutions have I missed, had I just been able to drag a slider to try variations?

I’ve tried to come up with solutions to this problem several times before. My first try was RemoteParameter. However, it is incredibly clumsy to integrate into the code being modified: modified variable must be KVO compliant, it only works for a single spot in the code (so it will fail if you’re e g modifying the background color for all cells in a list), and the editor UI is atrociously bad.

My second try was with NuRemoter. This tool I still actually use, and is somewhat useful in working with living code (sending calls in the middle of animations, debugging time sensitive operations, etc). However, it is not at all visual, and any code I improvise in the NuRemoter UI will be in Nu, which I can’t use as-is in my project without rewriting it into ObjC. I have a few thoughts here, though: first off, I should make it easier to hook Nu code into our code base, so that things that are naturally expressed in a script-y langage, can be. Secondly, the NuRemote protocol can be used as a payload delivery system for other tools. For example, if I ever write an animation construction kit, I could use NuRemoter to upload and inject animation descriptions into the app, and immediately try them out. (I’ve thought of integrating it into my Localizer localization app, too).

Wait, animation construction kit? Well, Core Animation is incredibly powerful and easy to use, but expressing animations in code is both unintuitive and slow (see: write/compile/deploy/test cycle), and maybe tobi is a better animator than me? I want something like the GLSL Shader editor: define input parameters (perhaps start and end locations, duration, intensity) and a sample image as a placeholder for the UIView being animated, and then define animations Keynote-style, possibly with values substituted for parameters. Output a json file, load it at runtime and use it instead of all the animation code. Shouldn’t be THAT hard to build?

6:10. My jaw drops just a tiny bit further, but enough to mark the time. Well, typing autocompletions into the autopreviewing document so that they run automatically is pretty obvious, but I love the idea of showing you what the thing does, rather than having you read documentation. Maybe the right sidebar in Xcode should have movies instead of help snippets?

7:43 and 8:12. How the hell did he do that‽

11:30. Again, how did he do that‽ He’s obviously not re-evaluating the whole file on each change. Maybe he’s been careful when writing the file so that it can be re-evaluated in the context of the currently “running” web page/js-vm? It looks like an incredibly nice way to code games, though. A little bit like Unity I guess; I remember having my mind blown the first time I saw that editor, too.

13:00 and 14:10. Now he’s just showing off. Surely this editor must be specifically written for this game engine he has constructed, and not something generic. Still want that code+editor, though!

Around 16:00 he re-iterates again how easy it is to be creative when you can visualize your results live, and how it doesn’t just apply to visuals or animations, but entire gameplay ideas. I’m so very sold on the idea.

Around 20:00, I really love the idea of supplying sample input data and see how that affects the code you just wrote.

Around 26:00, I should’ve realized that this is the same guy as the one who made That Math Video that blew my mind pretty thoroughly last year. I didn’t, until I checked his web site in preparation for this blog entry. He has plenty of new interesting entries there, if you haven’t checked them out lately.

However, out of all of these, the moment that perhaps isn’t the most interesting to me as a programmer, but never the less tipped me over from “yes, I am impressed” into a squealing, giggling pile of fanboyism, is 31:30. Probably the most amazing animation editor I’ve seen, and probably the most intuitive power user UI I’ve seen, and it’s just an experiment he made? MIND BLOWN.

I love how, instead of trying to solve the problem of tiny hit targets with giant thumbs, he has the user select the layer with her left hand, while manipulating it with her right. It’s a touch surface, and the user has two hands: why is nobody else using this fact‽ (my mind was similarly blown in ‘05 by the same multi-hand usage by TactaPad, which sadly never became an actual product).

Compositing all the different animations the user wants to get into the scene by just scrolling back time and immediately performing the next animation is equally genius (also seen, but not at all as well implemented, in Garageband.). Just look at the amazing speed at which he composes that scene!

However, out of all the amazing things Bret is showing us, none are as inspiring as his outro. Ever since I started programming, I’ve felt an idea or principle just at the edge of my consciousness. I know it’s related to creating a user interaction paradigm that is more natural, but I can’t describe it more clearly than that. I just know, for example, that multitouch is a step in the right direction. I haven’t thought about it too much in the past two years as I’ve been busy at Spotify, and Bret’s note at the end that doing only one thing will not give you insights really hit home.

Alright, enough gushing. I hope that after all that, you did actually watch the video, and that you found as much inspiration as I did. Right now, I just want to stop everything I’m doing and take a few months off life and make tools. I should probably finish a few projects first though…