JOE HEWITT

AirPlay TV

I've been reading a lot of speculation recently about what a future Apple TV might look like. Not nearly enough of these analyses have talked about AirPlay. It's clear to me that AirPlay would be so important to the Apple TV, you might as well call it AirPlay TV.

There's so much compelling video content and games available on the App Store, it's not really that big of a deal if you can't watch cable TV, DVDs, or play Xbox on the Apple TV. This trend is only accelerating. When Steve Jobs said "I finally cracked it", I'd bet he was thinking of how AirPlay and the App Store had eliminated the need to support old set-top boxes.

The new AirPlay Mirroring feature in iOS 5 doesn't get talked about much, but it's the key to making this work. Watch this demo of AirPlay Mirroring if you need to be convinced. AirPlay Mirroring with the current Apple TV is underwhelming because of the hassle of making it work. You need to find the TV remote, switch to the Apple TV input, find the Apple TV remote, wake up the Apple TV, then go back to your iOS device and turn on AirPlay. Ugh! Apple has to make AirPlay work effortlessly, and to do that, they need more than a set-top box. They need to control the whole TV.

The one big technical hurdle for Apple to overcome is the unreliable WiFi connection between your iOS device and the TV. There are a number of ways they could overcome this. They could put a WiFi router or extender in the TV to ensure a fast connection. They might try another high-bandwidth wireless networking standard that doesn't rely on flaky home WiFi routers. They might come up with a new split architecture in which apps run partially on the TV and partially on the iOS device. I don't know exactly how they'll solve this, but I am sure they will solve it.

Once the networking is reliable, the user experience can get really good. There would be no on/off switch on the TV. You'd just sit down on the couch, pick up your iPad (or iPhone or iPod touch), start using an app, and the TV would turn itself on and start displaying the AirPlay stream. No fumbling with remotes. No numeric keypads and up/down/left/right buttons. Just touch, swipe, and speak to find what you want to watch. You'll never plug a set-top box into this thing, and you'll never want to.

It's possible Apple could choose to just license AirPlay to TV manufacturers, but something tells me they won't be able to resist making a showcase TV that screams "Apple!" to everyone who walks into your living room. Once Apple announces this, iOS developers will be scrambling to support AirPlay. I'm sure Apple will give enough advance notice to ensure that a ton of amazing content is available on launch day. It will take a while for Apple to get all of the big TV/movie studios, sports leagues, and news networks on board, but they will eventually. Most them already have dipped their toe into the App Store waters (see HBO Go, MLB.tv, and CNN, for example).

If I were an iOS developer, I'd start investing in AirPlay right now.

Apps and Web URLs: Perfect Together

I've been thinking about ways to save the Web from being buried in the avalanche of native apps we've been seeing. As I recently wrote, hyperlinks are the defining characteristic of the Web, but most native platforms break the hypertext model. iOS and Android support URLs for navigating between apps, but the URLs they support are not portable to other platforms. For example, you can link to my Facebook profile on iOS using fb://profile/joehewitt, but this URL is useless on Android or in a desktop browser.

Many of the URLs that iOS and Android apps support have an equivalent URL for the Web. For instance, fb://profile/joehewitt translates to http://facebook.com/joehewitt on the Web. It would be ideal if the OS could automatically perform this translation for any third-party app, just as iOS redirects Google Maps and YouTube URLs to their native app equivalents.

We don't have to wait and hope that Apple or Google solve this problem for us. Here's a solution we could build now: a Web service that translates between Web URLs and native URLs using a community-maintained database. Domain owners could claim the mappings for their domain using an authentication process. There could be accompanying libraries that developers could link into their native apps which help to find the best way to display a URL. If you happen to have the Facebook app installed, and you tap a facebook.com URL, the service would redirect you to the native app. If it's not installed, it would open the URL in the default Web browser.

This service would make URLs more future friendly and really underscore their "universal" nature. It would allow hypertext to transition beautifully into this new world in which browsers are not the only way to consume Web content.

Fast animation with iOS WebKit

While building Scrollability I've learned a few tricks about how to make animations that start fast and run fast on iOS WebKit. The challenge of getting the scrolling physics right was nothing compared to the chore of making the animations as smooth and stutter-free as they are in native iOS apps. It took several iterations to get results that are acceptable.

First problem: cubic bezier curves

My motivation in starting Scrollability was that I was disappointed with iScroll, the most popular touch scrolling library out there. iScroll uses CSS transitions with the cubic-bezier timing function, and the results don't feel anything close to iOS native scrolling. Cubic bezier curves are simply unable to match the deceleration curve that Apple uses, so I knew right off the bat that CSS transitions could not be part of my solution.

Play with the iScroll demo and tell me if it doesn't immediately feel wrong to you, especially the way it bounces back at the edges.

Solution: animate with JavaScript, not CSS

My first iteration used JavaScript and setInterval to animate the -webkit-transform CSS property using translate3d. This achieved hardware acceleration, and it looked amazingly good on an iPad, but not nearly as good on an iPhone 4. While the physics were improved over iScroll, the animation was not nearly as smooth as the results iScroll got using CSS transitions.

Second problem: pixel doubling

Pixel doubling means that even though the iPhone 4 screen is 640 pixels wide, WebKit reports it as 320 pixels wide. The smallest increment you can animate something with JavaScript is two physical pixels, since WebKit always rounds coordinates up to the nearest integer before doubling them. That's right, writing translate3d(0, 0.5, 0) is the same as writing translate3d(0, 1, 0), and both will be multiplied to translate3d(0, 2, 0) in real coordinates. While you might be able to use the meta viewport tag to avoid pixel doubling, that causes a host of other headaches that I didn't want to burden Scrollability users with.

Play with this early Scrollability demo on an iPhone 4 (with iOS 4) and you will see how jerky the animations feels.

Update: Appears I was wrong about this. As this test by @amadeus shows, floating point values are not rounded as I thought. I'll have to do more research into why my first Scrollability animation was so poor on retina displays.

Solution: CSS keyframe animations

CSS keyframe animations, like CSS transitions, have the ability to animate in physical pixel increments on retina displays, but give you much more control over timing than cubic-bezier functions. Rather than simulating the physics progressively at each setInterval callback, I would simulate them all up front when the user released their finger, and use the resulting positions and times to create a fine-grained CSS keyframe animation like this:

@-webkit-keyframes scrollability {
    0% {
        -webkit-transform: translate3d(0, 10px, 0);
    }

    1% {
        -webkit-transform: translate3d(0, 11px, 0);
    }

    ... a bunch more keyframes ...

    99% {
        -webkit-transform: translate3d(0, 290px, 0);
    }

    100% {
        -webkit-transform: translate3d(0, 292px, 0);
    }
}

Turns out you can generate this CSS, insert it into a stylesheet, and start it playing faster than the wink of an eye. The results were just as good as I had hoped they would be.

Third problem: image animations stutter

While keyframe animations made my vertical scrolling demo fly, they caused a severe slowdown in my photo gallery demo. I knew the slowdown wasn't JavaScript-related, since my table demo was so fast. It had to be deep in the guts of WebKit. After much experimentation I noticed that galleries with only three photos did not stutter. Turns out the problem occurred only when the width of the animated element was greater than 1024 pixels.

Third solution: avoid repaints

WebKit does all of its painting on the CPU and then uploads the results to the GPU in the form of an OpenGL texture. This process is slow, but once you have that texture, it can be drawn to the screen incredibly fast. This is the key to native scrolling speed.

Turns out that on iPhones, WebKit creates textures that are no larger than 1024 by 1024, and if your element is larger than that, it has to create multiple textures. It really doesn't want to do that, due to limited GPU memory, so it creates these textures only when it needs to, and then throws them away immediately after. In our case, the textures are created at the start of a CSS animation and then discarded when it ends. Therefore, the stutter in my photo gallery is caused by those large images being painted on the CPU and pushed to new textures on the GPU at the start of the animation.

However, if your scrollable element only requires one texture, there is no stuttering, since that texture is probably already on the GPU when the animation begins. The solution was as simple as setting a max-width of 1024px on my photo gallery demo. Ok, it isn't that simple, since I have to do some other tricks to continually reposition the gallery images within that 1024px container, but the results are really excellent.

Fourth problem: iOS 4 WebKit sucks

Since this is the Web, where good things are always too good to be true, the 1024px texture trick works great on iOS 5, but has no benefit whatsoever on iOS 4.

Play with the photo gallery demo on iOS 5, but grab a box of tissues to dry your tears before you try it again on iOS 4. Kudos to Apple for optimizing iOS 5, but what do we do in the mean time?

Solution: wait for iOS 4 to die

No, seriously. If you want to use Scrollability, you need to have a fallback for legacy browsers anyway, and so I'm going to recommend that you limit Scrollability to iOS 5 WebKit. It's the future, right?

Dropbox is my publish button

I flipped the switch on my new blog last night. If you've been here before, you might not notice anything different. The design hasn't changed, but behind the scenes everything is new. I've written a CMS for the blog in Node.js and Express. It's hosted on EC2, S3, and Cloudfront. All the content is written in Markdown and pushed to my server using Dropbox. All the code is pulled from repositories on Github and NPM. I use responsive design to adapt nicely to all screen sizes, and use feature testing with has.js and hascan to adapt nicely to all browsers.

Like many nerd-minded bloggers, I prefer writing in Markdown in my programming text editor instead of writing in a browser. Many such systems like Jekyll are growing in popularity, but these usually require you to use Git to push your Markdown files to the server. Call me lazy, but I don't want to have to go down to the command-line. I just want to save the file to publish it. This is where Dropbox comes in.

All of my blog posts are now in Markdown files, and those files are synchronized to my Linux EC2 servers with Dropbox. My previous blog used Markdown also, but I had to FTP the files to my former Dreamhost server. I used ExpanDrive, which made the FTP process relatively painless, but FTP is still flaky, and ExpanDrive doesn't deal well with flaky connections. With Dropbox, the synchronization is effortless. A second or two after I save the Markdown file, the post is up on the blog, and I never have to think about it.

Setting up Dropbox

I run Dropbox on my EC2 server using the default Linux AMI, but this should work on most Linux distros.

Start by installing the Dropbox Linux command line client. At the end of the install script, it will ask you to link your Dropbox account by copying and pasting a URL into your browser. Do not do this yet. If you do, it will link your personal Dropbox account to the server, downloading the hundreds of megabytes of files you may have on Dropbox, and exposing potentially sensitive files to any intrepid hacker who breaks into your server.

You only want to synchronize the files needed by your CMS, so you need to create a new Dropbox account that is dedicated to this purpose. Of course, the Dropbox client only works with one account at a time, and your personal computer is going to be logged into your personal account. Luckily, Dropbox folder sharing solves this problem nicely. Once you have your new account, go to your personal account and share the folder containing your CMS files with the new account. Then you can link the new account to your server. Be sure you are logged into the new account in your browser, and then load that URL the installer gave you. Done.

About that CMS

Dropbox alone does not a blog make. My new CMS, called Nerve, processes those magically synchronized files and spits out a website. It's Apache-licensed and hosted on Github, but I'm not recommending that anyone other than me use it at this time.

Nerve takes advantage of Node.js's built-in file watching abilities to monitor changes made by Dropbox and then regenerate pages on the fly. It asks you for a path to your Markdown file, but this can include asterisk wildcards so you can use as many files as the pattern matches. My entire blog is two Markdown files. One contains all posts from 2002 to 2010 and the other contains all posts from this year. The files look something like this:

This is a new post
==================

Blah blah blah.

This is another post [9/30/2011]
================================

Blah blah blah.

Nerve uses Markdown's level-one header to demarcate the posts. It then looks for a bracketed timestamp in the header to distinguish posts that are public from ones that are still drafts. A password-protected drafts page allows me to preview drafts before publishing. Once they are done, I add the timestamp, save the file, and it's published.

A previous version of Nerve required creating a separate file for each post and giving it a name that contained the timestamp. I tested this system out on Laura's blog and found that the chore of creating new files was just enough friction to discourage her from starting new posts. One day I was talking to my friend, Dave Winer, and he told me that his entire blog is contained in a single OPML outline file. This was a big aha-moment for me, and I rewrote Nerve so that all posts could be within the same file. This was a big hit with Laura, and I've found her posting volume has gone up significantly ever since.

Caveats

The Dropbox Linux client works pretty well, but on a couple occasions I've found it has stopped syncing for no apparent reason. It's not the end of the world. The web server keeps running, but new posts won't get published until I SSH into my EC2 instance and restart the Dropbox daemon.

What the Web is and is not

My post the other day generated a lot of responses. I suspect that most of the positive responses were from people who, like me, have a depth of experience on other client platforms, and are able to judge the Web in comparison. It's hard not to be disappointed by HTML if you've developed for iOS, Windows, or other mature platforms as I have.

Then there are the people who reacted negatively to my post. A common theme throughout these responses is complacency. The Web is good enough and it's been getting much better lately, so we should be grateful, they say. The shortcomings are not so important as to risk that a centralization of ownership would lead to more IE6-style stagnation. The Web will catch up eventually.

This argument is futile, because nothing I yearned for in my post is possible. HTML and company are going to continue going through the same standards bodies, there will continue to be an array of subtly different implementations, and the Web "operating system" will continue to evolve at a far slower pace than centrally-controlled operating systems. That train left the station a long time ago.

Thinking about this in public helps me work out why I care so much about the Web. I'm beginning to see that some parts of the Web we take for granted are not what actually defines it. The Web is not HTML, CSS, and JavaScript. It's not DOM, SVG, WebGL, PNG, or Flash. The Web is really just HTTP over TCP/IP. What gets transported over HTTP does not define the Web.

There is, however, one other characteristic that does define the Web, and that is the humble hyperlink. Links are a feature of HTML, but they are not limited to HTML. Links are the connections that give the Web its name, and links are the biggest thing missing from native platforms. Some have pointed out to me that iOS and Android allow you to construct URLs that let users navigate between apps, but what they are navigating is not a network, but the tiny subset of the App Store they have installed on their devices. That is a far less powerful idea than the Web, where a single click is guaranteed (network willing) to take you to a self-contained application that begins running immediately.

So, my definition of the Web then is resources loaded over the Internet using HTTP and then displayed in a hyperlink-capable client. This definition is liberating. It helps me see a future beyond HTML which is still the Web. I can say now that when I exclaim my love for the Web, it's the freedom of driving the open Internet in a browser that I love, not the rendering technology. Hyperlink traversal matters. The Internet being global and decentralized matters. HTML does not matter.

The biggest thing HTML has going for it is that every computer in the world can render it. Not a small benefit, to be sure, but not a reason to limit ourselves to HTML forever. People can and will download new browsers with new rendering engines, given the proper motivation. It just so happens that we're currently in a major transition period, from desktops to mobile touch screen devices, where there is an incentive for people to try new browsers. I wonder, will someone capitalize on that?