The iPad as interactive storytelling device (with the right software)

If I tap here, we get to see a video of the bush burning and play the slapping game

With the hype dying down, TechCrunch recently published a great article detailing what could be an actual potential use for an iPad (a device that otherwise just seems like a great tablet computer without an operating system). It’s true – a device like this could be a new frontier for interactive storytelling, if only because there is finally an interface that will make it as easy to read/watch/experience an interactive multimedia story as it is to page through the morning paper or a good novel.

It’s often clunky and downright annoying to use a mouse to navigate through a longer interactive creative work, be it a story, script, graphic novel, or just a DVD. It takes about 45 minutes for a user to get all the way through my thesis, a fact I was not proud of when I tried to explain to my parents how easy it would be for them to read and understand it. This gets even worse when any of the media involved in the interactive story should be viewed with a 10-foot experience, on a television or other larger screen; how often do you want to throw your remote at the TV when it doesn’t scroll to the correct button?

That’s where a huge, responsive touchscreen, seamlessly integrated with a television, could come in handy. Imagine being able to combine the 10-foot experience with a more intimate storytelling experience. Viewer/users could watch a television show on TV and have an iPad automatically present context-relevant links, in real time. Want to dig deeper into a side character’s momentary appearance on camera? Wait until the commercial, then tap a link on your iPad and watch a short video, read a graphic novel or a short story, or play a casual game on the web that gives you even more narrative. The interface allows for an easy way to add an entire dimension to a visual narrative, a level of depth that previously only existed when users moved from TV to computer, or watched/experienced the entire narrative on a computer screen.

It’d be foolish not to touch on how beneficial this could be advertisers, with context-relevant links able to be presented to viewer/users as they watch a product integrated into a TV or web show. Interactive narratives using a well-designed interface could take a step out of the process from ad watching to product purchasing.

There’s more good news: the perfect software already exists to make this happen, and easily: it’s called Sophie. Sophie finishes a natural progression for its creator, Bob Stein, who was one of the first people to look at a video playing on the same page as text (at the MIT Architecture Machine Lab, the same people who brought us black text on white backgrounds on computers…that’s not a joke), and see the potential for a new form of communication. His first crack at the seamless integration of video with text became the Criterion Collection, which at its genesis essentially served as a model for DVDs and Blu-Rays today. Early Criterion titles included on their LaserDiscs scholarly essays with specific references to the films, and being able to play those clips as the reader went through the argument was a massive innovation in the early ’90s.

That was a mere drop in the hat compared to what Sophie intends to do. Stein now runs the Institute for the Future of the Book (among other things), and their Sophie project aims to make it relatively easy to create what are most easily described as interactive books; works that utilize multiple media and interactivity alongside text in order to get their point across.

My hope is that someone is already hard at work on a Sophie Reader for the iPad (you can get started making one now, Sophie is open source). I want to see the day when I can read a story on my iPad, then tap a link to start a context-relevant video playing on my TV; while I’m watching that, I can chat with other fans in the online community, then seamlessly go back to reference the text when something amazing happens on the TV. And after the TV episode is over, it can point me to more online videos and texts that can further immerse me in the narrative.

And as TechCrunch points out, that experience is something to pay for.