In the first of a new regular series of reports, Noah looks at how virtual reality could use fictional ideas like Sherlock Holmes’ ‘mind palace’ as inspiration for new productivity applications, helping us all to function at a higher level.
Noah Nelson (@noahjnelson) is the head writer of Turnstyle News, a public media website that covers tech and entertainment, and a project of the Peabody Award winning Youth Radio.
If you’re a certain kind of geek you’re probably familiar with the BBC’s Sherlock series. That show is the reason why we know how to pronounce the name Benedict Cumberbatch. This modern update on Holmes uses an old technique to keep his prodigious brain on track: Sherlock has a “mind palace.”
This is a trick that dates back to Ancient Greece. Orators would store information on top of physical scenes they had imprinted in their minds for easy recall. This is one way that storytellers were able to go on at length without written notes. The “ars memoria” was taught alongside other elements of rhetoric at certain schools.
In time the art of memory would fade from use—its rise and fall are the subject of a 1966 scholarly work by Francis A. Yates—but the core idea offers up a tantalizing possibility. Such memory palaces could be the basis for a branch of virtual reality application development. One that doesn’t get as much attention as games and cinema: productivity apps.
The cult of productivity apps grew up right alongside the tech blogosphere, shaping the way that we think about work in the first Internet Age. Productivity apps are the key to the soul of every tech blogger. Whether it’s quoting from David Allen’s Getting Things Done chapter and verse or arguing the finer points of Yojimbo versus Evernote there has been unholy amount of hard drive space taken up by thoughts about them.
One of the keys to getting the mass market to look at VR as something other than a novelty could be to have some killer productivity apps show up. Search-driven computing has done wonders for bringing up exactly the file you were looking for but what about when you‘re not quite sure what you need?
The boards that screenwriters use to piece together plots are familiar low tech stabs at data visualization. These tools emphasize relationships of data. The ubiquity of search has deemphasized the power of the link, that little innovation that made the World Wide Web what it is. Anyone who has lost a night to the pleasures of surfing Wikipedia knows the power those links hold.
Which brings me back to Sherlock’s mind palace and a current era productivity app that could benefit from the VR treatment.
The app in question is The Brain. This is a rather deep mind-mapping tool that works as a kind of personal wiki with a 2D visual link interface. In all my time hunting around for mind mapping tools The Brain is definitely my favorite. It also requires a not-trivial licensing fee, which means I’ve only ever used the stripped down trial version.
At its core The Brain is a flattened, externalized version of Sherlock’s memory palace. It’s a visual interface that allows for navigation of thoughts. This basic data visualization makes it easier—for some of us, at any rate—to make connections between disparate ideas. These relationships not only reflect how the user thinks about a set of ideas, but suggests new ways to think about them.
VR will make it easier than ever to spatially organize your thoughts and memories like a mind palace. You can imagine the potential for much more complex perspectives on your own personal data this would provide. The third dimension alone would create a new dynamic, but that’s just scratching the surface. There are decades of data visualization research ready to be unleashed on an unsuspecting marketplace. Tools that will start to bring to bear what science has learned about the way visual perception and attention works.
Getting inside our data won’t necessarily be a panacea—there’s no universal cure for information overload—but I’m convinced that it will unlock all kinds of creativity. As developers create more intuitive tools for dealing with that data we’ll start to uncover new ways of thinking about the world around us. The world influences the tools, and the tools influence the world.
This doesn’t have to be the far-flung future, the march towards a new interface paradigm can start now. With the apps we already have, brought into VR. This is a fun game you can play with yourself: imagine your most trusted productivity apps in VR.
In my fever dreams I see my task manager—the venerable Omnifocus—remapped into a version of the board game Risk. Tasks piled up in different “countries” (projects) and given form by the type and context of the action. This wouldn’t be a daily tool, but a potent review perspective to help wrap my head around the scope of my commitments.
Evernote could get rescued from years of uninspired UIs and become an infinite storage closet. One that can rearrange itself with a simple voice command: from Notebooks to Tags to Timeline and back again.
The point is that there’s a whole universe of use cases here that are waiting to be unlocked. Researchers at places as diverse as DARPA and Bloomberg have begun to play with the idea, and you’ve probably caught the concept videos that Microsoft has released for “mixed reality” apps in HoloLens.
See Also: Microsoft HoloLens Reveal Concept Video
Yet there is so much room for developers to begin laying the foundation here. The next great user interface revolution is going to start with how VR and AR let us interact with our personal data—computerized versions of our own memory palaces, which might make us as smart as Sherlock.
(Okay, that last bit just isn’t going to happen. Sorry.)
