Apple’s AR Experience Puts Products Ahead of Art

Carsten Höller's Through on the Today at Apple [AR]T Walk.


The trouble with [AR]T—an augmented reality initiative produced by Apple in collaboration with the New Museum—began when I tried to get tickets. Because it was framed as a free public art experience, I thought that the [AR]T Walk would be easily accessible, like a drop-in guided tour at a museum. But Apple’s home page offered no information about the collaboration, and the project was eight slides deep in the New Museum’s front-page carousel. By the time I found the registration page in the bowels of Apple’s website, all tour slots leaving from the Fifth Avenue Apple store in New York were booked a month out. No matter: press access! But it does matter. As of this writing, all New York tours are full—likewise the Hong Kong, London, Paris, San Francisco, and Tokyo ones—giving the international event the same veneer of exclusivity as a sneaker drop. Yet the project isn’t supposed to be exclusive, nor should it. “We hope attendees are inspired by the incredible AR creations in the [AR]T Walk and in-store installation, and we can’t wait to see what our visitors learn to create in the [AR]T Lab,” a press release says. If the point is really to inspire and increase access to new forms of creativity—albeit proprietary ones—it seems like a sold-out schedule should have led to more tours.

The sign-ups are necessary because the walk requires wireless headphones and iPhones provided by the store, as the augmented reality software used for the experience runs on a different operating system than the one currently available to the public. Tours are led by two employees of Today at Apple, the brand’s creative workshop arm. They cheerfully instruct visitors on how to “anchor” their phone cameras to geographic landmarks, much like one would focus on an object in an image. This provides visual data about the surroundings that allow the augmented reality layer to “lock” in place. Each art piece, tied to various locations in Central Park, is “activated” by a guide and lasts a few minutes. The walk is part tour, part tech demo, part art-appreciation seminar. At the end of each augmented reality vignette, the guides read from a slide deck of boilerplate facts about the artist and ask the visitors a series of questions to spark discussion.

A sample questions: “What kind of animal did you think that was?” It was asked in reference to Nathalie Djurberg and Hans Berg’s piece, the most successful of the six AR vignettes. Speech bubbles appear on-screen, ramping up the suspense, as they lead the reader toward an innocuous-looking tree. Locking onto the tree reveals an AR-generated hollow in the wood, where a tiny, genuinely touching scene plays out. The text of a poetic, nostalgic meditation on the enduring nature of selfhood appears in the speech bubbles, while a wolflike creature terrorizes a man in a bed to Berg’s soundtrack. The handheld scale of the animation makes it poignant—you lean in to look more closely. The two characters circle, race, and leap, contained in a hollow the size of an acorn.

The other [AR]T pieces are less successful precisely because of scale. Phone screens are small and reflective. Much of the work is carried by audio. John Giorno’s poem is affecting when read aloud by the author, but the flat graphics of a rainbow path overlaid with the poem’s text feel like an afterthought. Nick Cave’s contribution to the walk, a user-generated, sequin-patterned companion that bobs along the path, is playfully interactive. But the climax of the piece—the emergence of a shimmering dancer, clad in a costume that incorporates the characters designed by everyone in the tour group—winked out under the glare of morning sun on the Gapstow Bridge. Cao Fei’s dreamlike factory, a roller-coaster assembly line of boxes that goes nowhere, felt small and muted, overlaying my toes.

[Subscribe to the Program, a weekly newsletter about art and technology.]

The walk has its moments of physical joy. While chasing Pipilotti Rist’s deranged, screeching, rainbow-hued blob around Pulitzer Plaza, I realized that to outsiders I probably sported an smile just as wild-looking as Rist’s animation—but none of it felt as immersive, or transformative, or even just fun, as it could have been. Of all the pieces, only Carsten Höller’s, which prompts participants to find a randomly assigned partner among the others in the tour group, really takes advantage of augmented reality’s ability to, well, augment reality. It filters all visuals into grainy black-and-white, shifting a visitor’s sense of the world to encourage participation in the social experiment. The screen capture from Rist’s piece—a gooey mass of dish soap streaking across the sky—that has been used for much of the collaboration’s promotional material is exciting to look at, but it’s not true to the experience. The visual is far smaller on the iPhone screen than it is in on the Today at Apple website.

The screen muddles the encounter with art in another way. With a button for making screen captures and another for recording video both clearly visible at the bottom of the display, the interface encourages visitors to document the walk, forcing them into experience-optimization mode. The button freezes for a few seconds after each capture, so if you miss a shot, you don’t get another chance. With the exception of Cao and Höller’s pieces, which encourage interaction, the scenes play out in a linear fashion. There’s no rewinding. Foregrounding the process of capturing the experience means you miss the experience in the moment—instead of fully giving yourself up to the pops and whizzes of the AR technology, you’re too busy trying to nail a shot for Instagram. It’s less New Museum, more Museum of Ice Cream.

The [AR]T walk begs comparison to a much beloved, much more successful augmented reality initiative, Pokémon Go. Unlike [AR]T, which tightly contains its art experience within Apple store devices and guidance, users could play Pokémon Go on their own phones and on their own time, allowing for an expansive, layered experience that transformed the social, physical world. In the summer of 2016, it wasn’t unusual to see a small crowd of gamers clustered around the Hester Street Playground on the Lower East Side, and I often changed my walking path to stop by. Someone would place a lure—a pink, petal-cascading charm that attracts Pokémon—in the AR overlay of the park’s open space, and everyone gathered to play, catching the same creatures, covered in the same pink petals, sharing in a great, demographic-spanning, boundary-crossing experience. If my description makes it sound utopian, it’s because it certainly felt that way to me.

Would it have been so hard to make an [AR]T app that ran on a regular phone? Imagine a crowd of tourists in Central Park’s Grand Army Plaza, pairing up for conversation through Höller’s social experiment. It’s hard to ignore how proprietary of an attitude Apple has taken with this collaboration. As critic Ben Davis notes, this isn’t an exploration of augmented reality in general, but augmented reality as it exists within Apple’s corporate framework. Participants sign up on the Apple website, meet for tours at the Apple store, wear wireless Beats headphones and use borrowed models of the latest iPhones. Even though Cave’s in-store installation doesn’t require the borrowed tech, you still have to download the Apple Store app and be physically present in a brick-and-mortar store. There’s no reason to limit the range of that piece in such a way—it runs on the standard operating system already on your iPhone, and the architecture of the stores doesn’t feel particularly integral to the experience; it could work in any indoor setting. The whole endeavor feels like an insidious reminder of how our everyday lives have become intimately entwined with a handful of corporations that collect increasingly more data about our lives and our movements. Rather than provide a truly public experience, Apple in particular seems insistent on replacing the commons and becoming a universal, inescapable privatized framework. And it’s not even that fun. Why not just go for a walk in the park?