Playing like a machine: The New Aesthetic in Gaming

Guest columnist Meg Jayanth looks at the New Aesthetic, and how it's changing the way you play games.

The New Aesthetic

Picture by Ben Terrett

At this year’s SXSW there was a panel about something called The New Aesthetic: Seeing Like Digital Devices – to put it simply: it’s about how ‘new ways of looking at the world through technology’ are proliferating in art and design and products. Gizmodo says its about “process[ing] reality through the lens of technology”. So far, so complicated.

Really, the best way to understand the New Aesthetic is to scroll through James Bridle’s tumblr – which is where, arguably, the New Aesthetic began to take shape as an art movement. Like pornography, you know it when you see it. It’s a series of artifacts “not a movement” (though it kind of is one). It’s echoes and weirdness, QR codes and machine-readbility, render ghosts and animated GIFs, google map-grabs and pixellation, facial recognition and natural language processing. If you’re alive in the developed world today and you’re reading this article on some kind of technological device, you’re in a New Aestheticized world. Bruce Sterling, in his sprawling Wired article about NA says:

the New Aesthetic is culturally agnostic. Most anybody with a net connection ought to be able to see the New Aesthetic transpiring in real time.

That right there is what’s interesting about the New Aesthetic – it isn’t the science fiction future or the Baby Boomer nostalgia for jetpacks. It’s all around us, and we’re part of it, like it or not – congratulations!

A man in a Gameboy Costume

A man in a Gameboy Costume - Picture by Dark Boxty

Clearly the New Aesthetic is zeitgeisty – it’s of the moment. Bridle’s tumblr, the SXSW panel and follow-up discussions have generated many responses – applying the New Aesthetic to everything from fashion to storytelling  . There is no definitive manifesto. There are no edges or end point or secret club that you can sign up to. It’s a diffuse and ongoing discussion about what it means to live in the contemporary digitally connected world.

I tweeted about writing an article about the New Aesthetic and received a notification telling me I’d been retweeted by @NArtBot – a bot account that aggregates mentions of #newaesthetic in a continuous stream. That tells you a lot about what you need to know – is it a movement if it’s automated? Is it a clever way to avoid the problems of tastemaking? I have no idea, but I’m interested in where it’s going.

Windows Error Message Billboard Ukraine

A digital billboard malfunction in Odessa leaves an error message floating in the night sky. Picture by Felliniesque

Much – but not all –  of what’s been written and responded to so far has been about visual culture. The fact that the New Aesthetic’s version of primordial soup is a tumblr may have something to do with that. Image and animation are often the most jarring examples of digital incursions into the “real” world. So naturally there’s an emphasis on showing us what it’s like to see like a machine – and less on the question of who is doing all of this seeing? (And before you ask: yes, it’s important who is doing the looking. We’ll get into why that is a bit later, trust me on this for now.)

Partly, Bridle and the panelists get around the problem by talking about the devices as if they have agency: phones know their location, algorithms read the news, the camera-mounted car is an organ of sight for the diffuse Google Street View body that decides to look. Sterling is spot on when he asks: “where did the people go?

The softly spoken corollary to the New Aesthetic machine-readable world is – must be – humans reading machines, digital decoding. The digitised human and the humanised machine trying to converge their perspectives to see the world in perfect focus. Or at least, starting to come alive to the idea of cohabiting – seeing the traces left by each other, and accepting them. Like Goldilocks and the three bears, only they lived happily ever after in a house wallpapered with QR codes that are purely decorative.

Here’s a provocation: if the New Aesthetic is about seeing like digital devices, then it’s also about playing like machines.

Man in mocap suit

Picture by Loren Javier

Games are a strange absence in the discussion around the New Aesthetic – strange because NA so infused with play and technological innovation and visual design. I’m a gamer, so obviously there’s some bias here – but the gamer identity is so steeped in all of these critical and cultural concepts. If we’re talking about reality processed through the lens of technology, then we have to talk about games.

Because that’s what videogames are – a bunch of humans writing some code to respond to the inputs of another bunch of humans – a continuing cycle of humans reading machines reading humans reading humans with some fun thrown in to lubricate the exchange. Not in a dirty way. Okay, fine, sometimes in a dirty way. What I’m saying here is that videogames are, in essence, machine-readable play – and that’s interesting.

What’s more “human” than play? What’s more strange and disorienting than machines built to surprise and delight, living in that darkly lit place between the quantified self and inner child?

Girl playing on handheld console

Picture by Hans S

(Side note: Will the first machines to achieve artificial consciousness be NPCs in the latest Xbox title? Will Milo be their venerated ancestor, reaching out to clasp our ghostly meatspace hands?)

What’s more intimate than sharing joy? What’s more weird than sharing joy with a digital device, having your controller vibrate like a cat purring in your hands? Aesthetics have to move beyond the surface feelings of strange and jarring and weird – though those moments and places give us a good idea of where to dig.

As we see machines, they see us, and we see ourselves through each other – the relationship is dialectical. As we play with machines and build machines to play with, they start to change the nature of play – a nicely brought up 18th century gentleman would probably not recognise images of kids sitting in front of televisions with Wiimotes in their hands as “playing”. The places we play, what we play with, how we look at and think and talk about play are all increasingly entwined with the digital.

It’s the point at which you buy a floorlamp so that Kinect can see you better or spare a microsecond to think about whether the livingroom floorspace in the flat you’re thinking of renting is optimal according to Microsoft guidelines. That’s the demands of computer vision reaching out and making marks on the physical, using the best tool to hand: you.

Games and gaming technology are predicting and then making real – with code, consoles, peripherals – the way our digital future looks in that most private of spaces, our homes. The New Aesthetic – as it were – may not have a manifesto but Sony and Microsoft sure do.

Wonderbook is Sony’s vision of children playing digitally – holding physical objects, responding to digital cues, seeing themselves augmented by machine-vision and machine-graphics. Book of Spells is quite literally making magic machine-readable. Magical is the watchword – Apple was the first to realise this – the magic of technology intuitively understanding us, eliding the ways in which we teach it and ourselves a common language. Bridle says we need to see our technologies with “a new wonder”.

The book itself is a fascinating object – it’s human-readable in that it’s skeuomorphic book-shaped with pages that can be turned. It’s the right size and shape to fit on a bookshelf. Pull it down and open it up, a moment of strangeness: instead of text there are machine-readable markings on each page. It looks more like an ancient cypher, a tablet to an arcane and all-seeing god, cuneiform for the computer age. Really, it’s a Rosetta stone, allowing children and computers to translate each other.

Kinect’s achingly beautiful and frankly evangelical (“Xbox 360 set out to change the way we interact with games and entertainment”) “Kinect Effect” spot is all about machines reading humans in unexpected situations:

Where did all the people go? They’re right here – you can see the human-shaped holes in the technologies we’ve already brought into our homes. And not just to play with – machines are learning to understand human pursuits as varied as medicine and sports and music and business and learning. Any one of the frames from the “Kinect Effect” spot could work as the cover image for an imaginary and unnecessary New Aestheticist manifesto.

It’s important to point out that this is how games companies and technologists see the digital present and future – smiling children and smooth surfaces, perfectly working peripherals and middle-class domesticity. This is a tame New Aesthetic – it’s softly magical and certainly not too weird or disruptive. Machine intimacy presented as expected, unthreatening, part of the Ikea furniture. It does not glitch. Its user interfaces are all child-friendly. Hardware not software.

But these aren’t the only visions of machine intimacy on the market – even if we limit our scope to videogames. In Part II I’m going to talk about the very different relationships between humans and machines that are visible in scifi games like Deus Ex: Human Revolution, Syndicate and the as-yet-unreleased Watch Dogs – games that allow us to see like machines, and ask some pretty relevant questions about the ethics and aesthetics of technology.