Facebook and Google are building enormous neural networksâ€”artificial brainsâ€”that can instantly recognize faces, cars, buildings, and other objects in digital photos. But thatâ€™s not all these brains can do.
They can recognize the spoken word, translate from one language to another, target ads, or teach a robot to screw a cap onto a bottle. And if you turn these brains upside down, you can teach them not just to recognize images, but create imagesâ€”in rather intriguing (and sometimes disturbing) ways.
As it revealed on Friday, Facebook is teaching its neural networks to automatically create small images of things like airplanes, automobiles, and animals, and about 40 percent of the time, these images can fool us humans into believing weâ€™re looking at reality. â€œThe model can tell the difference between an unnatural imageâ€”white noise youâ€™d see on your TV or some sort of abstract art imageâ€”and an image that you would take on your camera,â€ says Facebook artificial intelligence researcher Rob Fergus. â€œIt understands the structure of how images workâ€ (see images above).
Meanwhile, the boffins at Google have taken things to the other extreme, using neural nets to turn real photos into something intriguingly unreal. Theyâ€™re teaching machines to look for familiar patterns in a photo, enhance those patterns, and then repeat the process with the same image. â€œThis creates a feedback loop: if a cloud looks a little bit like a bird, the network will make it look more like a bird,â€ Google says in a blog post explaining the project. â€œThis in turn will make the network recognize the bird even more strongly on the next pass and so forth, until a highly detailed bird appears, seemingly out of nowhere.â€ The result is a kind of machine-generated abstract art (see below).
On one level, these are party tricksâ€”particularly Googleâ€™s feedback loop, which evokes hallucinatory flashbacks. And it should be noted that Facebookâ€™s fake images are only 64-by-64 pixels. But on another level, these projects serve as ways of improving neural networks, moving them closer to human-like intelligence. This work, says David Luan, the CEO of a computer vision company called Dextro, â€œhelps better visualize what our networks are actually learning.â€
Theyâ€™re also slightly disturbingâ€”and not just because Googleâ€™s images feel like a drug trip gone wrong, crossing breeding birds with camels in some cases, or snails with pigs (see below). More than this, they hint at a world where we donâ€™t realize when machines are controlling what we see and hear, where the real is indistinguishable from the unreal.
Working alongside a PhD student at New York Universityâ€™s Courant Institute of Mathematical Sciences, Fergus and two other Facebook researchers revealed their â€œgenerative image modelâ€ work on Friday with a paper published to research repository arXiv.org. This system uses not one but two neural networks, pitting the pair against each other. One network is built to recognize natural images, and the other does its best to fool the first.
Yann LeCun, who heads Facebookâ€™s 18-month-old AI lab, calls this adversarial training. â€œThey play against each other,â€ he says of the two networks. â€œOne is trying to fool the other. And the other is trying to detect when it is being fooled.â€ The result is a system that produces pretty realistic images.
According to LeCun and Fergus, this kind of thing could help restore real photos that have degraded in some way. â€œYou can bring an image back to the space of natural images,â€ Fergus says. But the larger point, they add, is that the system takes another step towards whatâ€™s called â€œunsupervised machine learning.â€ In other words, it can help machines learn without human researchers providing explicit guidance along the way.
Eventually, LeCun says, you can use this model to train an image recognition system using a set of example images that are â€œunlabeledâ€â€”meaning no human has gone through and tagged them with text that identifies whatâ€™s in them. â€œMachines can learn the structure of an image without being told whatâ€™s in the image,â€ he says.
Luan points out that the current system still requires some supervision. But he calls Facebookâ€™s paper â€œneat work,â€ and like the work being done at Google, he believes, it can help us understand how neural networks behave.
Layers Within Layers
Neural networks of the kind created by Facebook and Google span many â€œlayersâ€ of artificial neurons, each working in concert. Though these neurons perform certain tasks remarkably well, we donâ€™t quite understand why. â€œOne of the challenges of neural networks is understanding what exactly goes on at each layer,â€ Google says in its blog post (the company declined to discuss its image generation work further).
By turning its neural networks upside-down and teaching them to generate images, Google explains, it can better understand how they operate. Google is asking its networks to amplify what it finds in an image. Sometimes, they just amplify the edges of a shape. Other times, they amplify more complex things, like the outline of a tower in a horizon, a building in a tree, or whoâ€™s knows what in a sea of random noise (see above). But in each case, researchers can better see what the network is seeing.
â€œThis technique gives us a qualitative sense of the level of abstraction that a particular layer has achieved in its understanding of images,â€ Google says. It helps researchers â€œvisualize how neural networks are able to carry out difficult classification tasks, improve network architecture, and check what the network has learned during training.â€
Plus, like Facebookâ€™s work, itâ€™s kinda cool, a little strange, and a tad frightening. The better computers get at recognizing whatâ€™s real, it seems, the harder it gets for us.
Go Back to Top. Skip To: Start of Article.