Machine and Deep Learning Limits

If decisions are to be delegated to artificially intelligent machines, we need to appreciate the limits of intelligence without cognition

---

Wired has just published an article – How to teach artificial intelligence some common sense – discussing the growing recognition that machine-based deep learning algorithms, whilst proving incredibly successful at undertaking and even exceeding us at some human-like tasks, such as face recognition and learning to play games, are still a long way from mimicking human-level cognition. An artificial intelligence learns optimal choices by training on vast quantities of data, either using existing data archives or by running thousands of simulations of a defined scenario. Make a small change to the scenario and a human will immediately adapt, even if with only limited success initially. We have the ability to create mental images about what impact the change will have, and adjust our actions accordingly. The AI doesn’t adapt, it has to relearn.

There is another limitation with AIs. They don’t really know anything about the real world. Sure, they can learn from massive data sets and potentially spot minor differences between images that would be difficult for the human eye to perceive. But that comes from studying millions of images down to the level of individual pixels. What the data archive doesn’t tell them is what are the natural limits for any one object. How does a human figure this out? Through experience and generalisations that enable us to separate the impossible from the possible and, more importantly, the implausible from the plausible.

To give an example:

A recent advancement in deep learning – the use of artificial neural networks to train a machine intelligence – has been the introduction of a generative adversarial network (GAN) method. The GAN is a neural network composed of two datasets that compete with each other. Without getting into the technical details (there’s a link under references if you want a good simple introduction), it identifies what makes two images both different and similar.

GANs can be used to create new images by incorporating features unique to each data set. For example, taking pictures of men wearing sunglasses, and women not wearing sunglasses, the algorithm will digitally create a realistic image of a woman wearing sunglasses that does not look like a simple copy/paste. The GAN can not only add sunglasses, it can create entirely new faces, with and without glasses, that look like real photos of real people but in reality are fake photos of non-existent people.

However, there is a limit to this ability. When research about GANs began to be made public, one of the examples was turning a zebra into a horse and vice versa. I was sitting in a seminar about this and, given I know a bit about horses, I paid a bit more attention to the results. Because one set was convincing, whilst the other was not. The image is included below:

To explain what is going on. Each image on the left is a real photo. Each image on the right has been created artificially by the GAN. In the top case (zebra -> horse), the real image is of a pair of zebras scratching. The GAN-generated image has taken the image of the zebras and converted it into horses. Now this example isn’t bad. Look closely and you can see some oddities. But if somebody presented the right-hand image to me, at first blush I would say it was a picture of a pair of native-breed horses… and there’s the rub. Horses come in all shapes and sizes, and include a lot of native breeds with short stubby legs and big ears. But horses can also have long legs and little ears. And their confirmation will determine how gracefully they move. I have yet to see a picture of a zebra that does not have short stubby legs. No zebra is going to win a horse race, dressage or jumping competition any time soon.

This leads on to the bottom pair of images, horse -> zebra. The image on the left is a real photo of a horse. The image on the right is GAN-generated to make a zebra from the picture of a horse. Not on Earth will you find a zebra moving like that. It’s a clever application of zebra colouring to a horse. But the outline and leg position means it is without doubt a horse.

The reason for labouring this point is that this example happened to be from a domain that I have expertise in. So I can easily see it is a fake. But it makes you wonder what other domains have reality limitations that an artificial image-generator fails to understand? If such techniques are used to create synthetic data for use in real-world scenarios, we need to be aware of those limitations.

A final example, mostly because it’s an excuse to include pictures of my old horses. The image below is of two of my showjumpers scratching on an early spring day back in 2004. I could hand this photo to a human with experience of horses and they would immediately be able to say a) it’s late winter/early spring, b) the bay (brown) horse is probably in competition, c) the grey horse is probably old, and d) they are both horses not zebras.

Why? The bay mare has been clipped (you can see the change in colour of her coat at the top of her legs) and is wearing protective boots. She’s also in competition condition. The grey mare looks woolly but is shedding her coat (bits of it are visible on the floor). And she has a dropped belly that never recovered post-pregnancy, typical of an older mare. Recolour them with zebra stripes and they still wouldn’t be zebras. 🙂

References

• How To Teach Artificial Intelligence Some Common Sense – Wired,
• Generative Adversarial Networks – A Deep Learning Architecture – Gautam Ramachandra, 5 November 2017
• Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks – Zhu et al, Aug 2018

---

Header image: author’s own photos (any excuse for horse pictures). Same two mares scratching at different locations. And the grey from her younger days before that dropped belly (with a much younger author on board)… 🙂