How Computer Science Can Help Art Historians

How Computer Science Can Help Art Historians
To sign up for our daily newsletter covering the latest news, hacks and reviews, head HERE. For a running feed of all our stories, follow us on Twitter HERE. Or you can bookmark the Lifehacker Australia homepage to visit whenever you need a fix.

I was the lead of a team of computer scientists at Rutgers that published a paper this past August titled, “Toward Automated Discovery of Artistic Influence.” In that paper we reported on our research, where we used object recognition techniques to compare paintings and identify similarities between them in colour, form, texture and content. The goal was to classify painting styles and determine how artists may have been influenced by their predecessors.

Reactions from art historians, critics, and artists ranged from enthusiastic to dismissive. London’s Daily Telegraph reported on our submission with the somewhat hyperbolic headline: “Could computers put art historians out of work?” Shortly thereafter, Professor Griselda Pollock, an art historian at University of Leeds, was more critical, writing a piece titled, “Computers can find similarities between paintings — but art history is about so much more.”

It’s clear that there’s a fundamental misunderstanding in the art history community about why a computer vision scientist would be interested in exploring resemblances among works of art.

For some background: in our research, we analysed roughly 1,700 digitised images of paintings by 66 masters from the 15th century to the 20th century.

Our computers noted interesting parallels. For example, Frederic Bazille’s Studio 9 Rue de la Condamine (1870) and Norman Rockwell’s Shuffleton’s Barber Shop (1950) possess strikingly similar compositions.

Using the results, we created a graphic of the 66 artists used in our study — what we called a “map of artists” — a graphic of artists’ names where proximity implies similarity. The computer also hypothesized about artists whose paintings may been influenced by a predecessor, like Degas and Delacroix. In many cases, these hypotheses matched those of art historians.

In her article, Professor Pollock explained that art history is not about tracing influences by comparing form, texture, and colour — what, she noted, is called “connoisseurial art history,” pioneered in the 19th century by Giovanni Morelli. Professor Pollock went on to (correctly) point out that the discipline of art history has evolved since then: it encompass politics, economics, literature, ideology, theology, philosophy, while embracing the perspectives of contemporary social movements.

Essentially, she was saying that our research was predicated on a definition of art history that’s outdated.

However we didn’t make any claims about what art history is or isn’t. We understand that influences between artists is not just about visual similarity — that, as a discipline, art history requires studying social, personal, ideological, and historical contexts.

Here’s what we find compelling: when people look at paintings, they can make sophisticated inferences beyond simply recognising a tree, a chair, or the Christ; they can predict which artist likely did the work, note the painting’s style, its genre, and its period. Learning and judging such complex visual concepts is an impressive ability of human perception.

Such ability intrigues researchers in my discipline. It’s our ultimate goal to make a machine that passes the traditional Turing test (when an interrogator is unable to guess which of two players is a machine and which is a human). Of course in the case of paintings we can think of a similar test, which I would call the “visual Turing test.” While passing this test may be a ways off, our results offer an important breakthrough: proof that a machine can learn to discover interesting similarities among paintings that humans can confirm.

Furthermore, Professor Pollock argued that it’s a fallacy to claim that computers can notice things that humans cannot. Computer vision promises — and in fact has proven — that it can overcome human vision’s spectral limitation and quantitative imprecision. In the domain of art, computer vision can offer a level of precise and objective analysis.

It’s true that influence discovery might be based on a century-old view of art history. But when a child is learning to walk, she first crawls before taking her first steps. Later, she runs.

Artificial Intelligence is still in its infancy, especially as it tries to tackle challenges of increasing complexity. If we hope to make a machine understand sophisticated connections between artists, it’s only natural to initially approach the challenge the same way Giovanni Morelli and connoisseurial art historians did a century ago.

The humanities are becoming digitised; paintings, photographs, letters, centuries-old books, volumes of journals and magazines — all are increasingly available online.

And perhaps there will be a day when the technology could evolve to look at the historical, social, and personal context of art — a day when computers could mine these vast stores of heterogeneous data to conduct an analysis of artistic influences that goes beyond the connoisseurial approach.

At that point, it would be up to art historians to figure out how to best utilize these technological advances.

Do we hope to put art historians out of work? Hardly. If anything, I tend to agree with a November 10 Washington Post article headline about our research: “Computer Science Putting Art Analysis on Faster Track,” it reads.The Conversation

Ahmed Elgammal is Professor of Computer Vision at Rutgers University.

This article was originally published on The Conversation. Read the original article.


  • Reminds me of the fact that robustly developed statistics give better analysis than expert opinion, in quantifiable areas of course. I think that comes from either Kahneman’s work, or maybe Gigerenza. But it relates to the size of the datasets: stats can handle any size dataset, an expert can only go on experience which is alway coloured and by the limited sized sets that the mind can handle.

  • The direction of time (sequence) implies influence once a ‘style’ first emerges and then is referenced or replicated by a second artist. The implication can only be deemed as coloration, not causation. Further, looking to purely formal elements such ‘form, texture, and colour’ but not subject or agenda (eg, political views expressed) are overlooked and are, one could argue, far more influential when looking at the relationship between artists.

    Finally, a use for that fine arts qualification.

Show more comments

Comments are closed.

Log in to comment on this story!