Computer Vision at Arcanum

Images and photos greatly enhance the quality of any article, blog post, or research essay. Although there are many image search services on the internet, authors working on subjects related to Hungary lack a single and robust service to find digitized press photos and figures.
The need for such a service led us to develop our image search engine that heavily relies on artificial intelligence.

In this article, we present the computer vision technology in ADT and how it helps our users.

Beyond the text

Arcanum’s advanced textual search engine has made searching in tens of millions of digitized pages convenient and efficient. However, millions of high-quality photos and other images in these printed publications have been left mostly untapped until now. By utilizing artificial intelligence (‘AI’), especially computer vision, Arcanum aims to help authors and researchers with a handful of smart tools to find relevant imagery sources.

Finding illustrated pages

We wanted to deploy a solution that marks pages with illustrations in our database. To achieve this, we took advantage of classification algorithms 1 , a branch of the convolution neural networks. After training, the AI recognizes and classifies pages based on whether they contain images. Finally, we added an Illustration filter to the existing set of filters at the Advanced search section.

In the following use case example, our goal is to collect illustrations of the Liberty Bridge (Szabadság híd) in Budapest. Start by typing the Hungarian name of the bridge, and enable the Illustrations filter. Our search will result in pages only which have illustrations. Arcanum’s service can therefore show photos that have been published in print but no other search engines find.

Face detection

Based on our user’s feedback, we’ve noticed that most of the searches target persons. Advanced convolutional neural networks 2 can recognize and mark human faces on digital images. Arcanum’s Face filter under the Illustrations category uses one of these algorithms: ADT can list only those pages on which the AI detects a human face.

Our next example shows how to find photos of the world-famous Hungarian composer Zoltán Kodály. Type in the name, then click the checkbox to enable the Face filter under Illustrations .

Face search from a file

The technology we presented here enabled us to collect millions of images of human faces from our database of digitized publications. We made this database available to our subscribers: once a face photo is uploaded, the algorithm analyses the features of the face, then it compares to other faces in the database.

The enhanced AI can find photos of the same person in her younger or older ages, even in low-quality digitized prints. Clicking on an image opens the page in the reader view from where you can read the whole issue. We hope our service can help find exciting stories of a distant relative or friend in an old family photo. Face search can also be a tool for researchers, scholars, and journalists when collecting creditable sources on famous personalities.

In our next example, we uploaded a photo of the world-famous Hungarian football player, Ferenc Puskás, which depicts him in his fifties. The AI recognized the unique features of his face, then it listed similar images. Note that the system lists photos of different ages and body shapes.

Read more on the face recognition in this study by Amazon:

Arcanum makes Hungarian heritage accessible with Amazon Rekognition
by Sinisa Mikasinovic and Cameron Peron | on 23 OCT 2020

*: Learn more about the ResNet classification algorithm on this page.
https://en.wikipedia.org/wiki/Residual_neural_network
** : Read more on the Single Shot MultiBox Detector algorithm here.
https://arxiv.org/abs/1512.02325z