How AI- and Machine Learning (ML)-based solutions can be used to restore ancient cave murals

The murals of the Ajanta and Ellora caves are national treasures with the oldest areas of the truly breath-taking Ellora complex dating back to at least 1000 BCE, according to certain sources. This UNESCO World Heritage Site contains monuments featuring Hindu, Buddhist and Jain traditions, signifying religious harmony. Preservation of such sites will go a long way in helping build a strong national identity.

However, these sites require constant attention. Having faced thousands of years of weathering, it is but a matter of time until the original murals at Ajanta disintegrate. In many other cases, such valuable treasures have been lost. Today technological tools like AI and automation help make the restoration process easier and more efficient. That was the aim of the Tech4Heritage hackathon: to digitally preserve and restore the murals recovered from these sites.

Our team comprised Arjav Jain and Aryan Prasad and the two of us, all second year students of B.Tech. Mechanical Engineering at IIT Roorkee. We started with damaged images from the Ajanta caves, which we downloaded from the Internet, to build a unique dataset of images. This served as a foundation upon which a generative model was trained to identify the damaged areas. These images were later restored, through AI models utilising Generative Adversarial Networks (GANs).

The hackathon had set a specific problem statement, which demanded technical knowledge in image processing and deep learning in computer vision. Our team had been exploring practical applications in these fields and this was a big help given how complex the issue was. Once the problem statement was communicated, we had 15 days to come up with viable solutions. We decided to pilot multiple strategies and proceed with the optimum solution. Although 15 days was not enough to produce a perfect solution, it created a strong base upon which our solution could be scaled and applied across so many cases.

How GAN helps

Given the nature of how GANs operate, we can improve the process of digital restoration over time, as it learns through the dataset of damaged paintings. Initially, we had a limited dataset take from the Internet. So, we decided to create our own. With image processing and machine learning, we were finally able to create a dataset of more than 800 images from 36 original ones. We then applied more algorithms to detect the spots and damaged parts, after which came the most important step: training the algorithm to fill these spots based on the surrounding context.

We’ve been told that this initiative doesn’t end with completion of the hackathon. One of the organisers, along with the top three teams, is planning to implement this on a much larger scale and preserve our work in Arctic World Archive. We’re grateful to have to got this unique platform to crack a complex real-world problem: the digital preservation of our shared heritage.