AI & Museum
Hackathon x Vol. 2 - 2021
From 25-27 June 2021, Hackathon x Vol. 2 took place as a cooperation between the Badisches Landesmuseum, Allard Pierson Amsterdam and Hack & Söhne. More than 140 people from all over the world took part in the online developer event, which was held under the motto "The Digital Museum" and dealt with challenges on the topics of "Games, Platforms and Artificial Intelligence" and developed creative solutions in 48 hours. Inspiring tech talks and mentoring, online yoga and social gaming on the digital platform Gather.Town accompanied the weekend. More than 20 projects were submitted by 100 people, of which 6 winning projects in 2 sections convinced an interdisciplinary jury.
Creative Solutions for Artificial Intelligence in the Museum
In the section on Artificial Intelligence in the Museum, data sets from the Badisches Landesmuseum and Allard Pierson were made available. The challenge was to develop concepts and applications that use artificial intelligence to make digital museum content available as personalised recommendations for users.
The winning projects of the AI Challenge are:
1. Prize: Etienne Bührle: William: Chat with William III
2. Prize: Andrii Strynzha, Raghavendra Vijayanagaram, Fabian Ulmer, Maxim Popov, Nikita Nesterov: Your Guide - Personalize your visit to museum using generated tour based on your interests
3. Prize: Nandakishor M: Image chat conversational AI app for museum arts
All projects can be viewed here:
https://hackathon-x.devpost.com/project-gallery
Perspective
The winners and other projects are invited to further elaborate their ideas in an AI development phase!
The jury of the AI Challenge:
- Susanne Schulenburg, Baden State Museum Karlsruhe
- Wim Hupperetz, Allard Pierson Amsterdam
- Matthias Wölfel, Karlsruhe University of Applied Science
- Jaap Kamps, University of Amsterdam
- Ariana Dongus, Karlsruhe University of Arts and Design
Funded by:
BKM Staatsministerin für Kultur und Medien
Mediapartner
DeHub Karlsruhe
AI Development Plan – Winning Projects
Following Hackathon x - vol 2 some teams were invited to continue working on their ideas and concepts at the Development Plan. Two teams took up this challenge:
- Nandakishor M - with "Image Chat”
- Richárd Ádám Vécsey & Axel Ország-Krisz – with "Pong"
In 4 weeks of developing the concepts and getting feedback from users and heritage experts, the ideas could be further elaborated. The results are understood as part of the development of "Creative User Empowerment" and made useful for the overall project. The winning projects and partial results are therefore presented here:
Moving towards Augmented Reality & AI
As a virtual guide in the museum, the project works with an AR audio bot (Blenderbot 2.0) with internet search function and AWS to obtain information about artists or the museum. The chatbot has a long-term memory and the ability to access the internet. It outperforms existing models in terms of longer conversations over multiple sessions and has more knowledge and factual consistency than before, according to reviewers. The model stores the knowledge gathered during the conversation in a long-term memory and uses this experience to have long-term conversations. During the conversation, the model searches the internet by making its own queries, reading the results and taking them into account when formulating an answer.
About the Developer:
Nandakishor is an Udemy Artificial Intelligence teacher and Managing Director of deepflow technologies Pvt Ltd, a deep tech startup from India. He is an expert in computer vision,NLP, robotics and IoT. www.deepflow.in
Laudatio:
"The image chat app based on conversational AI is an ambitious and timely idea, with already a working prototype. Using an image as a conversation starter is a great idea, and combining text and vision holds great promise, for providing additional information or recommending other museum objects of relevance to the user." (Jaap Kamps, University of Amsterdam, Member of the Jury)
https://github.com/kishorkuttan/Image-chat-conversational-AI-app-for-museum-arts-
PONG
Pong is an AI and ML based artwork recommendation system and provides artwork-level prediction based on different approaches. Recommendation is part of our everyday life. We can meet AI, ML or simple algorithm based recommendation systems all the time from choosing music or movies through the ideal program and washing time on washing machines till the autocorrect service of different messaging apps. The demo recommends 27.898 artworks based on 2074 keywords, 7 different colorfulness categories or outputs of AI or ML models. Pong wants to develop a system that is easily maintainable and the cost of operation is low because there is no need to use cloud infrastructure or services.
About the Developers:
Dr. Ország-Krisz Axel
Axel works as a freelance deep learning developer, data scientist and career jumper from law. He thinks artificial intelligence should be accessible for everyone to let people focus on important things like society and general values instead of problems of daily life.
Dr. Vécsey Richárd Ádám
Richard works as a freelance deep learning developer, data scientist who jumped from the field of law into IT. He likes to play with AI to create complicated solutions to easy problems.
Laudatio:
“In the Development Plan, Pong has further developed the idea of recommending artworks and has worked intensively with the museum's data and metadata. Their recommendation system was differentiated in relation to rating systems and options for reaction-based rating and personal impact-based rating were used. After a warm-up phase, users are suggested artworks based either on colour, content (keywords) or a combination. The critical exploration of recommendation algorithms is an important contribution to the future of the museum.” (Sonja Thiel, Project Lead)
https://github.com/hyperrixel/pong
