Shin Hanagata - 'A Garden of Prosthesis' "Walker AI"

Technical contribution to the performance work 'A Garden of Prosthesis' by Shin Hanagata, focusing on real-time human-AI interaction.

As the technical lead, I was responsible for implementing the AI application that formed a core part of this performance taking the role of a "Walker". The primary focus during development was to create a system enabling real-time interaction between the human performing artists and the AI, and to explore the transformations that emerge within the performers through this dynamic interplay.

Concept

Overview

(Excerpt from the official page)

A garden exists as a form in which various objects such as rocks, plants, and landforms are arranged... In this performance, not only rocks and trees, but also tools, machines, AI, virtual objects, and the human flesh are equally involved in the garden. This performance attempts to create "A Garden of Prosthesis," a garden in which these objects heal each other as parts of their own flesh, or “prosthesis,” as they are linked in a chain and mixed.

Inspired by the garden, the "gardener" treats artifacts, natural objects, virtual objects, and "objects," including human bodies, equally and acts as a catalyst for their relationship and blending to create the garden. Strollers, including photographers, AIs, and visitors, stroll through the garden, witnessing and becoming involved in the relationships among the flood of objects.

The performance involves three key elements:

• The Objects: Man-made objects, natural objects, virtual objects, and human bodies.
• The "Gardeners": Those inspired by the garden to compose it, including behind-the-scenes creators.
• The "Walkers": Visitors, photographers, and AIs who wander through the garden.

My role focused on building the AI systems that acted as both "Objects" and potentially influenced the "Gardeners" and "Walkers" through real-time interaction.

During the performance, input from a camera mounted on a robot vacuum cleaner and cameras recording the performance is sent to the application. The results of object recognition on these video feeds and the situation interpretation by the AI (OpenAI GPT-4 Vision) are projected in real-time.

The application's behavior is designed with a focus on whether this output could induce transformations in the performance of the objects (meat) and the gardeners.

The information shared with the audience and gardeners via text and voice consists of visual object recognition results (sometimes misrecognitions) and their interpretations.

The application is designed so that the contextual information (context prompt) given to the model for interpreting the situation can be selected in stages.

Consequently, towards the end of the show, the AI's behavior changes to articulate interpretations based on some understanding of the performance's concept.

Shows

TOKAS OPENSITE 8, TOKAS Hongo (Feb 2nd-4th, 2024)

Credit

(TOKAS

OPENSITE

8,

2024)

• Performance: YOSHIMITSU Yoshiya, MIYOSHI Karu, YAMAGUCHI Miina, Vacuum cleaner, Steel pipes, Trees, Meat (HAGIHARA Fujio, IMASHUKU Mew, HANAGATA Shin, etc.), AI, Virtual objects, etc.
• Technical: KOBAYASHI Atsuya, JACKSON kaki, SUGINO Shimpei
• Prosthesis production: YASUDA Nobuhiro
• Installation / Space Design: mooney / Munehiro OHTA, dodi
• Management: NISHIMURA Rioha
• Graphic design: NAKAMURA Harumichi
• Key Visual: MIYOSHI Karu ＋ JACKSON kaki ＋ YOSHIMITSU Yoshiya ＋ YAMAGUCHI Miina ＋KOBAYASHI Atsuya＋ OSAWA Itsuhiro ＋ IMASHUKU Mew ＋ HANAGATA Shin
• Filming: UENO Takahiro, NAKAO Ippei
• Photography: OHNO Ryusuke, Photo courtesy of Tokyo Arts and Space
• Support: Project to Support Emerging Media Arts Creators, Agency for Cultural Affairs

Learn more about the work on the artist's official page: https://www.shinhanagata.com/works/agop

Other

links

HANAGATA Shin “A Garden of Prosthesis”｜ Exhibitions / Performances ｜ Archives ｜ Tokyo Arts and Space

A Garden of Prosthesis/Thanatosis (Previous work of the AGOP project)