About
Background
The MMM was ideated during Fall 2018 in Professor Kimiko Ryokai’s course on Tangible User Interface (TUI) design, where Arianna, Justine, Fang, and Rachel worked towards an interface that would facilitate novel interaction between people. Inspired by projects that brought people together through touch, we created a musical mat that amplified touch-based interactions through sound. Rachel, having studied the interactions of non-speaking autistic individuals for years, saw the potential in bringing the project into their homes and schools. Rachel, Arianna, and later Rebecca thus began working closely together, iterating on the mats, building a range of musical palettes, and having fun along the way.
Motivation
Non-speaking autistic children often have to accommodate to the participatory expectations of speaking others. Towards inclusive practices, how can interaction embrace the expressive repertoires of the autistic child? What sociomaterial environments might surface such interactions? The MMM goes beyond speech, embracing touch and music as a means for people to connect. The MMM removes interactional asymmetry between diverse people and surfaces the basic human capacity to connect with one another.
Work to date
The MMM has undergone several design-research cycles. In 2019, our team developed the environment further with autistic children and their therapists in a clinic at San Francisco State University. Through close observations of the children’s interactions, as well as feedback from everyone, we improved on the form factor of the mats. In 2020, our team created a variety of musical palettes beyond our initial version. We played with timbre, chords, melody, and other musical properties, mapping sound onto touch-based gesture in different ways. From 2021, MMM was brought into the homes of non-speaking autistic children. Through collaboration with their families, careful observation of their home lives, and thoughtful consideration of their physical home environments, the MMM was customized to each child. The MMM surfaced intimate touch-based and musical interactions between these children and their family members. One project can be read in Rachel’s dissertation. Results of our latest project is still being written. Through these phases of the project, the MMM underwent more iterations, attuned to the everyday lives of autistic children.
Future plans
As of July 2023, Rachel has completed her PhD in Special Education at UC Berkeley and San Francisco State University. Arianna and Rebecca have graduated from UC Berkeley. Rachel has plans to develop and disseminate MMM within community and clinical partnerships in her current faculty position at Nanyang Technological University. She hopes to concretely involve non-speaking individuals as part of the future team behind MMM. Music, improvisation, and sound design feeds Rachel’s soul, and she is excited to keep growing MMM as a rich environment for musical improvisation. Currently, Rebecca is generously working with Rachel to wrap up this current phase of the project, and otherwise geeking out with her about music, slam poetry and life. Arianna is no longer involved in MMM, but is still Rachel’s number one crafting buddy.
Funding
Barbara White Bequest; Jacobs Innovation Catalyst Grant (winner of Jacobs Design Showcase 2019); Berkeley Center for New Media fellowship; Institute of Cognitive and Brain Sciences