We have two papers accepted at the ACM conference on Intelligent User Interfaces (IUI). The two papers addresses investigates interactive machine teaching in two different context: 1) the use interactive machine teaching for arm prosethesis personalisation; 2) the collaborative strategies in interactive machine teaching.

Vaynee Sungeelee, Nathanaël Jarrassé, Téo Sanchez, Baptiste Caramiaux. Comparing Teaching Strategies of a Machine Learning-based Prosthetic Arm. ACM IUI, 2024

Abstract: Pattern-recognition-based arm prostheses rely on recognizing muscle activation to trigger movements. The effectiveness of this approach depends not only on the performance of the machine learner but also on the user’s understanding of its recognition capabilities, allowing them to adapt and work around recognition failures. We investigate how different model training strategies to select gesture classes and record respective muscle contractions impact model accuracy and user comprehension. We report on a lab experiment where participants performed hand gestures to train a classifier under three conditions: (1) the system cues gesture classes randomly (control), (2) the user selects gesture classes (teacher-led), (3) the system queries gesture classes based on their separability (learner-led). After training, we compare the models’ accuracy and test participants’ predictive understanding of the prosthesis’ behavior. We found that teacher-led and learner-led strategies yield faster and greater performance increases, respectively. Combining two evaluation methods, we found that participants developed a more accurate mental model when the system queried the least separable gesture class (learner-led). Our results conclude that, in the context of machine learning-based myoelectric prosthesis control, guiding the user to focus on class separability during training can improve recognition performances and support users’ mental models about the system’s behavior. We discuss our results in light of several research fields : myoelectric prosthesis control, motor learning, human-robot interaction, and interactive machine teaching.

Behnoosh Mohammadzadeh, Jules Françoise, Michèle Gouiffès, Baptiste Caramiaux. Studying Collaborative Interactive Machine Teaching in Image Classification. ACM IUI, 2024

Abstract: While human-centered approaches to machine learning explore various human roles within the interaction loop, the notion of Interactive Machine Teaching (IMT) emerged with a focus on leveraging the teaching skills of humans as a teacher to build machine learning systems. However, most systems and studies are devoted to single users. In this article, we study collaborative interactive machine teaching in the context of image classification to analyze how people can structure the teaching process collectively and to understand their experience. Our contributions are threefold. First, we developed a web application called TeachTOK that enables groups of users to curate data and train a model together incrementally. Second, we conducted a study in which ten participants were divided into three teams that competed to build an image classifier in nine days. Qualitative results of participants’ discussions in focus groups reveal the emergence of collaboration patterns in the machine teaching task, how collaboration helps revise teaching strategies and participants’ reflections on their interaction with the TeachTOK application. From these findings we provide implications for the design of more interactive, collaborative and participatory machine learning-based systems.

We are thrilled to announce that the HCI Sorbonne group will present 2 posters, 5 demos and 3 papers at IHM2024.

Papers

Étude de l’impact des sensibilités sensorielles visuelles et haptiques sur la détection d’illusions visuo-haptiques en réalité virtuelle. Flavien Lebrun, Gilles Bailly, Sinan Haliyo, Malika Auvray, David Gueorguiev.

Effets des aptitudes visuo-spatiales sur l’apprentissage par la réalité virtuelle. Eya Jaafar, Philippe Gauthier, Geoffroy Canlorbe, Ignacio Avellino.  https://hal.science/hal-04454386v1

Comprendre et générer des distributions de commandes réalistes. Julien Gori, Mohamed Ali Ben Amara, Gilles Bailly. https://hal.science/IHM-2024/hal-04451461

Posters (Late Breaking Work)

Understanding remote endoscope control in surgical telementoring. Océane Lelièvre.

Using live tags for summarizing surgical videos collaboratively. Alexandre Haddad.

Demos (https://hal.science/hal-04500046v1)

Surgical Cockpit. Océane Lelièvre.

Co-creation of musical interfaces for children with Autistic Spectrum Disorder (ASD). Théo Jourdan, Baptiste Caramiaux.

Boîte à outils pour les illusions visuo-haptiques. Benoit Geslain, Bruno Jartoux, Flavien Lebrun et Gilles Bailly.

TeachTOK: Système d’Apprentissage Automatique Interactif et Collaboratif pour d’un Classifieur d’Images. Behnoosh Mohammadzadeh, Jules Françoise, Michèle Gouiffès, Baptiste Caramiaux.

Peroscope. Alexandre Haddad.

 

We have the pleasure to announce that Eya Jaafar has received the 2023 award for innovation in pedagogy from the French Academy of Surgery – Prix de l’Innovation Pédagogique 2023 de l’Academie de Chirurgie.

Congratulations!

https://www.academie-chirurgie.fr/les-laureats

Who: Cedric Honnet (MIT Media Lab)
When: May 25, 11:00AM
Where: ISIR Lab, H20
Title: Sensitive Human-Computer Interactions
Abstract: This talk will be about 3 open source sensing devices and wearable projects:

– HIVE Tracker: a tiny, low-cost, and scalable device for sub-millimetric 3D positioning
– PolySense: Augmenting Textiles with Electrical Functionality using In-Situ Polymerization
– MetaSense: Integrating Sensing Capabilities into Mechanical Metamaterial.
If there is interest, the talk will conclude with an ongoing project, merging the ideas explored above.
Bio: With a background in embedded systems, he has been exploring the connections between physical computing, interactivity and the arts by traveling the world of research labs and hackerspaces. He worked as a firmware engineer and “InterHacktivist” in the Silicon Valley, co-founded a couple of companies developing tangible interfaces, and created interactive systems/installations worldwide. He has worked on eTextile music controllers, augmented immersive systems, interactive art pieces, modular implants, 3D positioning systems and many other Open Source projects, most of them being documented here: http://honnet.eu

In France, the Habilitation is a step in the research career of junior/intermediate researchers. It gives candidates the right to direct a doctoral thesis independently (instead of co-directions only). It also allows them to apply for higher-rank positions such as full professor or research director positions.

On May 15th, 2023, Baptiste Caramiaux has defended his Habilitation, covering ten years of research and supervision. The Habilitation is called “Machine Learning in Interaction: Tool, Material, Culture“.

The defence has been recorded and can be watched on YouTube: https://youtube.com/live/O9wfzKIa8ic

Summary
In this habilitation, I emphasise the importance of a perspective on machine learning (and artificial intelligence) technologies situated in interactions with people and contexts of development and use. Through my research over the past ten years, in collaboration with my students, post-docs, and colleagues, I articulate three perspectives: machine learning as a tool, material, and culture. First, I show that machine learning can be seen as a tool acting on objects situated in their context. This view has several limitations, which are highlighted in creative applications. Secondly, I show that machine learning has materiality, useful for system design. This materiality makes the technology dynamic and expressive, highlighting its cultural dimension developed in the third part. As a culture, I show that this technology is often communicated in a normative way, reducing the possibilities of discourse and interaction. And I propose examples of alternative discourses on technology from artistic works. In conclusion, I discuss the interrelationships between these three perspectives and how they can be linked in a human-computer interaction research programme.

Jury
– Susanne Bødker, Professor, Aarhus University (Reviewer)
– Elisa Giaccardi, Professor, Delft University of Technology (Reviewer)
– Myriam Lewkowicz, Professor, Université de Troyes (Reviewer)
– Alex Taylor, Reader, City University of London (Examiner)
– Rebecca Fiebrink, Professor, University of the Arts London (Examiner)
– Jean-Daniel Fekete, Director of Research, Inria (Examiner)

website of the PEPR ENSEMBLE: http://pepr-ensemble.fr
  1. Announcement of the priority themes  – 27 March 2023
  2. Submission of Ph.D. projects – 11 April, 2023
  3. Pre-selection of candidates – 12 May, 2023
  4. Candidates interviews – 1-2 June, 2023
  5. Start of the Ph.D. program – October – December 2023

The HCI Sorbonne group have presenteed three papers at the French conference in Human-Computer Interacion (IHM’23 https://ihm2023.afihm.org):

  • Sungeelee, Loriette, Sigaud, Caramiaux. Co-Apprentissage Humain-Machine: Cas d’Étude en Acquisition de Compétences Motrices. IHM’23, article
    https://hal.science/IHM-2023/hal-04014981v1
  • Ferrier-Barbut, Avellino, Vitrani, Canlorbe. Les Visiocasques dans la Formation et la Planification Chirurgicales : Une Revue de la Littérature. IHM’23, article
    https://hal.science/IHM-2023/hal-04019531v1
  • Rigaud, Bailly, Jansen. Ressources de connaissances dans les ateliers de fabrication: objectifs et défis. IHM’23, article
    https://hal.science/IHM-2023/hal-04014973v1

And we demoed Marcelle, an open-source toolkit for building interfaces and applications with machine learning:

  • Françoise, Caramiaux. Marcelle : un toolkit pour la conception d’interactions humain-apprentissage automatique. IHM’23, démo
    https://hal.science/IHM-2023/hal-04043369v1

Théo Jourdan and Baptiste Caramiaux present their two papers at NIME (New Interfaces for Musical Expression) 2023 respectively called:

– “Machine Learning for Musical Expression: A Systematic Literature Review”

– “Culture and Politics of Machine Learning in NIME: A Preliminary Qualitative Inquiry”

Papers publicly available soon.

Who: Valentin Bauer (Université Paris-Saclay)
Where: Room H20 ISIR
When: February 9th, 11:00AM
Title: Exploring Sensory and Extended Reality Mediation Approaches with Autistic Children: Example of Magic Bubbles
Bio: Valentin is a PhD candidate at Paris-Saclay University, CNRS, LISN, VENISE team, working around Extended Reality, Spatial Audio, Autism, and Human-Computer Interaction. In 2017, he graduated from the Advanced Music Production Program (FSMS) at Conservatoire National Supérieur de Musique et de Danse de Paris (CNSMDP), after having developed solid skills in sound engineering, music, and science for audio. In particular, he worked on spatialized music production during my Master’s internship at Radio France Quality and Innovation Department. After that, he worked as a music consultant for Toulouse Capitole National Orchestra, and as an audio engineer on several projects. He then engaged in the Media and Arts Technology Master Programme (MAT) at Queen Mary University of London (2018/2019). There, he digged more into computer sciences, and especially into Audio Augmented Reality during my second Master’s intership at BBC R&D. In 2019, started my PhD entitled “Exploring Multisensory Extended Reality Approaches for Autistic Children: Improve Well-Being and Assess Auditory Perception”.

Flavien Lebrun, supervised by Gilles Bailly and Sinan Haliyo, will defend his thesis entitled “Study of Visuo-Haptic Illusions in Virtual Reality: Understanding and Predicting Illusion Detection” on December 14, 2022.