Date de identificare proiect

	Cod proiect: PN-III-P4-ID-PCE-2020-0434; Numar contract: PCE29/2021
	Director proiect: Radu-Daniel Vatavu
	Finantat de UEFISCDI, Romania
	Programul de finantare: P4 - Cercetare fundamentala si de frontiera
	Perioada de implementare: Ianuarie 2021 - Decembrie 2023 (36 luni)

Rezumat

Scopul proiectului este reprezentat de conceptualizarea, proiectarea si implementare de noi dispozitive purtabile si medii inteligente pentru sustinerea abilitatilor senzorimotorii. In acest sens, introducem conceptul Realitatilor Senzorimotorii (SRs) ca un nou tip de realitate fizico-digitala, distincta de Realitatea Virtuala (VR), Augmentata (AR), Mixta (AR) si Mediata (XYR), cu suport in Realitatea Extinsa (XR), teoria controlului motor si proiectarea bazata pe abilitati. Aplicatiile SRs vizeaza persoanele cu dizabilitati motorii, dar si persoanele fara dizabilitati care pot experimenta noi abilitati senzorimotorii prin intermediul dispozitivelor purtabile, mediilor inteligente si tehnologiilor imersive XR.



Figura de mai sus ilustreaza diverse abilitati senzorimotorii: (a) atingerea precisa a unui adult tanar pe un dispozitiv purtabil de dimensiuni reduse; (b) un copil de sase ani asambland mici piese Lego; (c) o persoana cu leziuni ale maduvei spinarii folosind o telecomanda TV cu ambele maini; (d) un utilizator de scaun cu rotile sustinand o tableta pe coapsa; (e) o persoana nevazatoare deplasandu-si mainile pe un dispozitiv HMD pentru a intelege forma acestuia; si (f) un utilizator imersat intr-un mediu virtual. Realitatile Senzorimotorii furnizeaza un nou cadru conceptual pentru a caracteriza si sustine contributii stiintifice si practice privind sisteme interactive care mediaza astfel de abilitati, capitalizand pe eterogenitatea abilitatilor senzorimotorii umane.

Obiective

Obiectivul principal al acestui proiect de cercetare fundamentala este conceptualizarea Realitatilor Senzorimotorii, un nou tip de realitate fizico-digitala care permite augmentarea abilitatilor senzorimotorii ale utilizatorilor prin operarea conjugata a dispozitivelor purtabile (en.: Wearable Computing), mediilor inteligente (en.: Ambient Intelligence) si tehnologiilor imersive.

Rezultate asteptate

Principalul rezultat asteptat este reprezentat de conceptul SRs. Rezultate asociate includ:

1. Spatiu conceptual pentru SRs
2. Taxonomie a abilitatilor senzorimotorii augmentate
3. Prototipuri de dispozitive purtabile, medii inteligente si tehnici de interactiune pentru SRs
4. Seturi de date din evaluari si studii utilizator
5. Publicatii stiintifice in reviste si conferinte relevante si rapoarte anuale de cercetare

Echipa

Publicatii stiintifice

	1.	Adrian-Vasile Catana, Radu-Daniel Vatavu. (2024). Fingerhinter Takes Center Stage: User Experience Insights from Informal Encounters with a Finger-Augmentation Device. Proceedings of AIxVR '24, the 6th IEEE International Conference on Artificial Intelligence & Extended and Virtual Reality. IEEE, USA, 5 pages PDF
	2.	Radu-Daniel Vatavu. (2023). From Natural to Non-Natural Interaction: Embracing Interaction Design Beyond the Accepted Convention of Natural Proceedings of ICMI '23, the 25th ACM International Conference on Multimodal Interaction. ACM, New York, NY, USA, 684-688 PDF | DOI | ACCEPTANCE RATE: 18.8% (3/16) | ARC CORE B | WOS: 001147764700079   1st PLACE BLUE SKY PAPER AWARD
	3.	Alexandre Nevsky, Timothy Neate, Radu-Daniel Vatavu, Elena Simperl. (2023). Accessibility Research in Digital Audiovisual Media: What Has Been Achieved and What Should Be Done Next?. Proceedings of IMX '23, the ACM International Conference on Interactive Media Experiences. ACM, New York, NY, USA, 94-114 PDF | DOI | ACCEPTANCE RATE: 38.9% (14/36) | WOS: 001117075100009   HONORABLE MENTION AWARD
	4.	Radu-Daniel Vatavu. (2023). Leveraging Sensorimotor Realities for Assistive Technology Design Bridging Smart Environments and Virtual Worlds. Proceedings of PETRA '23, the 16th International Conference on Pervasive Technologies Related to Assistive Environments. ACM, New York, NY, USA, 247-253 PDF | DOI | ACCEPTANCE RATE: 51.6% (79/153) | WOS: 001124478300044
	5.	Milad Jamalzadeh, Yosra Rekik, Laurent Grisoni, Radu-Daniel Vatavu, Gualtiero Volpe, Alexandru Dancu. (2023). Effects of Moving Speed and Phone Location on Eyes-Free Gesture Input with Mobile Devices. Proceedings of INTERACT '23, the 19th IFIP International Conference on Human–Computer Interaction. Springer, Cham, 469-478 PDF | DOI | ACCEPTANCE RATE: 31.0% (18/58) | ARC CORE B
	6.	Cristian Pamparau, Ovidiu-Andrei Schipor, Alexandru Dancu, Radu-Daniel Vatavu. (2023). SAPIENS in XR: Operationalizing Interaction-Attention in Extended Reality. Virtual Reality (27). Springer, 1765–1781 PDF | DOI IF: 4.2, 5-Year IF: 5.5, Q1 quartile (JCR 2022) | WOS: 000937114600002
	7.	Adrian-Vasile Catană, Radu-Daniel Vatavu. (2023). Fingerhints: Understanding Users' Perceptions of and Preferences for On-Finger Kinesthetic Notifications. Proceedings of CHI '23, the CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, Article No. 518, 17 pages PDF | DOI | ACCEPTANCE RATE: 27.6% (879/3182) | ARC CORE A* | WOS: 001037809506001
	8.	Laura-Bianca Bilius, Ovidiu-Ciprian Ungurean, Radu-Daniel Vatavu. (2023). Understanding Wheelchair Users' Preferences for On-Body, In-Air, and On-Wheelchair Gestures. Proceedings of CHI '23, the CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, Article No. 78, 16 pages PDF | DOI | ACCEPTANCE RATE: 27.6% (879/3182) | ARC CORE A* | WOS: 001037809504027   HONORABLE MENTION AWARD
	9.	Radu-Daniel Vatavu. (2023). iFAD Gestures: Understanding Users' Gesture Input Performance with Index-Finger Augmentation Devices. Proceedings of CHI '23, the CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, Article No. 576, 17 pages PDF | DOI | ACCEPTANCE RATE: 27.6% (879/3182) | ARC CORE A* | WOS: 001037809504026   HONORABLE MENTION AWARD
	10.	Laura-Bianca Bilius, Ovidiu-Ciprian Ungurean, Radu-Daniel Vatavu. (2023). An Expressivity-Complexity Tradeoff?: User-Defined Gestures from the Wheelchair Space are Mostly Deictic. CHI EA '23, Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, Article No. 35, 8 pages PDF | DOI | ACCEPTANCE RATE: 33.8% (327/967) | ARC CORE A*
	11.	Radu-Daniel Vatavu. (2023). Vulnerabilities of Mediated Embodiment: Towards Unmasking Security and Privacy Risks of Ability-Mediating Wearables. Proceedings of WAMWB '23, the Workshop on Advances of Mobile and Wearable Biometrics, co-located with MobileHCI '23, the 25th ACM International Conference on Mobile Human-Computer Interaction 2023 (Athens, Greece). CEUR-WS vol. 3517, Article no. 6, 65-69 PDF | DOI
	12.	Radu-Daniel Vatavu. (2022). Sensorimotor Realities: Formalizing Ability-Mediating Design for Computer-Mediated Reality Environments. Proceedings of ISMAR '22, the 21st IEEE International Symposium on Mixed and Augmented Reality (Singapore). IEEE, USA, 685-694 PDF | DOI | ACCEPTANCE RATE: 21.1% (93/441) | ARC CORE A* | WOS:000923767800074
	13.	Radu-Daniel Vatavu, Ovidiu-Ciprian Ungurean, Laura-Bianca Bilius. (2022). Interactive Public Displays and Wheelchair Users: Between Direct, Personal and Indirect, Assisted Interaction. Proceedings of UIST '22, the 35th Annual ACM Symposium on User Interface Software and Technology (Bend, OR, USA). ACM, New York, NY, USA, Article No. 45, 17 pages PDF | DOI | ACCEPTANCE RATE: 26.3% (98/372) | ARC CORE A* | WOS:001046841800051
	14.	Ovidiu-Ciprian Ungurean, Radu-Daniel Vatavu. (2022). Ability-Centered Examination of People with Motor Impairments' Interaction with Television Towards More Accessible Smart Home Entertainment Environments. Proceedings of ISAmI '22, the 13th International Symposium on Ambient Intelligence (L'Aquila, Italy). Lecture Notes in Networks and Systems 603. Springer, Cham, 32-43 PDF | DOI | WOS:000927582100004
	15.	Cristian Pamparau, Radu-Daniel Vatavu. (2022). The User Experience of Journeys in the Realm of Augmented Reality Television. Proceedings of IMX '22, the ACM International Conference on Interactive Media Experiences (Aveiro, JB, Portugal). ACM, New York, NY, USA, 161-174 PDF | DOI | ACCEPTANCE RATE: 40.4% (19/47) | WOS:001117055700015
	16.	Radu-Daniel Vatavu. (2022). Possi(A)bilities: Augmented Reality Experiences of Possible Motor Abilities Enabled by a Video-Projected Virtual Hand. Proceedings of ISEA '22, the 27th International Symposium on Electronic Art (Barcelona, Spain). ISEA International, 825-828 PDF | DOI | ACCEPTANCE RATE: 28.0% (308/1100)
	17.	Bogdan-Florin Gheran, Santiago Villarreal-Narvaez, Radu-Daniel Vatavu, Jean Vanderdonckt. (2022). RepliGES and GEStory: Visual Tools for Systematizing and Consolidating Knowledge on User-Defined Gestures. Proceedings of AVI '22, the International Conference on Advanced Visual Interfaces (Rome, Italy). ACM, New York, NY, USA, 5:1-5:9 PDF | DOI | ACCEPTANCE RATE: 24.2% (15/62) | ARC CORE B | WOS:001051742000014
	18.	Bogdan Popoveniuc, Radu-Daniel Vatavu. (2022). Transhumanism as a Philosophical and Cultural Framework for Extended Reality Applied to Human Augmentation. Proceedings of AH '22, the 13th Augmented Human International Conference (Winnipeg, MB, Canada). ACM, New York, NY, USA, 6:1-6:8 PDF | DOI | WOS:000944005200003
	19.	Alexandru-Tudor Andrei, Alexandru-Ionut Siean, Radu-Daniel Vatavu. (2022). Tap4Light: Smart Lighting Interactions by Tapping with a Five-Finger Augmentation Device . Proceedings of AH '22, the 13th Augmented Human International Conference (Winnipeg, MB, Canada). ACM, New York, NY, USA, 4:1-4:2 PDF | DOI | WOS:000944005200008
	20.	Radu-Daniel Vatavu. (2022). Are Ambient Intelligence and Augmented Reality Two Sides of the Same Coin? Implications for Human-Computer Interaction. Proceedings of CHI '22 EA, the CHI Conference on Human Factors in Computing Systems Extended Abstracts (New Orleans, LA, USA). ACM, New York, NY, USA, Article no. 362, 8 pages PDF | DOI | ACCEPTANCE RATE: 35.7% (258/722) | ARC CORE A*
	21.	Radu-Daniel Vatavu, Jacob O. Wobbrock. (2022). Clarifying Agreement Calculations and Analysis for End-User Elicitation Studies. ACM Transactions on Computer-Human Interaction 29(1), Article No. 5, pp 1-70 PDF | DOI IF: 3.7, 5-Year IF: 4.6, Q2 quartile (JCR 2022) | WOS:000921867700005
	22.	Radu-Daniel Vatavu, Ovidiu-Andrei Schipor. (2021). Formalizing Digital Proprioception for Devices, Environments, and Users. Proceedings of ISAmI '21, the 12th International Symposium on Ambient Intelligence (Salamanca, Spain). Lecture Notes in Networks and Systems vol. 483. Springer, Cham, 1-10 PDF | DOI
	23.	Alexandru-Ionut Siean, Laura-Bianca Bilius, Radu-Daniel Vatavu. (2021). Assistive Technology in the Synchrony between Ambient Intelligence and Mixed Reality for People with Motor Disabilities. Proceedings of ISAmI '21, the 12th International Symposium on Ambient Intelligence (Salamanca, Spain). Lecture Notes in Networks and Systems vol. 483. Springer, Cham, 23-33 PDF | DOI
	24.	Irina Popovici, Radu-Daniel Vatavu, Pu Feng, Wenjun Wu. (2021). AR-TV and AR-Diànshì: Cultural Differences in Users’ Preferences for Augmented Reality Television. Proceedings of IMX '21, the ACM International Conference on Interactive Media Experiences (Virtual Event, NY, USA). ACM, New York, NY, USA, 50-60 PDF | DOI | ACCEPTANCE RATE: 42.5% (17/40) | WOS: 001117065300006   BEST PAPER AWARD
	25.	Ovidiu-Ciprian Ungurean, Radu-Daniel Vatavu. (2021). Coping, Hacking, and DIY: Reframing the Accessibility of Interactions with Television for People with Motor Impairments. Proceedings of IMX '21, the ACM International Conference on Interactive Media Experiences (Virtual Event, NY, USA). ACM, New York, NY, USA, 37-49 PDF | DOI | ACCEPTANCE RATE: 42.5% (17/40) | WOS: 001117065300005
	26.	Radu-Daniel Vatavu. (2021). Accessibility of Interactive Television and Media Experiences: Users with Disabilities Have Been Little Voiced at IMX and TVX. Proceedings of IMX '21, the ACM International Conference on Interactive Media Experiences (Virtual Event, NY, USA). ACM, New York, NY, USA, 218-222 PDF | DOI | WOS: 001117065300024

Alte publicatii stiintifice: Conferinte

	27.	Adrian-Vasile Catana, Cristian Pamparau, Radu-Daniel Vatavu. (2022). Using Mediated Reality to Induce Sensorimotor Impairments. ASSETS '22 Workshop on Multidisciplinary Perspectives on Designing Accessible Systems for Users with Multiple Impairments: Grand Challenges and Opportunities for Future Research (Athens, Greece/virtual). PDF
	28.	Radu-Daniel Vatavu. (2022). Designing Interactive Computer Systems within the Framework of Sensorimotor Realities. Invited Paper at ECCO '22, the International Conference on Electronics, Communications, and Computing (Hybrid Event, Chisinau, Republic of Moldova). PDF | DOI
	29.	Radu-Daniel Vatavu. (2022). Designing Interactive Experiences in the Interplay between Ambient Intelligence and Mixed Reality Proceedings of CHI '22 Extended Abstracts, the CHI Conference on Human Factors in Computing Systems Extended Abstracts (New Orleans, LA, USA). ACM, New York, NY, USA, Article no. 140, 1-3 PDF | DOI | ARC CORE A*
	30.	David Geerts, Radu-Daniel Vatavu, Alisa Burova, Vinoba Vinayagamoorthy, Martez Mott, Michael Crabb, Kathrin Gerling. (2021). Challenges in Designing Inclusive Immersive Technologies Proceedings of MUM '21, the 20th International Conference on Mobile and Ubiquitous Multimedia (Leuven, Belgium). ACM, New York, NY, USA, 182-185 PDF | DOI | WOS: 001112164200021 | ARC CORE B

Premii

Media

Rapoarte anuale

1. Raport stiintific final (2021-2023) PDF
2. Raport stiintific 2022 PDF
3. Raport stiintific 2021 PDF

Alte resurse (software, diagrame, seturi de date)

1. Harta conceptuala pentru Realitatile Senzorimotorii: PDF
   
   Copyright © 2021 Radu-Daniel Vatavu.
2. Spatiu conceptual pentru Realitatile Senzorimotorii: v1 PDF | v2 PDF
   
   Copyright © 2021-2022 Radu-Daniel Vatavu.
3. Diagrama Venn pentru pozitionarea Realitatilor Senzorimotorii fata de alte tipuri de realitati: PDF
   Copyright © 2021 Radu-Daniel Vatavu.
4. Taxonomia de gesturi iFAD pentru dispozitive destinate augmentarii degetelor (en.: index-finger augmentation devices) PDF
   Copyright © 2023 Radu-Daniel Vatavu.
5. Codul sursa pentru aplicatia HoloLens ARTV (ISMAR '22): GitLab
   Aplicatie dezvoltata de si copyright (2022) Cristian Pamparau.
6. R software care implementeaza teste statistice pentru analiza datelor provenite din studii de tip solicitare a preferintelor utilizatorilor (en.: end-user elicitation studies). Mai multe detalii sunt disponibile in lucrarea publicata in revista ACM Transactions on Computer-Human Interaction . Aceste resurse software sunt disponibile sub licenta New BSD .
   
7. SAPIENS-in-XR, o arhitectura software destinata proiectarii si implementarii de sisteme interactive pentru interactiune periferica in medii de realitate extinsa (XR). O aplicatie software web pentru simularea interactiunilor periferice in XR, care permite acces la codul JavaScript si mesajele JSON transmise intre diversele componente. Mai multe detalii sunt disponibile pe pagina web dedicata SAPIENS-in-XR si in lucrarea publicata in revista Virtual Reality .
   Aplicatia software este disponibila sub aceasta licenta .
8. Setul de date Fingerhints dataset & R software constand in 210 notificari 'fingerhints' propuse de catre 21 de participanti pentru diverse tipuri de continut digital, e.g., mesaje text, social media, stiri, etc. Mai multe detalii sunt disponibile in lucrarea publicata in cadrul CHI 2023 .
   
   Setul de date si aplicatia software sunt disponibile sub aceasta licenta .
9. Setul de date iFAD dataset & C# software constand in 6,369 de gesturi colectate de la un numar de 20 de participanti folosind un dispozitiv de augmentare a degetului (en.: index-finger augmentation device). Mai multe detalii sunt disponibile in lucrarea publicata in cadrul CHI 2023 .
   
   Setul de date si aplicatia software sunt disponibile sub aceasta licenta .
10. Setul de date on-body, on-wheelchair dataset & R software constand in 231 de gesturi colectate de la un numar de 11 de participanti, utilizatori de scaun cu rotile privind executarea unui numar de 21 de comenzi frecvente pentru sisteme interactive. Mai multe detalii sunt disponibile in lucrarea publicata in cadrul CHI 2023 .
    
    Setul de date si aplicatia software sunt disponibile sub aceasta licenta .