Inclusive VR for People with Limb Disabilities

March 2018 - November 2018


Accessibility    UX Design    Virtual Reality

Background

We started this project as an Inclusive Design Challenge given by Microsoft in our UX Design class. The challenge was to design for inclusivity in a "deskless workspace". After exploring multiple options, my team decided to work on making digital art in VR accessible and used Google Tilt Brush as the starting point.



Team

Cherisha Agarwal, Joanna Yen, Pratik Jain, Simi Gu, Srishti Kush, Raksha Ravimohan



Problem

VR is inaccessible to people with limb disabilities due to the use of hand controllers.

300 million people of the world's population has some sort of limb disability and due to the use of hand controllers, most VR applications are not accessible to people with these disabilities. People who have cerebral palsy, multiple sclerosis, paralysis and someone who is missing an arm is completely excluded from utilizing the full potential of VR.






Applications like Google Tilt brush, Medium, Blocks and Quill, which require the use of hand controllers is completely inaccessible to people with limb disabilities.


How might we enable people with limited limb mobility to create art in Virtual Reality?




Process





Research

User Research

To understand the mobility issues and experience with creating art, we conducted one on one user interviews at various centers like the ADAPT community, hospitals and disability rights centers, and spoke to people who been diagnosed with cerebral palsy, multiple sclerosis and people who have limited mobility due to an injury.




Perception about VR
Most of our potential users were not familiar with VR and to introduce them to the technology, we made them watch a VR movie. The experience was very exciting to them and they wanted to learn more.



Current methods of drawing
The users mostly used hands on hands technique, a wacom tablet or a smartphone to make drawings depending on the medium they were interested in.



Level of comfort with technology
Most of the experts that we interviewed with said that voice commands and eye gestures would be an easy of interacting.


Subject Matter Expert Interviews




"Interaction using eyes can be helpful and you could explore mixed reality as well"

- Serap Yiggit Elliot, UX researcher at Google



"There is very little work done in accessibility for VR. Tobii is creating headsets with eye tracking built in."

- Todd Bryant, Professor at NYU



Solution



Prototyping

Interaction Design for 3D space

To envision the basic interaction of controlling the interface through eye tracking and voice commands, we conducted a role play where the members of the team would behave like the interface elements.






Designing the art tool

After gaining clarity on the different tasks and interactions that should be present in the tool, we proceeded to create a more detailed prototype by incorporating advanced tools and a menu consisting of other system functions as well. The idea for this interface was derived from exisiting design tools such as Sketch and Illustrator.



Interface visualized on paper
    Tools


Testing with Laser

We tested the paper prototype with both experts and potential users from ADAPT community to understand if the interface is intuitive enough for the users. We learnt that people who have no experience using art tools found it difficult to navigate and understand what the tools meant.

Laser pointer used as gaze tracker
Role playing the interface



Final Prototype

The final prototype was implemented using the Unreal Engine and combined all the learnings from the previous stages of user testing. To help the user navigate the system easily, we decided to include an AI assistant who can have conversations with the user and also provide onboarding.


Key Values

New medium of expression

The app will introduce many people into the world of VR and help them experience this new medium.

Easy to use interface

With onboarding, AI assistance and clearcut visual for tools, the interaction becomes more natural and intuitive.

Experience of translocation

The different environments within the app provide the feeling of visiting different places.


Social connect

The user will be able to connect and draw with their friends from across the world.

Therapy

The different environments and strokes are very pleasing and therapeutic.

Skill building

The app provides opportunities to create and share artwork thereby helping them build skills and generate income.


Demo and feedback



"It feels like it's uplifting me a lot!"

- User, Axis Project


"It feels amazing! I want to take this home with me."

- User, Haym Solomon Home


"We have been working on VR applications for people with spinal cord injury and they find it to be very therapeutic."

- VR expert, Mount Sinai


"The application should be able to adapt to people with different physical capabilities"

- Roland Dubois, Accessibility expert


Iteration

Neck comfort limit

Place elements in such a way that people who cannot move their head too much will also be able to use it effectively.

Resizable interface

Increase contrast in the menu to make it suitable for people with low vision and allow them to increase the size of the menu.

Automate part of the drawing

Instead of free strokes which can be jittery, automate some of the drawing by using pre-existing shapes or replicating patterns.



XR startup bootcamp

This project received a grant of $10,000 from NYC Media Lab's XR Startup Initiative. During the 12 weeks intensive sessions, our team worked on the business model and product-market fit, and conducted about 120 customer and expert interviews. We also showcased our prototype at various exhibits - NYVR Expo '18, Media Lab Summit '18, Exploring Future Reality '18, R- Lab XR Showcase, Science Fair BCG Ventures and Verizon 5G Labs.


Startup pitch for the Exploring Future Reality conference




Documentation

Medium Article
Process Booklet