Seeme
“See” your social behavior through a non-visual mirror
The goal of this project is to create a non-visual way to help blind students enhance their social skills and be more confident in their future social activities. Project has been published in Student Concept iF Design Guide.
Oct, 2019 | 4 months project | Individual project | Beijing
#User research #Usability test
* All videos and photos were published with the permission from participants. Some pictures are processed to be blurred to protect personal privacy.
Young blind people in China facing social isolation
The living situation of the youth population who are blind is even more serious than older age groups. Due to the competitive educational system followed in China, this group of people is often left with no choice but going to the special schools with other visually impaired students.
Photo: “Passing Through Darkness” by Nathan Gibson
The living challenges that a blind juvenile would face in China
How does blindness affect social interactions?
The lack of vision makes it difficult for blind students to get visual information during conversations. I also observed some idiosyncratic behaviors displayed by that student during the interview.
From blind student participant:
“I would like to talk with sighted people, but it can be a great challenge for me …”
I spent a day with a senior student (who had lost his vision since birth) and had a short interview with him.
In the later stage of my research, I administered a conversation experience between a sighted and a blind student. A blind person will be asked to have a face-to-face conversation with a sighted person. From the experience of the sighted participants, they rarely tend to get effective information from the gestures of blind students.
On the contrary, blind students care much about their self-performance in social communications, like whether they have conveyed appropriate information to their sighted partners in the conversation.
From blind student participant:
“ I wonder whether my facial expressions look silly? ”
Conversation experiment between blind and sighted students
◉ Problem statement
How should we guide young blind people to improve their social skills?
The blind students in China have certain difficulties in conveying appropriate non-verbal messages to sighted partners in a conversation, which in turn leads to negative impacts on social interaction.
Introducing “Seeme”
The “Seeme” concept allows blind students to self-check their social behavior through a non-visual mirror.
“See” your smile
Recognizes the smile of a blind person and gives sound feedbacks in real-time to represent different smile levels.
“See” your gesture
The application is embedded with some basic gesture tutorials. It uses gesture recognition technology to guide blind people correct their gestures by voice assistance in real-time.
“See” your chat
A video chat function that allows blind people to have online conversations with sighted partners (community of volunteers).
Meanwhile, I created a side feature that the blind users can get a sound emoji after scanning their smile. They can share it as another expression of their emotions in online chatting.
Play the video below
◉ Behind the scenes
In a nutshell, this is a self-motivated and user-centered design project.
At the very beginning, I did not have a concrete design topic. However, driven by my interest, I clearly set my sights on the special group of blind students. I was close to users throughout the entire designing process. By getting along with blind students, I could empathize with their current living situation. Meanwhile, I self-planned a series of user research, and made design decisions from the insights I got from the users. Towards the end, I narrowed down my focus area on improving the social skills of the young blind population.
1. Shadowing with blind students
In the music class
In the subway station
I spent a half day to follow with a group of blind students in a special education college in Beijing.
2. What is the feeling of talking in dark?
To experience the sentiment of talking in complete darkness, I planned a blindfolded test among college students. One participant was asked to wear an eye mask and talk with another sighted person face to face. The social behaviors of the sighted people in the test were different from the blind people since blindfolded participants performed fewer body languages than before.
“I become nervous about being stared at by others.”
— Blindfolded participants
“I feel I get less visual feedback from the blindfolded partners. ”
— Sighted participants
Blindfolded dialogue test
3. How is the talk between blind students?
Later, I organized face-to-face conversations between blind people and recorded video of the process. Each dialogue lasted 5 minutes. I rewatched the video to observe the behavior of the blind people. I found some social behaviors which are generally seen among blind students during conversations, such as they are always touching something or they tend to put on an inappropriate smile.
Blind participant having the dialogue test with another blind student
4. Does the communication with the sighted/blind partner affect you?
In this stage, a blind participant was asked to have a face-to-face conversation with a sighted person.
I found blind students care much about their self-performance on social communication. Blind students care more about whether they have conveyed appropriate information to the sighted people.
Blind students’ social performance cause ineffective communication.
Blind students having the dialogue test with sighted people, Apr.2019, Beijing.
5. Why do blind people lack social skills?
Blind people lack adequate social skills because they don’t get the opportunity to receive visual feedback from their talking partners, or learn those skills by observing how others behave.
Learning process of social skills
6. Wireframes, Prototyping & User test
Sketches of the wireframe
Building arduino prototype
The purpose of usability test is to check whether the interaction flow of application is reasonable, whether the accessibility functions can meet the blind student’s requirements. In addition, I also requested participants to give me an evaluation of the overall functions in this app.
Blind participant tested the “see” your gesture function
◉What I have learned?
This is my first design project that purely focuses on user interaction, aiming to create a design solution for their problems.
# People don’t know what they want until you show it to them
While conducting user interviews, it is important to know that you can never aptly interpret the problem you can make a design for from what the users are saying. People don’t know what they want until you show it to them. Sometimes, it is also a bit convoluted to make a judgement on whether what their expectation is, is their actual need. Like when you asking people what do they want, it is not a fast horse but a faster means of transportation.
# Keep an ongoing relationship with your participants
Make your participants feel safe and comfortable to talk about their situation. Always keep in mind that the purpose is to create values for them. Keep them posted, and try to involve them throughout the process in your design project.
# There is no end to a design project
As a self critique, I also learned a design project can never come to an end, it is always in the process of iteration and development. One important point that can be noted from the feedback I received is that the blind people want to know what kind of social scenario their smile is suitable for. This could be one of the future direction that my project can be improved upon.
Reference
M. Knapp, J. Hall, and T. Horgan, Non-verbal communication in human interaction, Harcourt College Pub, 4th ed., Nov. 1996.
Zainal, Khalim (2011). Assessment of Social Skills Among Visually Impaired Students. The International Journal Of Learning. 17(12). 89–971.