Problem

“Every single class period every single day, ask a question and get crickets... One day, in my second period, 23 students turned their cameras on and I literally cried in front of them. It brought so much joy, just to see their faces but that's rare like that happened once.” — Cynthia, 8th Grade Science and Biology

Student Engagement is Low

Missing Visual and Auditory Feedback

Re-thinking the Education System

Remote learning environments have magnified the problem of students' disengagement for a variety of reasons including students feeling more self-consciousness, lack of structure, and growth mindset.

Many teachers have reported that they are facing great challenges to engage and evaluate students effectively in remote and blended learning environments because of missing multi-sensory levels of communication beyond a webcam.

Though many problems have risen after the switch to online instruction, remote learning has turned out to be a good option for students with diverse and special needs and offers more adaptability and flexibility.

How Might We...

Help 6th to 9th-grade teachers assess and encourage student engagement in remote and blended learning environments?

Dory helps middle school teachers...

Evaluate students' engagement

Teachers can quickly evaluate students' engagement by understanding how students are feeling (via emoji reactions) at the moment. This visual feedback gives teachers information that they can adapt on the fly.

Facilitate conversation or class activities

regarding the feeling of the class by viewing the shared mood space where students' moods are displayed in a circular layout.

Guide students build a growth mindset

by asking students to think retrospectively and deeply about their learning journey through a self-reflection activity 

Have a holistic view of

how students are doing and identify any possible pattern in a list format

Build stronger relationships over time

by clicking into an individual student's page to view more detailed reflection and leave notes

Research

Stepped into ambiguity, we gained a deeper and holistic understanding of the problem space through a rigorous research process.

We believe in the power of education and even more so with technology. But with all the problems with educational technology and remote format reported due to the sudden COVID-19 pandemic outbreak, we perceived it as a wake-up call that it might be time to re-think the complex nexus of technology and education. Our research has revealed reasons that contribute to students’ low engagement and we see an opportunity for leveraging the online platform and technology to better support teachers.

This research report outlines our research processes which include literature review, SME interviews, field observations, follow-up interviews, contextual inquiries, and competitive analysis, our salient insights, and principles that will guide us to the design phase.

Subject Matter Expert (SME) Interviews

• S1 - Professor at UW Information school
• Andy (S2) - Teacher Educator at School of Education at UPenn
• S5 - Visiting Researcher at MIT for Remote Learning Education
• S4 - Assistant Principal for Seattle Public Schools
• Randy (S3) - Assistant Director for Learning Tech at UW Information School
• Mike (S6) - AI Software Innovation Manager at Intel Corporation
• Dana (S7) - Founder of Leo AR EDU

Participants: 7 subject matter experts in the field of education including one university professor, two education consultants/researchers, one school administrator, and three educational technologists.

Duration: 45 mins - 60 mins via Zoom

Goal: Get an in-depth understanding of the problem space by obtaining multiple perspectives such as the administrative or educator's perspective regarding the challenges of the current educational system posed by the pandemic and the role technology can play in future education. 

Insight 01. 

Teachers struggled to identify and understand students’ unique interests and needs in this environment, impeding their ability to establish and grow unique student bonds, bonds that could encourage engagement.

“If a kid plays a sport and you're their teacher or they do some kind of activity and you could go watch them do it, they'll love you forever and they'll feel really great and they'll talk to you about it and you'll connect in that way… You also listen to them, not just what they say, but all the different things that you can get from them that they give off.” —— Andy, Educational Researcher

Insight 02. 

Creating a well-thought curriculum is a refining process that requires teachers to take into consideration the context where learning takes place, and with whom. In this process, technology serves as an empowering tool to allow teachers to have more configuration of curriculum design.

“So a lot of my prep begins first with thinking about learning objectives, thinking about pedagogy, thinking about student identity, student goals, and trying to really imagine a course that actually synchronizes and helps reconcile all of those tensions between who students see themselves as, what their aspirations are, what the purpose of this class is in their life, what kinds of knowledge I can bring to them that might help sustain their goals, or support their goals, or change their interests or knowledge or identities.” —— Amy, University Professor

Field Observation & Contextual Inquiry

Through Field Observation, we joined classes with students and...

• Understand online interaction between students and teachers
  
• Collect behavioral data through observing both teachers and students’ behaviors and workflows
• Observe how technology is used in the remote learning environment

Through Contextual Inquiry, we asked teachers to...

• Learn about teachers’ working environment and teaching tools used frequently
  
• Understand the process of how teachers prepare a remote class
• Serve as a backup plan in case we couldn’t get any opportunity to observe a classroom
  

[Fig 1] All Observation sessions took place in Google Classroom

[Fig 2a, b, c] Teachers' working station in-home (upper) and at school (lower)

Insight 03.

Remote learning environments have magnified the problem of students’ disengagement for a variety of reasons including students feeling self-consciousness and lack of structure.

"It could also be teaching eighth-graders they're very self-conscious in the online environment and nowhere near as self-conscious in person." —— Cynthia, 8th Grade Science and Biology Teacher

Insight 04.

Teachers had to use different criteria to evaluate student engagement due to the limitations imposed by online education such as missing multi-sensory levels of communication beyond a webcam.

“We're not right there. We can't walk past their desk, tap on them and get them back focused again.” —— Colleen G., 8th Grade Math Teacher

Insight 05.

Teachers can harness educational technology to create more adaptive learning environments that fulfill the needs of students who previously struggled in person. 

 I’ve noticed that for some of those kids, just being able to type and to think in and give themselves time to think about what they're writing before they hit send makes them feel safer than raising their hand and having to talk out loud in front of everybody.”  —— Abigail, 6-8 Grade Math Teacher

80 Ideas → 6 Affinitized Groups → Dot Voted to 8 Ideas → Initial Quest Idea → Pivot → Final Dory Idea

Design

Drawing inspiration from the enlightening research findings and our 5 important insights, we formed 3 design principles to serve as a guardrail that helped us achieve desired design outcomes.

Encourage Student Engagement

Our design should provide ways for teachers to evaluate student engagement that are suitable for remote and blended learning environments. (Driven by Insight 4)

Environmentally Adaptable

Our design should be adaptable to any environment, for students who are attending in person or virtually, synchronously or asynchronous, individually or as a group. (Driven by Insight 2 & 5)

Support Deeper Relationship

Our design should enable the growth of personal relationships between students and teachers. (Driven by Insight 1)

This document lays out all important aspects of Dory’s design which include architecture, models, & systems, and visual systems for the interface design to help developers build the product.

From 80 Ideas, We Downselected to 8 Ideas... 

[Fig x] We downselected ideas based on these major themes.

For Teachers

1. We imagined the solution would provide teachers with some analytics to better understand students’ performance.
2. It would allow teachers to grab the contextual information behind students’ certain behaviors.
3. It should be a powerful tool that helps teachers to facilitate in-class activity and then evaluate the engagement level. 

For Students

1. We hope that the tool can give students more agency in controlling their own learning process and be able to help them build stronger relationships with others.
2. We want to ensure students are having fun while using the tool, we want the product to be playful and fun.

[Fig 3] Dot-voted on Miro board

We Combined Ideas and Formed the Initial Quest Idea...

Quest is a classroom platform and learning progress tracking app that helps teachers and students jointly personalize learning journeys. The core concept behind this idea was to let teachers recognize that each student has his/her own learning pace and therefore, teachers should offer help and make adjustments based on that.

However, it failed to support 2 design principles

• Fail to Encourage Engagement

• Fail to Enable Relationship

By overly focusing on an individual's experience specifically on assignments, we miss the social collaboration aspect and so fail to build the emotional connection between students.

The office hour feature we proposed was not the best way to connect students with teachers because it might potentially add an emotional burden on students because they might think they do something wrong.

We Re-considered and Made the Pivot...

“How can the solution support the entire classroom experience and individual learning journey?”
— Ryan

“Not just about fixing assignments, how can we manage relationships between students and teachers beyond using traditional assessments?”
— Scott

Final Design Idea - Dory

We believe that asking students to do self-reflection would allow them to think retrospectively and have a better understanding of themselves and allowing teachers to get access to their reflection provides a more contextual understanding of what is going on in students’ life.

We also leverage the RULER method, which is a systematic approach to help people better unpack and express their emotions. 

Dory is able to support all design principles

• Able to Encourage Engagement

• Able to Adapt to Environments

• Able to Enable Relationship

By entering the platform, the Mood Space, which is a shared common space where both teachers and students can get access to and interact with it either during or after the class.

By having the flexibility of allowing teachers to determine kinds of self-reflection activities (both in-class or homework) based on their teaching plan or pedagogy. 

By having the platform for students to safely share their feelings and allowing teachers to give feedback such as encouragement and/or guidance to the students.  

Architecture and User Journey Map

I want to put more emphasis on mapping out the entire user experience not just the signature experience before stepping into a robust prototyping process. So we created both the swimlane diagram and user journey maps, illustrating key moments and touchpoints.

[Fig 4] Dory Swimlane Diagram

[Fig 5] Teacher Journey Map 

[Fig 6] Students Journey Map 

Wireframes → Low-Fi Prototype → Usability Tests → Iteration → Dory High-Fi Prototype → Dory Video

Prototyping

From wireframe to high fidelity interactive prototype, I took the initiative, creating iterative designs and visual assets for the final deliverables. 

Wireframes

Low-Fi Prototype

[Fig 7] We tested the low-fi prototype with two middle school teachers. 

Usability Testing with Two Teachers

Major Tasks:

1. Create a new reflection for an in-class activity
2. Navigate to the Mood Space 
3. Get access to individual student's pages and provide feedback

Main Goals:

1. If teachers can successfully create and publish an activity? 
   
2. If teachers can easily navigate to the Mood Space?
3. If teachers notice any mood patterns from the Mood space?

User Testing Insights & Iteration

Problems:

• Teachers like the emoji orbs with feeling words because it's more informative, but patterns are hard to perceive when emojis (with words) are scattered around.
• Confused by the different sizes of the orbs

Refine:

• An informative infographic with a clearer view of the distribution of the color-coded dots in four different quadrants.
• Feeling words and counts are displayed together in a more organized way. 
  

Problems:

Teachers perceived that the color-coded tags were clickable.

Refine:

Redesign the tags with a new style and color scheme.

User Flow: Create a Reflection Prompt

User Flow: Self Reflection Activities

User Flow: Mood Space

User Flow: View Student Reflections

Next Steps

1. Expand to a multi-classroom experience so teachers can collaborate
2. Develop a dashboard page for insights and big picture trends relating to moods and reflections
3. Develop self-reflection prompts for teachers to reflect on their own teaching journey
4. More user testing with more teachers and students
5. Develop playful iterations of the student reflection flow to encourage self-motivated reflections