Nudge Pet is a mixed reality widget that helps you stay aware of your mental and emotional state in instances where you are vulnerable.
2024
Overview
This project was created at the MIT Reality Hack 2025 hackathon, where our team was deeply inspired by one of our teammate’s emotional support animal (ESA). Really, we simply wanted to have fun at this hack and build something we could all be proud of, and we did just that!
Because ESAs are not accessible to everyone, we wanted to create a virtual version that could provide the same benefits. We used the MetaQuest 3 to create a mixed-reality experience that dynamically responds to your stress levels that are detected through OpenBCI’s EmotiBit.
Problem
43% of all behaviors are habitual, and these habits are formed by a cue, a routine, and a reward. In moments of stress or anxiety, it can be difficult to break these habits and form new ones. In these vulnerable moments, it’s easy to mindlessly fall back into old habits that are detrimental to our mental health.
Solution
Nudge Pet is a mixed reality widget that helps you stay aware of your mental state in instances where you are vulnerable. By providing a virtual ESA that dynamically responds to your stress levels, we aim to help you stay aware of your mental state and nudge you towards healthier habits.
Research
Our research for NudgePet focused on two key areas: the psychological benefits of human-animal interaction and the use of physiological signals to detect emotional states.
Human-Animal interaction
Studies have shown that animals, particularly pets, can provide significant emotional support and stress relief for humans. This is partly due to the perception of animals as non-judgmental entities, which allows people to confide in them more easily. The unconditional acceptance provided by animals can be beneficial when human relationships are affected by mental illness. Animals can act as “social catalysts”, helping pet owners form more friendships and social connections.
In the context of virtual pets, studies have demonstrated that interaction with computer-simulated animals can promote empathy and positive attitudes in children. Virtual pets have also been used successfully to encourage healthy behaviors and provide comfort in therapeutic settings.
Physiological signals
To accurately assess users’ emotional states, we focused on three key physiological signals:
Electrodermal Activity (EDA): EDA has been shown to be one of the most effective measures of anxiety, emotional status, cognitive status, and physiological activation. It measures changes in the electrical properties of the skin related to sweat gland activity.
Photoplethysmography (PPG): PPG can provide information on heart rate, respiration, oxygen levels, and hydration. While not as precise as ECG for heart rate variability (HRV) measurements, it offers a good balance between accuracy and convenience for wearable applications.
Body Temperature: Measured using a thermistor, body temperature can provide additional context for emotional state assessment.
Research has shown that combining these physiological signals can provide a comprehensive picture of an individual’s stress levels and emotional state. By integrating these research findings into NudgePet, we aimed to create a virtual companion that could effectively detect users’ emotional states and provide appropriate support, leveraging the psychological benefits of human-animal interaction in a digital format.
Process
As the team formed and started discussing ideas, I naturally fell into the role of a product manager. I was responsible for setting the vision, defining the scope, and ensuring that the team was aligned on the goals and objectives. Because I was also the only one with experience in 3D modeling and spatial design, I took on the additional role of creating the 3D assets and smoothly integrating them within Unity.
We went through the whole design strategy process, going from divergent to convergent ideation. (I love to use “what if?” statements. The key is to say them out loud—it’s meant to foster conversation.)
I had to work with a horrible pre-made rig to tweak animations for my custom model of Snuffles. While it was extremely tedious and made me question whether it really was worth spending time on this, it really did bring in that level of polish needed to place us in the top 10.
Big state machine for Unity animations! I also had to make many tedious tweaks here…
Pet interactions shown on the left, and immersive meditative environment on the right for extremely high stress situations.
Props to my team for getting the hardware to work so smoothly! We all expected much more blockers from working with the hardware.
Next steps
We plan to refine our emotion detection algorithms to enhance accuracy and expand the range of pet behaviors and interventions to provide more tailored support. Implementing AI will allow the pet to learn and adapt to individual users over time, creating a more personalized experience. We aim to explore partnerships with mental health professionals to integrate Nudge Pet into therapeutic practices, potentially broadening its impact in supporting emotional well-being.
Reflections
Once again, it’s all about the people (I literally say this every time but it’s genuinely my biggest takeaway)
This was all of our first times working with hardware and doing at hardware hack, yet we somehow managed to pull through and create a project that made it to the top 10 finalists. We were all incredibly proud of what we had accomplished in just 2.5 days. Last year, I didn’t know what to expect and was just excited to be there and to learn from the experience. Since then, I think I have matured a little more as a professional and really wanted to just have fun with this hack. That’s probably what made it such a life-changing experience for me while also being so successful for our team.
Special thanks to…
All of the friends I have made at ICXR and others in this community. I love you all.
If you want to hear more about my personal journey, check out my journal entry!
