Case Closed: Designing Tangible-Digital Interactions Through Narrative Play
Led the coding and design of a mixed-media narrative mystery game, integrating tangible and digital interaction for small-group play, earning 88% positive feedback.
Blending Narrative, Tangible Interaction, and Digital Design
Most detective games tend to focus on either physical escape room setups or digital-only experiences, but they rarely integrate both. We recognized an opportunity to bridge this gap by creating a hybrid mystery game that merges hands-on interactions with screen-based storytelling, enhancing immersion and player engagement.
My duties as a team leader, designer, and developer
Led the narrative and puzzle design to align story progression with interaction flow
Programmed physical interactions using Arduino and p5.js
Conducted user research and usability testing across 3 player groups
Managed the project timeline and iterative development process
Final Design
A Cross-Channel Detective Experience
In Case Closed, players are tasked with solving a murder case through the hands-on exploration of real-world props, such as a mechanical safe and an oil painting that conceals hidden clues. Each prop is connected to a digital dialogue and feedback system, enriching the overall experience.
Teaser of our detective game, turn on the volume for the full experience.
Interactions
Designing Story-Driven Physical Interactions with Arduino
To keep players immersed, we designed interactions that feel purposeful, narrative-driven, and tactile. Each clue wasn’t just a puzzle, it was a moment of story progression, triggered through physical actions and digital feedback powered by Arduino.
Early on, we struggled to make the story and interactions feel cohesive. Puzzles felt disconnected from the plot, and the flow was disjointed. To solve this, we shifted to designing the narrative and interactions in parallel, ensuring every mechanic served a story purpose.
How Physical and Digital Experiences Come Together
We mapped out the full player journey to ensure every physical action was reinforced by digital feedback, and every story moment had a tangible touchpoint. From onboarding to final deduction, each interaction was carefully designed to feel cohesive, meaningful, and immersive.
To enhance immersion, we designed core interactions to be physical, giving players tactile ways to uncover clues, while using an RFID-powered dialogue system to support narrative progression through interactive interviews.
Storyboard showing how physical interactions drive story progression across the game space.
Mechanical Safe
Players rotate a physical dial to unlock a safe using auditory and LED feedback. The closer they get to the correct code, the louder the sound and brighter the lights — simulating the tension of cracking a real lock.
Players open a physical safe by rotating a dial and listening for auditory cues, with LEDs confirming each correct position.
Hidden Painting Message
Shining a flashlight at a specific angle reveals a hidden message embedded behind an oil painting. A photoresistor detects the light, triggering LEDs that illuminate the clue from behind.
We placed the painting in a dimly lit corner of the scene to enhance the atmosphere and make the hidden visual clue easier to reveal with a flashlight.
A flashlight aimed at the painting activates a hidden message, revealed through backlit LEDs when a photoresistor is triggered.
RFID Dialogue System
Each suspect’s item contains an RFID tag. When scanned, these trigger custom dialogues on screen, allowing players to uncover backstories and piece together the mystery.
Players scan objects to interview each suspect. RFID tags trigger on-screen dialogue, letting players piece together the story through interactive interviews.
User Feedback
Iterating with Players to Refine Story and Interaction
We conducted three rounds of usability testing with different player groups, including both solo and multiplayer sessions. Our goal was to evaluate clarity, immersion, and the balance between story and interaction.
Key Feedback
The onboarding felt disconnected from the rest of the experience
Players especially enjoyed the physical interactions, particularly the safe and the flashlight-triggered painting
Multiplayer groups found the experience more engaging, as collaboration allowed them to discuss theories and co-create the story in real time
What We Improved
Introduced a crime report to help players onboard and keep track of clues
During onboarding, the facilitator role-played as a police officer, guiding players into the narrative to support immersion from the start
Refined the dialogue system with clearer fonts to help players distinguish between characters
Added contextual clues throughout the scene to help players identify the right interactions more intuitively
All of the players found the experience impressive and engaging.
“That safe was so satisfying to crack, I felt like I was in a real crime show”
“We kept making up new theories while scanning the items, it was fun seeing how the story changed based on what we found.”
“At first I didn’t know where to start, but once the ‘police officer’ explained the case, I was totally in it.”
Lessons Learned
What I'd do differently next time.
Story and interaction can’t live in separate silos
When we designed puzzles and narrative together, every clue became more than just a task, it became a moment of story progression.
Test early, fail smarter
Our lo-fi prototypes weren’t pretty, but they saved us. We caught confusing moments, pacing issues, and gaps in the story, before it was too late.
Immersion exists in the small details and in the joy of unexpected surprises
From how the flashlight revealed a hidden message to how players reacted to unexpected clues, the smallest moments made the experience feel real.
This project reminded me that compelling interaction design not only guides users but also invites them to feel, explore, and co-create the experience.
Special thanks to my incredible teammate, Chenghao Wang, for bringing fun ideas to life together.
