Multimodal Health Sensing
Designing systems to support people in reconciling complex multimodal health data with their mental models
Connecting the Dots in Health Data
Most health apps treat the body as a set of disconnected parts. But health is interconnected and people want tools that help them understand how signals like sleep, mood, hormones, and energy relate over time.
This three-phase research and design project used menstrual tracking as a lens to explore how consumer health tools can better support holistic, goal-driven reflection.
Current Prototype Demo Video
The Problem
People want to understand their health, not just track it.
Many people turn to health tracking tools in the hopes of understanding their bodies and achieving personal health goals. Yet most apps treat health as fragmented focusing narrowly on one domain at a time, like exercise, diet, or hormones.
Health doesn't happen in silos
This siloed design makes it hard for users to grasp the bigger picture, even though health reflects a complex interplay of hormonal, metabolic, emotional, and behavioral factors.
Current tools oversimplify complexity
Today's tools reduce that complexity to calendar dots and symptom logs leaving users to make sense of patterns on their own.
What I Did
End-to-End Research
Led UX research, design, app development, deployment, and evaluation across 3 project phases
Multimodal Data Study
Studied how users track health signals with wearables (Fitbit, Dexcom, Mira)
Interface Design
Designed and tested new interfaces to support exploration, comparison, and pattern recognition
Prototype Evaluation
Evaluated a custom prototype over 100 days to assess impact on health understanding
Choosing Menstrual Health as a Focus Domain
Menstrual cycles are complex, cyclical, and deeply personal making them a powerful lens for studying health sensemaking. Most period apps focus on predicting dates or logging symptoms but fail to help users draw connections between hormones and overall health.
Varied User Goals
- Avoiding or achieving pregnancy
- Preparing for days of menstruation
- Managing symptoms such as chronic pain, mood changes, or fatigue
- Determine health status and diagnosing hormone conditions
- Learning how hormonal shifts impact wellbeing such as energy, sleep, stress, or focus
- Increase body and health awareness
Research Advantages
Cyclical, semi-perdictable fluctuations provide checkpoints to observe tracking behaviors
Goals can vary dramatatically overtime within and between individuals
Spans puberty, adulthood, pregnancy, postpartum, perimenopause, menopause
The Project
I established and led a design initiative across three phases:
Contextual Inquiry: Real-World Multimodal Tracking
Understanding current practices and pain points in cross-signal health reasoning
Research Goal
How do people engage with multimodal health tracking tools in everyday settings, especially when those tools offer unfamiliar or unconventional signals?
- How and why users form questions and insights from varied data streams (hormone, glucose, sleep)
- Where they experience breakdowns in cross-signal reasoning, reflection, or health literacy
Methodology
3-month contextual inquiry with 50 menstruating participants using commercial tracking devices in their daily routines
50 menstruating individuals (ages 18-29, 8 ethnicities, diverse health tracking experience)
Tools & Data Streams
| Tool | Data Collected | Rationale |
|---|---|---|
| Mira | Estrogen (E3G), LH | Hormonal fluctuations |
| Fitbit Sense | Sleep, heart rate, stress | Activity + recovery |
| Dexcom G6 | Continuous glucose monitoring | Metabolic signals |
| Custom App | Mood, flow, pain, notes | Context + symptoms |
Participants used native device apps, not a unified dashboard. This preserved real-world friction and revealed how users naturally reason across siloed interfaces. This way I can surface unmet needs in current tools without priming toward specific insights.
Data Collection
Open Coding: Three researchers independently coded early transcripts line-by-line. Codes captured emergent ideas like "tracking beyond menstruation," "frustration with unexplained patterns," or "reliance on hormonal spikes."
Axial + Thematic Analysis: Grouped codes into mid-level categories and broader cross-cutting themes. Focused on shifts over time, contradictions between goals and behaviors, and interface design impact on interpretation.
Key Insights
Motivations: Anticipate symptoms, explore patterns over time, monitor long-term health changes, understand menstrual-wellness connections
Methods:
- Tracking beyond menstruation days, especially around ovulation and hormonal peaks
- Toggling between apps (Dexcom ↔ Fitbit ↔ Mira) to align and compare signals
- Revisiting historical entries to verify recurring patterns or unexpected symptoms
- Combining structured tracking with personal journaling when native apps lacked flexibility
- Fragmented apps, fragmented thinking: Switching between tools disrupted reasoning and increased cognitive load
- "Data fatigue" limited exploration: Cognitive effort managing different signals caused users to default to familiar data types
- Cross-signal reasoning rarely emerged: Tools lacked support for integrating streams, leaving users to "connect the dots" alone
Design & Prototyping: From Siloed Signals to Insightful Interfaces
Translating user needs into functional design requirements through interactive prototyping
Research Goal
How might we design interfaces that help people explore, understand, and act on multimodal health data in ways that feel insightful?
- Derive clear design requirements to guide interface development for integrated, exploratory health tools
- Test early concepts with users to validate design direction
🧪 Methodology
Mid-fidelity Figma prototypes enabling interaction and concept exploration. Live iteration during interviews allowed participants to co-shape designs in real time.
30 menstruating individuals with 3–5 years average tracking experience
Live co-editing of Figma files during interviews enabled real-time design iteration. Participants actively shaped interfaces through feedback rather than just focusing on evaluating the presented features.
Design Concepts
Concept: Human-avatar interface visualizing symptoms and health signals on a body figure over a linear timeline.
- Linear calendar view shows past/future phases
- Body-based icons map symptoms to relatable areas
- Phase-specific stats for each symptom
- Cycle comparison with historic + peer data
- Signal plotting with stackable, independent y-axes
Concept: Node-link graph visualization presenting health signals as connected nodes orbiting around a cycle hub.
- Circular timeline reinforces cyclical nature
- Signal nodes with relationship highlights
- Drag-and-drop signal comparison in unified chart
- Summary popups with min-max/avg values
- Support for peer data & aggregate views
Key Findings → 5 Functional Design Requirements
Build, Deploy, and Evaluate: Testing a Multimodal Health Tracker
Longitudinal field study of a working prototype integrating multiple health data streams
Research Goal
Building on Phase 2's functional requirements, I developed a working prototype integrating menstrual, physiological, and subjective signals into a single interface.
- How people interpret and reflect on interconnected signals over time
- Whether integrated tracking fosters insight, confidence, or lasting behavioral change
- How tools can reduce cognitive load of long-term health engagement
Methodology
100-day longitudinal field study with mixed-method evaluation focusing on lived engagement, motivation, reasoning, and post-tracking impact
20 individuals (diverse in gender, race, goals) with prior health tracking experience
Data Collection Framework
| Data Source | Details | Purpose |
|---|---|---|
| Custom App Logs | Timestamped usage: time-on-app, screens viewed, comparison features used | Behavioral patterns |
| Fitbit Integration | Sleep, heart rate, activity data | Physiological signals |
| Mira Integration | Hormone levels (E3G, LH) | Cyclical patterns |
| Biweekly Reflections | Open-ended check-ins via app | Insight formation |
| Surveys and Interviews | Pre/Post/Follow-up surveys and 60 minute semi-structured sessions | Tracking goals and reflection |
Analysis Approach
Usage modeling: Average session duration, signal comparison frequency over time
Feature mapping: Connected feature use to participant goals and reflection content
Open coding: Captured cognitive strategies, emotional states, behavioral shifts
Thematic triangulation: Compared app usage traces with interview narratives and journaled reflections
Key Findings
Participants moved from "logging" to connecting and seeing how physiological signals and hormones influence one another across the cycle.
- Viewing all signals in one place helped surface relationships
- Users broke out of single-device thinking (Fitbit vs. Mira)
- Overlaying abstract data (hormones) with felt experiences increased trust
Usage patterns revealed a potential learning curve as users developed internal health models.
- Early weeks: High engagement (~19.3 min/day) during pattern discovery
- Mid-study: Sharp decline (~46-51 sec/day) as patterns internalized
- Late resurgence: Return to active use (~7.6 min/day) for new insights
Many participants retained insights and applied them beyond the study period.
- Recognized recurring patterns (e.g., energy dips post-ovulation)
- Changed planning for exercise, work, and rest based on insights
- Several stopped daily tracking but maintained pattern awareness
- Some returned periodically to confirm or explore new patterns
Technical Implementation
Mobile App: React Native for cross-platform deployment
Data Integration: Real-time sync with Fitbit Web API, Mira Cloud API
Analytics: Custom event tracking for detailed usage logging
Backend: Node.js with PostgreSQL for participant data management
Built data normalization layer to handle different device sampling rates and data formats. This enabled seamless cross-signal comparison despite varying temporal resolutions (e.g., continuous glucose vs. daily hormone readings).
Reflection
This research demonstrates the potential of multimodal tracking technologies to foster more comprehensive understanding of health. I identified new opportunities for designing personalized, adaptive tools that help individuals:
