Selected research projects in computational biology and human-computer interaction.
Research
My research sits at the intersection of computational biology and human-computer interaction. I'm interested in multimodal representation learning for biomedical data—integrating imaging, omics, and clinical modalities for retrieval, fusion, and natural-language-driven exploration—and, earlier on, in how novel sensor and material design can elicit new kinds of tangible interaction. Below are a few projects from this work; see publications for the full list of papers, posters, and talks, and CV for the complete research timeline.
LLM-Supported Exploration of Highly Multiplexed Imaging
Description: Multiplexed immunofluorescence (mIF) imaging enables visualization of a large number of biomarkers simultaneously, offering high-resolution insight into spatial tissue architecture and immune landscapes. Current visualization tools make it hard for non-expert users to interpret the biological relevance of diverse biomarkers, and the volume of data makes it difficult to locate regions containing patterns of interest within the tissue. To address this, I built a natural-language-driven multi-agent system that interprets user queries, retrieves relevant biomarker information, and identifies spatial regions relevant to the query within the imaging data. By combining SQL-based querying, dynamic tool execution, and contextual reasoning, the system lets users explore biomarker relationships, highlight spatially distinct regions, and dynamically update views from plain-language input. The implementation builds on Vitessce, a visual integration tool for spatial and single-cell datasets, and can be deployed as an interactive widget inside a Jupyter notebook. We demonstrate the approach on multiple mIF datasets spanning diverse tissue and cancer types.
Contributions:
- Designed and implemented the chatbot interface for Vitessce.
- Built a multi-agent orchestration layer for biomarker and region-of-interest (ROI) search.
- Implemented few-shot learning and a RAG approach for accurate retrieval and structured output.
Multimodal Pathology Image Search Between H&E Slides and Multiplexed Immunofluorescent Images
Description: An approach for multimodal pathology image search that uses dynamic time warping (DTW) on a Variational Autoencoder (VAE) latent space, fed into a ranked-choice voting scheme, to retrieve the multiplexed immunofluorescence (mIF) imaging most similar to a query H&E slide. By training the VAE and applying DTW, we align and compare mIF and H&E slides, integrating morphological H&E data with mIF immunophenotyping to give clinicians a richer perspective on disease state and support differential diagnosis and treatment decisions. The technique is demonstrated on colorectal cancer and healthy tonsil samples; an ablation study validated the search engine's ability to map the two modalities into a unified vector space, with the system remaining robust despite extreme class imbalance.
Contributions:
- Developed and trained the VAE on HPC clusters to generate patch-level latent vectors.
- Implemented the DTW-based alignment and ranked-choice retrieval pipeline.
- Conducted validation and ablation studies to assess robustness despite extreme class imbalance.
Trace to Touch: Eliciting Gestures from Capacitive Touch Electrodes
Description: Capacitive touch relies on electrodes to detect and interpret touch gestures. These electrodes are conventionally designed as rigid, grid-like structures, optimized for manufacturing efficiency. We develop a textile-silicone sensor composite using embedded conductive yarn as a capacitive touch electrode, and by deviating from the grid pattern, explore how alternative patterns can inspire novel, playful, and expressive gestures. We describe our design process for conceptualizing gestures from electrode design principles and iteratively test gesture detection with an off-the-shelf CNN model. The resulting textile-silicone sensor with unique electrode designs supports creative, comfortable, and customizable haptic interfaces.
Contributions:
- Fabricated layered silicone composites with unique electrode designs.
- Collected and processed gesture datasets for CNN-based classification.
- Achieved 97% accuracy in gesture recognition from sensor-reading heatmaps.
Sentura: Cognitive Affordances of Silicone Microtextures in Tangible UI Design
Description: We laser-cut and 3D-printed molds to fabricate silicone samples with distinct micro-textures and studied how those textures shape haptic perception, emotion, and decision-making. I built a React and Flask web application to collect participant responses and observe how users perceived the different silicone textures, and helped develop a layered silicone composite sensor with unique electrode designs to study how new tactile gestures emerge and how they correspond across modalities.
Contributions:
- Lasercut and 3D-printed molds for silicone samples with different micro-textures.
- Built a React/Flask web application to collect and analyze user texture-perception responses.