Sophie Ostmeier

I am a PhD student in the SPY Lab, supported by the ETH AI Center. I completed a master's degree in CS at Stanford University, where I researched at the AIMI Center. My research was generously supported by the German Research Foundation (DFG) (2023-2025). I completed an M.D. and Dr.med. (Prof. Daniela Pfeiffer) at the Technical University of Munich.

Outside the lab you will find me running, cycling, bikepacking, swimming, playing pickleball, or hiking. I follow a wide range of sports, especially tennis, triathlon, and track and field, but I can appreciate almost any activity with movement.

Contact  /  Google Scholar  /  Github  /  LinkedIn

Research

My research focuses on how machine-learning models learn, how they make predictions, and how we measure that, with a current emphasis on safety and security aspects. I am also curious about a path toward reliable intelligent decision-making systems and wonder how they would practice medicine.

Some recent papers (Google Scholar)

We investigated if a method can estimate answer correctness with access only to the attention-based model, the prompt, and its answer. Our method, HeadEntropy, uses a simple statistic that measures how sensitive each attention pattern would be to gradient updates. With no training, HeadEntropy reaches 0.74 AUROC over 5 models and 5 benchmarks, beats all 6 training-free baselines, and matches a trained hidden-state probe out-of-domain.

We investigate how to increase a transformer's capacity to learn positional information for n-dimensional inputs. We generalized rotary position encodings (RoPE) from fixed 2D rotation blocks to learned, high-dimensional rotation matrices by using their Lie group structure, and we tested this approach on 2D and 3D vision tasks.

We investigate how we can use the skills of a language model to evaluate the difference between a model generated and the human generated radiology report. We report present GREEN, an open-source metric that employs language models to spot and explain clinically significant errors in radiology reports, providing interpretable feedback, and commercial-grade performance.

We investigate evaluation metrics for medical image segmentation that account for uncertainty, small structures, and empty reference annotations. We find that model performance generalizes better to clinical practice when evaluated by the Surface Dice at Tolerance. We further proposed to set a volume threshold for very small volumes or empty reference masks and then use image-level classification metrics such as AUC

Miscellaneous

Website Template source code.