Alex Gill

About.

I'm a second-year PhD student in the Kahlert School of Computing and a member of UtahNLP, advised by Prof. Ana Marasović. I study LLM agents on long-horizon, tool-using tasks: how to evaluate them, what their behavior reveals, and how the signals we train them on shape what they learn.

Benchmarks that measure what matters. KNOWS (EMNLP Findings 2026) tests web agents on the part that comes after search: synthesizing, organizing, and presenting what they find in real documents, spreadsheets, and slides. Before that, I studied what we lose when evaluation data is generated by LLMs instead of people (EMNLP Findings 2025).

Reading outcomes from behavior. In Divergent Paths, which I started as a research intern at Handshake in San Francisco (January to May 2026), I treat agent trajectories as paths through a shared embedding space of actions and observations, where successful and failed attempts at the same task pull apart. We train a model, Precog-RM, on embedded rollouts which learns to separate successful from failing rollouts from just a few examples on tasks and models unseen during training. Often, Precog-RM can already predict a rollout's outcome well before the rollout ends. Used to decide when to stop GRPO training early, it cuts training compute by up to 47% at comparable performance.

Current directions

How graders shape agents. Agents are increasingly post-trained with RL on rewards from LLM graders, and those graders make mistakes. I'm studying how specific kinds of grader error change the strategies agents learn, not just their success rates.

Agent swarms under selection. How groups of LLM agents behave collectively when they cooperate, compete, and are shaped by selection pressure and scarce resources, and where that behavior matches or departs from what we know about human and biological groups.

Most of my free time is spent climbing, running and skiing in the Wasatch — see 8a and Strava.