Consistency in multi-agent LLM systems
I approach this problem in three steps:
-
Define
What does “consistent” mean for a multi-agent system?
Consistency is usually defined for one model and one prompt. I'm working on a definition that fits settings where many agents interact.
-
Diagnose
Where does inconsistency come from?
I'm tracing inconsistent behavior back to its root causes, so that it can be understood and fixed rather than just observed.
-
Measure
How do we evaluate it?
I'm working toward metrics and benchmarks that measure consistency in multi-agent settings, so different systems can be compared.
Results from this project will be posted on my publications page as they appear.