ERL — Emergent Retrieval Landmark

ERL asks where useful evidence is lost between retrieval and a final answer. The project began with a graph-routing hypothesis: could intermediate memory items help a system reach evidence that direct retrieval would miss? Working on PNEUMA had made this distinction between storing information and recovering usable evidence particularly important to me.

I designed and implemented controlled comparisons that separate access to candidate evidence, selection within a candidate pool, and the reader’s use of the selected context. The work spans conversational memory and multi-hop question answering, using LoCoMo, MuSiQue, and 2WikiMultihopQA in distinct experimental settings. Matched inspection budgets and paired comparisons help isolate what a proposed change actually improves.

In the initial LoCoMo comparison, learned routing and BM25 both reached full-support recall of 0.3671 at an inspection budget of 34. That result did not support the expected routing advantage and shifted my attention toward evidence selection.

Selection changes improved support recall against simpler comparators, but much of the gain diminished against a reranker fine-tuned on the target dataset. Some settings left very little room for further improvement, making the strength of the baseline and the available room for improvement central to interpretation. The same selection strategy could also hurt particular question types.

Better evidence recall did not translate consistently into better answers. Effects depended on the reader and context, and the tested closed-corpus pipeline showed no end-to-end benefit. Candidate pools often lacked required support in that setup; changing selection alone could not recover absent evidence. These observations are specific to the tested systems and do not establish a general limitation of retrieval.

My role covers the research questions, experimental design, implementation, and analysis. The study builds on established datasets, retrieval methods, and pretrained models. I am now examining how the findings depend on retrieval strength, scorer supervision, and the reader’s ability to use evidence.

Further experiments and manuscript development are ongoing. The planned broader evaluation has not yet been executed, and reader-output accounting needs correction before stronger answer-level conclusions can be drawn. ERL’s manuscript is in preparation.