it's unclear what the trajectory of LLM model performance will look like in the next 10 years, but progress might continue to surge not due to model improvements but rather in how we coordinate agents.

In some ways, it's a HCI/UX issue where we better tailor LLMs to specific tasks. In particular, tasks of the nature where humans can describe what they want to do, but they need to do it slightly different in 1000+ experiments (combined with paper research).

we've seen google's deep think models assisted mathmaticians with this type of work, so it's natural for this to appear in the bioscience realm as well.

while we might just be seeing cherry picked results (there could be more dead ends with co-scientists than that are positive outcomes). we will need a lot more users to fully decide that.

i definitely felt early adopters of coding agents were cherry picking results as well, but adoption and success followed as the UX tools and the models greatly improved. the question is, will scientific research also need better or tailored models, or just better coodination of agents?

one of the paper is worth a skim because they've published examples of prompts they have used for their positive outcome report.