Skip to content

Toolformer: Language Models Can Teach Themselves to Use Tools

Summary

Toolformer fine-tunes a language model on a self-curated dataset of API calls, teaching it when to invoke external tools (calculator, QA system, search engine, translator, calendar) inside its generation. Tool-call insertion points are chosen by the model itself; supervision is generated by sampling candidate insertions and keeping those that reduce perplexity on the suffix.

Contribution

A practical self-supervised recipe for teaching tool use that sidesteps the need for hand-labeled tool-call corpora and works across diverse tools. Made the case that moderate-size models augmented with tools can match or beat much larger un-augmented models on knowledge-heavy tasks.

Method

Pretrained LM (6.7B GPT-J-style) is prompted to insert candidate API calls; each candidate is evaluated by an in-context utility metric (reduction in loss on continuation) and accepted/rejected. The filtered dataset is then used to fine-tune the model. Evaluation spans LAMA, math word problems, multilingual QA, and temporal reasoning.

Relevance to RISE

Toolformer is a canonical reference for the tool-use primitive underneath nearly every RISE pipeline in the catalog. Its self-supervised data-construction idea also prefigures the artifact-driven training loops some RISE projects (notably agent-laboratory and the Sakana lineage) rely on for iterative improvement.

Critique / open questions

  • Self-curated supervision can reinforce a model's existing tool-call biases — failure modes that escape the loss-reduction filter propagate at scale.
  • Tools in the original paper are clean APIs; tool use in scientific pipelines must contend with rate limits, ambiguous schemas, and domain-specific failure modes (e.g., dataset access errors, hallucinated citations) that Toolformer-style training does not directly address.
  • The paper predates the multi-agent turn — orchestration of multiple tool-using agents is out of scope.