DSPy Compiling Declarative Language Model Calls (Khattab et al., 2023)

URL: https://arxiv.org/abs/2310.03714

The paper proposes DSPy, a programming model that replaces hand-coded prompt templates with systematic optimizable language-model pipelines. The framework treats LM pipelines as "text transformation graphs, i.e., imperative computational graphs where LMs are invoked through declarative modules." Modules are parameterized; a compiler optimizes any DSPy pipeline to maximize specified metrics. The empirical claim is over 25 percent and 65 percent improvement over standard few-shot prompting across multiple benchmarks.

Adopted

DSPy is the foundational framework that the RLM research direction extends. The "writing programs that compile down to prompts" framing is conceptually downstream of the substrate-vs-glue diagnostic this graph names: programs need a substrate to compile into; pipelines need a runtime to run on. Khattab's own subsequent work (RLM, Agent Skills) elaborates on the framework's substrate dependencies, naming the variable-vs-token-space distinction this graph's harness-as-tool inversion ([[Agent Harnesses Drive the Runtime, Not the Reverse]]) makes load-bearing. eOS Continuum is the substrate-LAYER answer to the substrate question DSPy raises and works around in user-space.

Not adopted (yet)

DSPy operates as a Python-runtime compilation framework; the substrate underneath is whatever Python provides, which is the wrong substrate for the workload (no orthogonal persistence, no capability separation, no atomic operations as primitives). The compilation framework itself is downstream of substrate concerns; eOS Continuum does not adopt DSPy's compilation model -- it adopts the framework's diagnosis that LM-pipelines-as-programs need a substrate, and provides the substrate.

Sources

Relations