Veer

A programming language for LLM programs whose compiler proves KV-cache memory, prefill cost and numeric error bounds before anything runs.

Role
Solo
Stack
  • Rust
  • Type systems
  • Abstract interpretation
  • LLM inference
Links

Live site for VeerCode for Veer

Docs site sourceRelease binaries

The Veer documentation site home page
Docs and downloads at veer-lang.vercel.app

Problem

Agents, RAG pipelines and tree search are limited by KV-cache memory and prefill time. Serving engines guess at cache reuse, editing an agent’s history quietly re-prefills thousands of tokens, and the only way to know whether a quantized model is still accurate enough is to run evals. I wanted those costs to be visible in the program and checked by a compiler.

What I built

Contexts are linear values. Every operation (append, generate, fork, splice, speculate, drop) consumes its context, so leaks, use-after-free and ambiguous edits are compile errors.

A static planner, written as an abstract interpreter over paged KV segments, proves upper bounds on peak KV bytes, prefill and decode tokens, context length and splice re-prefill cost. @budget(kv = 2GiB, prefill = 12000) is a compile-time contract.

Precision types carry a proven error bound through quantization, matmul and activations across FP8, INT8, MXFP4 and NVFP4, so @accuracy(rel <= 0.25) is also checked at compile time.

A six-crate Rust workspace: parser and diagnostics, bit-exact quantization simulators, type checker, planner, a runtime with a simulated engine and an OpenAI-compatible HTTP engine, and the veer CLI (check, plan, run, types, explain).

Results

  • v0.1.0 released with binaries for macOS (arm64, x64), Windows (x64, MSI) and Linux (x64, arm64), plus shell and PowerShell installers.
  • The test suite includes soundness property tests for the planner and error-bound algebra, and every program in the docs is compiled in CI.