DynaFlow-style Orchestrator

Executable agent coordination with an honest solo-versus-pipeline cost comparison.

Role
Solo, AI-assisted prototype session
Stack
  • Python
  • AST interpreter
  • Ollama
  • LLM evaluation
Links

Code for DynaFlow-style Orchestrator

Published-subset measurementsSeparate calibration measurements

Bar chart of actual tokens per solved task for solo, fixed and generated workflows on math and multi-hop calibration tasks
Tokens per solved task on the easy calibration set, from RESULTS.md

Problem

Agent teams add calls and context. The useful question is whether task-specific coordination improves cost per solved task against the same model running alone.

What I built

An AI-assisted prototype prompts a lead to emit constrained executable Python workflows. Frozen solvers can run in parallel, followed by verification and aggregation, with full original context and prior outputs. Real Qwen3 0.6B Q4 CPU calls exercise the coordination.

An AST interpreter permits only coordination functions. Two separately reported datasets compare solo, fixed planner/solver/reviewer and generated workflows: six published AIME plus six HotpotQA questions, and twelve original easy calibration tasks. Raw responses, programs, token usage and failed plans are retained.

Results

  • On the published subset, every setup scores 0/6 AIME; HotpotQA scores are 0/6 solo, 1/6 fixed and 0/6 generated. The run uses 87 successful provider calls and 84,774 actual tokens, with no provider errors. Cost per solved task is undefined when no task is solved.
  • On the separate easier calibration, math accuracy is 1/6 for every setup; actual tokens per solved task are 1,039 solo, 4,875 fixed and 3,257 generated. Multi-hop is 4/6, 4/6 and 1/6, at 292, 1,060.5 and 2,915 tokens per solved task. Generated orchestration loses token efficiency by 3.13× and 9.98×.
  • Local API fees are $0, excluding hardware and energy. All unsolved-task and invalid-plan tokens are charged. Ten tests pass. These tiny-model results, with different output budgets for the two datasets, do not reproduce the trained 27B paper’s headline accuracy.
  • The GitHub Pages explorer exposes both saved datasets, measured tables, actual workflows and complete call traces. It runs no live model inference.