Same user prompt, two architectures: a deterministic Etch expert grounded in GLC APIs versus asking Claude Opus to “figure it out” with general knowledge and brittle JSON guessing.
What “deterministic” means here
Given the same inquiry text and the same GLC database state, an expert follows the same tool sequence and produces the same structured output. Reasoning strings are templated from actual tool results — not sampled from a probability distribution. There is no invented order ID, invoice line, or stock count.
Etch’s orchestrator LLM can paraphrase or occasionally misread the expert JSON when writing the final reply. That is why experts return explicit handoff payloads and the UI lets you expand the expert monologue separately.
Measured expert runs (this environment)
| Scenario | Tool calls | Total tokens | Expert wall time | Throughput |
|---|---|---|---|---|
| Order tracking | 4 | ~1,040 | ~90 ms | ~11,000 tok/s |
| AP three-way match | 6 | ~1,110 | ~120 ms | ~9,000 tok/s |
| Procure-to-pay | 14 | ~2,010 | ~430 ms | ~4,700 tok/s |
| Sales margin (5 pages) | 4 | ~3,330 | ~1,060 ms | ~3,100 tok/s |
Full chat turns also include Etch orchestration (often 15–30+ seconds) because the wrapper LLM connects, routes, and narrates. The expert portion above is the part you would run in production at scale without re-paying frontier token rates per API lookup.
Illustrative comparison: expert vs Claude Opus
We have not benchmarked Opus against GLC in this repo, but the gap follows familiar patterns for tool-grounded ops work. Numbers below are order-of-magnitude estimates for a single comparable task (e.g. order lookup + timeline + draft reply):
| Dimension | Etch SDK expert | Claude Opus (agentic) |
|---|---|---|
| Data grounding | Every fact from GLC API responses | Must be tool-wired; otherwise confabulates IDs, dates, totals |
| Error rate on structured ops | Logic bugs only — reproducible, fix in code | Higher variance run-to-run; prompt-sensitive |
| Expert / core compute time | 0.1–1.5 s | Often 20–60 s for multi-tool chains |
| Employee wait cost | Sub-second core path | Tens of seconds × hourly loaded cost per employee |
| Token spend (core task) | ~1k–3k scripted tokens; no frontier inference on expert path | Often 20k–80k+ input/output tokens across turns |
| Approx. API cost per run | ≈ $0 (compute only) | ≈ $0.15–$0.75+ at Opus-class pricing |
| Audit trail | Full trace: tools, args, timings | Opaque model states; harder compliance story |
At 500 such lookups per day, the expert path saves hundreds of dollars in API fees alone — before counting employee time lost to waiting. The UX looks the same in the chat panel; the economics and reliability do not.
When to use which layer
- Expert — Repeatable workflows with clear APIs: AP match, order status, inventory scans, KPI pulls.
- LLM orchestrator — Intent disambiguation, merging multiple expert outputs, tone and formatting.
- Frontier model alone — Exploratory analysis with no API, or tasks where occasional hallucination is acceptable.
See live transcripts: Order tracking, AP match, Procure-to-pay, Margin analysis.