BV-SALA · In production

BiVelio Savings Layer

Spend less on every AI call.

What it is

BV-SALA sits between your application and the AI API. Before your prompt goes out, it applies a series of optimizations and modifications —compression, rewriting and removal of redundant context— so the same task consumes fewer tokens and therefore costs less.

How it works

1
Receives your prompt and the context you were about to send.
2
Applies optimizations: it removes redundancy, compresses and rewrites while preserving intent.
3
Sends the minimum sufficient version to the API you already use.
4
Returns the response to you along with the metrics of what was saved.

Fig. 2 — A layer that optimizes and rewrites your prompt before sending it to the API, to lower the cost per call.

The paper, in detail

§1 The problem, formalized

The provider charges a function essentially linear in tokens, counted with its own tokenizer. BV-SALA picks π* = argmin E[C(π(x))] subject to the quality contract Q ≥ τ. The intrinsic difficulty: Q is not observable before executing, so every transformation in the catalog carries its own guard — it applies only where accumulated evidence indicates intent is preserved, and in doubt the request travels untouched.

C(x) \;=\; c_{\text{in}} \cdot T_{\text{in}}(x) \;+\; c_{\text{out}} \cdot T_{\text{out}}(x)

(1)

\pi^{*} \;=\; \operatorname*{arg\,min}_{\pi \,\in\, \Pi} \; \mathbb{E}\big[\, C(\pi(x)) \,\big] \qquad \text{s.t.} \qquad Q(\pi(x)) \,\ge\, \tau

(2)

§2 Three ledgers that do not add up

Every optimized request is attributed to exactly one ledger by dominant mechanism: avoiding the call entirely, removing what never needed to be sent, or compressing what remains. Each figure speaks over its own denominator — aggregating them would count the same euro twice — and a ledger that does not clear zero against both control arms is published all the same, with its label, but never advertised.

S_{\ell} \;=\; 1 \;-\; \frac{\sum_{x \in R_{\ell}} C\big(\pi^{*}(x)\big)}{\sum_{x \in R_{\ell}} C(x)}

(3)

§3 Measurement: the floor of the interval

Same model in two control arms — against an unoptimized integration and against the same integration already hand-optimized — with cost quoted by the provider's own meter, no modeled baseline. Scenarios are aggregated with a stratified bootstrap, weight one per scenario, and we publish the one-sided lower bound of the 95% confidence interval, never the point estimate. Hence the headline is advertised as ≥ 20.0% less bill on gpt-4o-mini: below what the stricter of the two controls clears.

S_{\text{pub}} \;=\; \hat{S} \;-\; z_{0.95}\,\widehat{\operatorname{se}}\big(\hat{S}\big)

(4)

§4 Net accounting and honest caching

Analyzing every request also consumes: all accounting is net, and the published saving discounts the cost of the layer itself. And a cache only counts if the reused answer is still valid: keys incorporate model, version, visible tools and effective context — on any difference, we call.

S_{\text{net}} \;=\; 1 \;-\; \frac{C\big(\pi^{*}(x)\big) + C_{\text{layer}}(x)}{C(x)}

(5)

k_{\text{cache}} \;=\; H\big(\text{model} \,\Vert\, \text{version} \,\Vert\, \text{tools} \,\Vert\, \text{context}\big)

(6)

What it delivers

Fewer tokens sent for the same task.
Works with the APIs you already use, without switching providers.
Savings metrics visible on every call.

Status

Fully productionized as a standalone product: savings.bivelio.com — licensed SDK on npm and savings measured against the provider's own meter, with its confidence interval alongside.

The figures we publish are measured, not estimated — never projections. All our research runs on NVIDIA inference and hardware.

Read the full article All algorithms