Optimize for Fast Decisions or Fast Outcomes
Github repo: https://github.com/juustesout/contextual-vector-bandit-in-typescript
We built and evaluated a standalone contextual bandit system to answer a practical product question: when should you prefer a classic per-arm approach like LinUCB, and when does a vector-native shared model create better outcomes?
The short answer is nuanced and useful:
LinUCB is often stronger when each arm behaves independently and there is little transferable structure between actions.
A vector-native bandit is often stronger when actions share meaningful context or embedding similarity, because each update can generalize to nearby actions.
If you optimize only for per-decision compute latency, classic methods can look better.
If you optimize for time-to-outcome (for example, how many interactions it takes to reach a successful event), vector-native methods can become compelling even with higher compute per decision.
The main conclusion is not “one algorithm wins.”
The main conclusion is: choose the algorithm that matches the structure of your action space and the business objective you actually care about.
Why We Ran This Evaluation
In real recommendation and decision systems, teams often default to one of two narratives:
“Use a simple bandit, keep inference cheap.”
“Use embeddings everywhere, transfer learning across actions.”
Both narratives can be right, both can be wrong, and both are incomplete without context.
Most teams also benchmark with narrow metrics, usually average inference time and aggregate reward. Those are useful, but insufficient for decision-making. A model can be slower per call and still deliver better business outcomes if it needs fewer interactions to reach success.
So we set up the evaluation around a broader question:
How does each approach behave across different environment structures and different objective lenses, including outcome-oriented metrics?
What We Compared
We compared:
Classic LinUCB (disjoint, per-arm model)
Vector-native bandit (shared model over joint context-action representation)
Conceptually:
LinUCB keeps independent statistics per arm. It excels when each arm is truly distinct and you cannot transfer much from one arm to another.
Vector-native modeling shares one model over context-action features, so feedback on one action can update confidence around nearby actions.
We also tested variants and implementation profiles, but the central narrative is this LinUCB vs shared vector-native trade-off.
Fairness in the Methodology
A lot of benchmark claims break because the protocol is not fair. We explicitly addressed this.
The evaluation uses a fair harness with these guardrails:
Shared contexts across algorithms in the same run.
Shared candidate sets across algorithms in the same run.
Common random noise policy so stochastic effects are matched.
Multi-seed studies, not single-seed storytelling.
This matters because otherwise differences can come from data ordering, RNG drift, or retrieval variance, not from algorithm quality.
We also compare across multiple environment types instead of one synthetic world, because bandit behavior is environment-dependent by design.
The Environment Lens: Where Each Model Is Strong
The most important finding is structural: performance is tightly coupled to environment assumptions.
1) Per-arm independent structure
When actions are effectively independent and there is weak cross-arm similarity structure, LinUCB tends to perform better.
This is exactly where per-arm local modeling is a strength, not a limitation.
Interpretation:
LinUCB’s disjoint updates are not “wasted capacity” in this setting; they are aligned with the true data-generating process.
A shared vector model can over-generalize and smear signal between arms that should stay separate.
2) Shared contextual structure
When arms share meaningful latent structure (for example, embedding neighborhoods that reflect similar response behavior), vector-native modeling tends to perform better.
Interpretation:
Shared representation unlocks sample efficiency.
One update can inform nearby actions.
The model climbs faster in environments where transfer is valid.
3) Mixed structure
In mixed environments, behavior is hybrid:
Vector-native approaches can still show strong gains in outcome metrics when shared structure dominates enough of the interaction stream.
LinUCB can remain competitive or superior in slices where local arm idiosyncrasy dominates.
This is often the closest to production reality.
Time-to-Decision vs Time-to-Outcome
One of the most important outcomes of this work is metric clarity.
Time-to-decision
This is model compute latency per interaction.
It answers: how fast can I score one step?
LinUCB can look excellent here, especially in disjoint setups and smaller dimensions.
Time-to-outcome
This is interaction efficiency and end-to-end progress toward business success.
It answers: how many attempts and how much elapsed decision effort does it take before I get a useful result?
This can favor vector-native methods in shared-structure worlds, even if per-step compute is higher.
If someone visits webpages with ads, and clicks every 10 seconds, and the vector bands picks a good offer and makes the sale in 3 tries, I get a lot more sales than the normal bandit with 5 or 6 tries, and on that timescale, it doesn’t matter that the linUCB function takes 1ms and the vector function 15ms.
That distinction is operationally critical. A few milliseconds of extra scoring time can be irrelevant if it reduces failed interactions and reaches success much earlier from a user or revenue perspective.
What We Added to Make This Measurable
To avoid hand-wavy claims, we added explicit outcome metrics in the benchmark protocol.
The study now tracks episode-level and outcome-level metrics such as:
- Success rate under a defined success threshold
- Average turns to success (for successful episodes)
- Average turns to outcome under censoring rules
- Decision-time accumulation to outcome
This makes statements like “model A reaches useful outcomes in fewer turns” testable, not rhetorical.
It also makes the benchmark directly useful for product planning, because these metrics can be mapped to interaction costs and value-per-conversion assumptions.
Practical Read of Findings
An objective read should avoid universal winners. Here is the practical framing.
When LinUCB is the better default
Choose LinUCB-first if:
- Arms are mostly independent.
- You have low confidence in embedding quality or relevance.
- You need very low computational overhead per call.
- You are in a regime where transfer between actions is likely harmful.
Why this is coherent:
LinUCB’s disjoint structure is robust and interpretable in locally heterogeneous arm landscapes.
It can be simpler to reason about under strict real-time constraints.
When vector-native is the better default
Choose vector-native-first if:
- Action embeddings are meaningful and stable.
- You expect substantial behavioral similarity across neighboring actions.
- You need sample efficiency in large catalogs.
- You care about reducing decision count to reach success.
Why this is coherent:
Shared representation captures transferable structure.
The model can improve faster in sparse-feedback settings.
Outcome-oriented metrics can favor this path strongly in the right environment.
When to run both in production
Use a dual-path strategy if:
- Your traffic is structurally mixed.
- You can segment users, contexts, or verticals.
- You can route by confidence or observed environment signatures.
A common strategy
- Start with conservative LinUCB on segments suspected to be per-arm.
- Use vector-native on segments with embedding-trust and demonstrated transfer benefits.
- Continuously re-evaluate routing policy from live outcomes.
Interpreting the “Per-Arm vs Shared” Result Correctly
It is tempting to over-simplify:
“LinUCB wins per-arm, vector wins contextual.”
That is directionally true, but still too coarse.
Nuances that matter:
Embedding quality is a first-order variable. Poor embeddings can erase vector-native advantages.
Retrieval strategy affects candidate quality and thus downstream bandit behavior.
Exploration tuning can flip local rankings.
Noise models and reward clipping policies can mask or amplify differences.
Mixed environments can produce non-linear crossover behavior.
So treat environment labels as structural anchors, not deterministic guarantees.
Limitations and What We Did Not Claim
Objective review means being explicit about limits.
We did not claim:
- Universal superiority of vector-native methods.
- Universal inferiority of LinUCB in modern stacks.
- One benchmark run is enough for production rollout.
- Synthetic environments directly equal real-world business outcomes.
We also acknowledge:
- Synthetic benchmarks approximate but do not replace online A/B outcomes.
- Distribution shift can change winner identities.
- Operational constraints (SLOs, infra cost, deployment risk) matter as much as offline gains.
What the evidence supports:
LinUCB is strong and often better when action behavior is disjoint.
Vector-native modeling is strong and often better when similarity structure is real and usable.
Outcome-oriented metrics reveal advantages that per-decision latency alone can hide.
Robust benchmarking requires fairness controls and multi-seed reporting.
Production decisions should be segmented, iterative, and tied to business outcomes.
If you force a universal answer, you will likely deploy the wrong model in half your traffic.
If you accept structural nuance and evaluate on time-to-outcome as well as time-to-decision, you get a practical strategy that scales.
That is the real finding.
Github repo: https://github.com/juustesout/contextual-vector-bandit-in-typescript
Protocol (Reproducible)
- Seeds: 20
- Rounds: 2000
- Actions: 200
- Candidates per round: 25
- Dimension: 16
- Joint dimension: 64
- Outcome protocol: max turns 10, success threshold 0.8
- Tuning: per-algorithm alpha tuning over grid [0, 0.25, 0.5, 1, 2]
Fairness controls: shared contexts, shared candidate sets, and deterministic common-random-noise across algorithms.
Main Quality Results (20 Seeds, Tuned)
| Environment | Regret % vs LinUCB (Vector) | Reward Delta (Vector – LinUCB) | Inference Ratio (Vector / LinUCB) | Winner |
|---|---|---|---|---|
| Bilinear (shared structure) | +65.30% | +0.1025 | 14.46x | Vector-native |
| Per-arm (independent arms) | -29.25% | -0.0490 | 11.47x | LinUCB |
| Mixed | +53.98% | +0.0748 | 14.25x | Vector-native |
Outcome Results (20 Seeds, Tuned)
| Environment | LinUCB Success Rate | Vector Success Rate | Success Lift (pp) | LinUCB Avg Turns to Outcome | Vector Avg Turns to Outcome | Turns Reduction % |
|---|---|---|---|---|---|---|
| Bilinear | 0.3840 | 0.7383 | +35.43 | 8.101 | 5.754 | 28.97% |
| Per-arm | 0.3976 | 0.2391 | -15.85 | 8.041 | 8.856 | -10.13% |
| Mixed | 0.2353 | 0.5081 | +27.29 | 8.890 | 7.339 | 17.44% |
1) LinUCB is stronger in truly per-arm regimes
In the per-arm environment, LinUCB wins both quality and outcome metrics. This is expected and important: when arms are genuinely independent, disjoint per-arm learning is exactly the right inductive bias.
2) Vector-native is stronger where structure is shared
In bilinear and mixed environments, the vector-native shared model materially outperforms LinUCB on regret, reward, success rate, and turns-to-outcome. This supports the transfer-learning hypothesis: updates from one action inform nearby actions when embeddings capture real similarity.
3) Compute latency and outcome latency are different objectives
Vector has higher per-decision compute in this benchmark, but still wins in time-to-outcome in shared-structure regimes because it needs fewer interactions before success. For product systems, this distinction often matters more than micro-latency alone.
Important nuance. The outcomeTimeReductionPct numbers in the raw API are based on model compute-time only and can be negative for Vector because Vector does more math per decision. That does not contradict fewer turns-to-outcome. It means: if your real-world interaction loop is dominated by user/network/business latency per turn, fewer turns can still be the bigger win.
Speed Profile Snapshot (Serving Path)
From the latency matrix sweep in this repo (same app, k in {25,100,250}).
| k | Best Total Latency Profile | Best Total (ms) | Best Retrieval Profile | Best Select Profile |
|---|---|---|---|---|
| 25 | factored_nocache | 4.293 | dense | factored_cache |
| 100 | factored_cache | 5.956 | factored_cache | factored_cache |
| 250 | factored_nocache | 10.015 | factored_nocache | factored_cache |
Objective Conclusion
The right claim is not: one algorithm is universally best. The right claim is: model choice should follow environment structure and business objective.
- If your domain is per-arm and independent, LinUCB is a strong default.
- If your domain has reliable contextual similarity, a vector-native shared model can deliver better outcomes in fewer interactions.
- For decision systems that monetize successful outcomes, optimize for time-to-outcome, not only time-to-decision.
