From request to decision.
Parelyx makes one bounded decision per request, then learns from the outcome offline. This is adaptive decision-making — not a fixed sequence of operations every request must pass through.
- 01What kind of request is this?
Understand the workload
Lightweight signals are derived from the incoming request: its size and structure, the kind of workload it belongs to, and whether it could be eligible for reuse.
Consideredlive path- Context length
- Workload type
- Structure
- Reuse eligibility
- 02What must this request respect?
Establish constraints
The latency budget, quality requirement and cost sensitivity that apply to the request are resolved from configuration and policy.
Consideredlive path- Latency budget
- Quality requirement
- Cost constraint
- Pinned models
- 03Would any optimisation create net value?
Evaluate opportunities
Each candidate strategy is weighed against its expected benefit and its own cost. If nothing is expected to pay off, that is the answer.
Consideredlive path- Expected saving
- Added work
- Quality risk
- Remaining budget
- 04Which path should this request take?
Select the execution strategy
The engine selects one path — direct execution, reuse, optimisation, model or provider selection, runtime-aware execution — or bypasses optimisation entirely.
Consideredlive path- Selected strategy
- Selected model
- Reason recorded
- 05What actually happened?
Execute and measure
The request executes on your infrastructure. The decision, its reason and the observed outcome are recorded — by design, off the response path.
Consideredlive path- Latency
- Tokens
- Estimated cost
- Errors
- 06Should future decisions change?
Learn and improve
Validated historical evidence is used offline to evaluate and refine decision policies. Improvements reach the live path only after validation and controlled rollout.
Processoffline- Offline evaluation
- Validation
- Staged rollout
If it doesn't pay for itself, it doesn't run.
Step 03 is where most requests are decided. An optimisation that adds more time than it removes is worse than no optimisation at all.
The direct path would overrun the budget. Deciding and optimising take less time than they remove from execution, so the optimised path completes inside it.
Different requests, different decisions.
“How do I rotate the API key for my workspace?”
- Latency budget
- Tight
- Quality requirement
- Standard
- Cost sensitivity
- High
- Reuse eligibility
- Eligible
- Expected net benefit
- Positive
- Safe bypassskips the decision step
- Direct executionconsidered · not chosen
- Semantic response reuseselected
- Selective prompt optimisationconsidered · not chosen
- Model & provider selectionavailable
- Runtime-aware executionavailable
Semantic response reuse
A previously validated response to a semantically equivalent request is returned without a model call.
- A validated answer to an equivalent request already exists.
- The answer does not depend on user-specific or time-sensitive data.
- Reuse avoids a model call entirely, comfortably inside the latency budget.
- Direct execution
- Would repeat work already done for an equivalent request.
- Selective prompt optimisation
- Prompt is already short; nothing meaningful to remove.
The learning lifecycle.
How validated workload evidence becomes better decisions — and where learning never runs.
Requests are served by the current, validated policy.
Policies roll out gradually, are monitored, and roll back on regression.
Latency, throughput, cost and quality feedback are captured asynchronously.
Candidate policies are refined and evaluated against historical workloads.
Only candidates that meet latency and quality constraints on held-out workloads proceed.
- 01liveProduction workloads
Requests are served by the current, validated policy.
- 02offlineOutcome intelligence
Latency, throughput, cost and quality feedback are captured asynchronously.
- 03offlineOffline policy learning
Candidate policies are refined and evaluated against historical workloads.
- 04offlineValidated policies
Only candidates that meet latency and quality constraints on held-out workloads proceed.
- 05liveBetter execution decisions
Policies roll out gradually, are monitored, and roll back on regression.
- 01Observe
Record workload characteristics and execution conditions for each request.
- 02Collect outcomes
Capture latency, throughput, cost and — where available — quality feedback.
- 03Build datasets
Assemble privacy-conscious workload datasets, with customer-controlled retention.
- 04Evaluate
Replay existing decision policies against historical workloads.
- 05Refine
Improve policies using offline learning techniques.
- 06Validate
Test candidates on held-out workloads against latency and quality constraints.
- 07Deploy gradually
Roll out through controlled, staged exposure.
- 08Monitor & roll back
Compare against the previous policy and revert on regression.
- 09Repeat
New evidence feeds the next cycle.
Offline policy evaluation, contextual bandits and constrained reinforcement learning suit this problem well — with latency budgets and quality floors as the constraints.
Operational telemetry alone is not RLHF. Preference-based learning needs explicit human preference signals and is used only where customers provide them. Parelyx refines its decision policies — it does not retrain your foundation models.