Skip to content
parelyxHow it works

From request to decision.

Parelyx makes one bounded decision per request, then learns from the outcome offline. This is adaptive decision-making — not a fixed sequence of operations every request must pass through.

Steps 01–05 · live pathStep 06 · offline
  1. 01
    What kind of request is this?

    Understand the workload

    Lightweight signals are derived from the incoming request: its size and structure, the kind of workload it belongs to, and whether it could be eligible for reuse.

    Consideredlive path
    • Context length
    • Workload type
    • Structure
    • Reuse eligibility
  2. 02
    What must this request respect?

    Establish constraints

    The latency budget, quality requirement and cost sensitivity that apply to the request are resolved from configuration and policy.

    Consideredlive path
    • Latency budget
    • Quality requirement
    • Cost constraint
    • Pinned models
  3. 03
    Would any optimisation create net value?

    Evaluate opportunities

    Each candidate strategy is weighed against its expected benefit and its own cost. If nothing is expected to pay off, that is the answer.

    Consideredlive path
    • Expected saving
    • Added work
    • Quality risk
    • Remaining budget
  4. 04
    Which path should this request take?

    Select the execution strategy

    The engine selects one path — direct execution, reuse, optimisation, model or provider selection, runtime-aware execution — or bypasses optimisation entirely.

    Consideredlive path
    • Selected strategy
    • Selected model
    • Reason recorded
  5. 05
    What actually happened?

    Execute and measure

    The request executes on your infrastructure. The decision, its reason and the observed outcome are recorded — by design, off the response path.

    Consideredlive path
    • Latency
    • Tokens
    • Estimated cost
    • Errors
  6. 06
    Should future decisions change?

    Learn and improve

    Validated historical evidence is used offline to evaluate and refine decision policies. Improvements reach the live path only after validation and controlled rollout.

    Processoffline
    • Offline evaluation
    • Validation
    • Staged rollout
Step 03 · the net-value rule

If it doesn't pay for itself, it doesn't run.

Step 03 is where most requests are decided. An optimisation that adds more time than it removes is worse than no optimisation at all.

Latency budgetConceptual · not to scale
Direct path
Model execution
With optimisation
Model execution
Decision
Apply the optimisation

The direct path would overrun the budget. Deciding and optimising take less time than they remove from execution, so the optimised path completes inside it.

Step 04 · in practice

Different requests, different decisions.

Execution decisionConceptual illustration
Incoming request

“How do I rotate the API key for my workspace?”

Short promptCommon intentNon-personalised
Parelyx
Decision engine
bounded decision budget
Latency budget
Tight
Quality requirement
Standard
Cost sensitivity
High
Reuse eligibility
Eligible
Expected net benefit
Positive
  1. Safe bypassskips the decision step
  2. Direct executionconsidered · not chosen
  3. Semantic response reuseselected
  4. Selective prompt optimisationconsidered · not chosen
  5. Model & provider selectionavailable
  6. Runtime-aware executionavailable
Selected strategy

Semantic response reuse

A previously validated response to a semantically equivalent request is returned without a model call.

  • A validated answer to an equivalent request already exists.
  • The answer does not depend on user-specific or time-sensitive data.
  • Reuse avoids a model call entirely, comfortably inside the latency budget.
Considered, not chosen
Direct execution
Would repeat work already done for an equivalent request.
Selective prompt optimisation
Prompt is already short; nothing meaningful to remove.
Illustrative scenarios. Signals and decisions are described qualitatively, not measured.
Step 06 · learn and improve

The learning lifecycle.

How validated workload evidence becomes better decisions — and where learning never runs.

Workload learning loopConceptual illustration
  1. 01live
    Production workloads

    Requests are served by the current, validated policy.

  2. 02offline
    Outcome intelligence

    Latency, throughput, cost and quality feedback are captured asynchronously.

  3. 03offline
    Offline policy learning

    Candidate policies are refined and evaluated against historical workloads.

  4. 04offline
    Validated policies

    Only candidates that meet latency and quality constraints on held-out workloads proceed.

  5. 05live
    Better execution decisions

    Policies roll out gradually, are monitored, and roll back on regression.

  1. 01
    Observe

    Record workload characteristics and execution conditions for each request.

  2. 02
    Collect outcomes

    Capture latency, throughput, cost and — where available — quality feedback.

  3. 03
    Build datasets

    Assemble privacy-conscious workload datasets, with customer-controlled retention.

  4. 04
    Evaluate

    Replay existing decision policies against historical workloads.

  5. 05
    Refine

    Improve policies using offline learning techniques.

  6. 06
    Validate

    Test candidates on held-out workloads against latency and quality constraints.

  7. 07
    Deploy gradually

    Roll out through controlled, staged exposure.

  8. 08
    Monitor & roll back

    Compare against the previous policy and revert on regression.

  9. 09
    Repeat

    New evidence feeds the next cycle.

Learning approaches

Offline policy evaluation, contextual bandits and constrained reinforcement learning suit this problem well — with latency budgets and quality floors as the constraints.

On human feedback

Operational telemetry alone is not RLHF. Preference-based learning needs explicit human preference signals and is used only where customers provide them. Parelyx refines its decision policies — it does not retrain your foundation models.

Questions

Common questions.

Read the Architecture
Does every request go through all six steps?
No. Steps 01–04 are a bounded decision, and the result is often “do nothing” — the request executes directly. Step 06 never runs on the request path at all.
What happens if Parelyx itself has a problem?
The request fails open: it bypasses the decision step and is forwarded to its default execution path, so an internal fault in Parelyx does not become a failed request.
How much latency does the decision add?
The design goal is that decision overhead stays inside a bounded budget and that no optimisation is applied unless it saves more than it costs. Because the answer depends on your workload, we measure it with you on your own traffic rather than quoting a general figure.
Can I force a specific behaviour?
Yes. Applications can pin models, and individual optimisations can be enabled or disabled in configuration. Explicit configuration always takes precedence over learned policy.