Orchestration gate#

Orchestration: choose which module a robot relies on, from calibrated confidence and cost.

A robot often has several ways to act on a situation: a vision model, a reinforcement-learning policy, an imitation-learned skill, a fault-recovery routine, a human model that predicts what a person will answer. Each reports a confidence; each costs compute, time or energy. A gate picks the module with the best confidence-minus-cost trade-off,

\[k^* = \arg\max_k\; \hat c_k - \lambda\, \text{cost}_k ,\]

where \(\hat c_k\) is the module’s confidence after recalibration. The gate is only as good as those confidences: an over-confident module wins too often. This module provides

  • Module, the description of one module (name, cost);

  • ConfidenceGate, the cost-penalised gate, optionally with a recalibrator per module and a compute budget;

  • MetaCalibratedGate, which learns each module’s confidence offset online from outcomes, starting from a prior meta-learned on earlier tasks (empirical Bayes), so that it is well calibrated from the first steps of a new task;

  • RoutingTask and simulate_routing(), a synthetic task distribution (the one used in the supplementary simulation of the companion review on meta-learning orchestration) for testing gates before deploying them.

All simulated numbers are synthetic; nothing here is robot data.

Examples

>>> import numpy as np
>>> from quantum_mind.applications.orchestration import Module, ConfidenceGate
>>> gate = ConfidenceGate([Module('vision', 1.0), Module('rl', 3.0)], lam=0.05)
>>> gate.select([0.80, 0.85])          # RL is 0.05 more confident but costs 2 more: vision wins
0

Classes

Module

One module the gate can call.

ConfidenceGate

Cost-penalised confidence gate.

MetaCalibratedGate

Gate that learns each module's confidence offset online, from a meta-learned prior.

RoutingTask

A synthetic orchestration task with four modules and four situation types.

Functions

simulate_routing

Run a gate on a synthetic task.

meta_fit_offsets

Meta-learn the prior of MetaCalibratedGate from earlier tasks (empirical Bayes).
class Module(name, cost=1.0)[source]#

Bases: object

One module the gate can call.

Parameters:

  • name (str) – Name, for reports.

  • cost (float) – Relative cost per call (compute, latency or energy, in any consistent unit).

class ConfidenceGate(modules, lam=0.05, calibrators=None, budget=None, window=50)[source]#

Bases: object

Cost-penalised confidence gate.

Parameters:

  • modules (sequence of Module) – The modules, in the order of the confidence vectors.

  • lam (float, optional) – Price of one unit of cost in units of success probability.

  • calibrators (sequence, optional) – One object per module with a transform(conf) method (for example IsotonicCalibration), or None entries to leave a module’s confidence as reported.

  • budget (float, optional) – Largest average cost per call over a sliding window of window calls. When the window is over budget, only modules at most as expensive as the cheapest one plus the remaining headroom are considered.

  • window (int, optional) – Length of the budget window.

Variables:

history (list of int) – Indices of the modules chosen so far.

calibrated(conf)[source]#

Apply the per-module recalibration.

Parameters:

conf (array_like) – Reported confidences, shape (K,) or (n, K).

Returns:

numpy.ndarray – Same shape.

scores(conf)[source]#

Gate scores \(\hat c_k - \lambda\,\text{cost}_k\).

Parameters:

conf (array_like) – Reported confidences.

Returns:

numpy.ndarray

select(conf)[source]#

Choose a module for one situation and record the choice.

Parameters:

conf (array_like) – Reported confidence of each module, shape (K,).

Returns:

int – Index of the chosen module.

select_batch(conf)[source]#

Choose modules for many situations at once (no budget, no history).

Parameters:

conf (array_like) – Shape (n, K).

Returns:

numpy.ndarray – Chosen index per situation.

class MetaCalibratedGate(modules, prior_mean=None, prior_n=1.0, lam=0.05, explore=0.05, seed=0)[source]#

Bases: object

Gate that learns each module’s confidence offset online, from a meta-learned prior.

Each module’s reported confidence is assumed to be off by an unknown, task-specific offset \(b_k\) (for example an imitation-learned skill that is systematically over-confident). After calling module k with confidence \(c_k\) and observing the outcome \(y \in \{0, 1\}\), the residual \(c_k - y\) is an unbiased sample of \(b_k\). The gate keeps the posterior mean

\[\hat b_k = \frac{n_0\, \mu_k + \sum (c_k - y)}{n_0 + n_k},\]

a running mean shrunk toward the prior mean \(\mu_k\) with \(n_0\) pseudo-observations. meta_fit_offsets() estimates \(\mu\) and \(n_0\) from logs of earlier tasks.

Parameters:

  • modules (sequence of Module) – The modules.

  • prior_mean (array_like, optional) – Prior offset per module (default 0).

  • prior_n (float or array_like, optional) – Pseudo-counts of the prior (default 1: learn mostly from scratch). A very large value freezes the offsets at the prior (no online learning).

  • lam (float, optional) – Price of cost.

  • explore (float, optional) – Probability of calling a random module, so that every module keeps being checked.

  • seed (int, optional) – Seed of the exploration.

property offsets#

current estimate of each module’s confidence offset.

Type:

numpy.ndarray

select(conf)[source]#

Choose a module (with a small exploration probability).

Parameters:

conf (array_like) – Reported confidences, shape (K,).

Returns:

int

update(k, conf_k, success)[source]#

Learn from the outcome of a call.

Parameters:

  • k (int) – Module that was called.

  • conf_k (float) – Confidence it had reported.

  • success (bool or int) – Whether it succeeded.

class RoutingTask(offsets, mix, noise=0.08, p_good=0.85, p_bad=0.35, names=('Vision', 'RL', 'Imitation', 'Fault-recovery'))[source]#

Bases: object

A synthetic orchestration task with four modules and four situation types.

Each situation type has one module that succeeds 85% of the time; the others succeed 35% of the time. Each module reports its success probability plus noise plus a task-specific offset.

Parameters:

  • offsets (array_like) – Confidence offset of each module in this task.

  • mix (array_like) – Probability of each situation type.

  • noise (float, optional) – Standard deviation of the confidence noise.

  • p_good (float)

  • p_bad (float)

  • names (tuple)

COST = (1.0, 3.0, 2.0, 4.0)#

Relative costs of the four default modules.

MEAN_OFFSETS = (0.0, 0.3, 0.25, 0.05)#

reinforcement learning and imitation are over-confident.

Type:

Mean offsets across tasks

classmethod sample(rng, spread=0.1)[source]#

Draw a task: offsets around MEAN_OFFSETS and a random situation mix.

Parameters:
Returns:

RoutingTask

modules()[source]#

The four modules with their default costs.

Returns:

list of Module

step(rng)[source]#

One situation.

Parameters:

rng (numpy.random.Generator)

Returns:

simulate_routing(gate, task, steps, rng, learn=True)[source]#

Run a gate on a synthetic task.

Parameters:
Returns:

dict – success (0/1 per step), cost (cost per step) and choice (module per step).

meta_fit_offsets(logs, residual_var=0.25)[source]#

Meta-learn the prior of MetaCalibratedGate from earlier tasks (empirical Bayes).

Parameters:

  • logs (sequence) – One entry per earlier task: a list of (module index, reported confidence, success) tuples, collected with an exploratory policy so that every module was tried.

  • residual_var (float, optional) – Variance of one residual \(c - y\) (about \(p(1 - p)\), at most 0.25).

Returns:

  • prior_mean (numpy.ndarray) – Mean offset per module across tasks.

  • prior_n (numpy.ndarray) – Pseudo-counts \(n_0 = \sigma^2_{\text{within}} / \sigma^2_{\text{between}}\), clipped to [1, 200]: many when tasks agree, few when they differ.