Perception and fusion#

Adapters that turn perception outputs into likelihoods, three fusion rules, and the bistable-perception models. The equations are on the mathematics page.

Legend: blue inputs, violet internal steps, green outputs, coral people, dashed grey classical baselines. Each entry is labelled by origin: Unique to Quantum Mind designed in this library; it builds on the methods cited with it, Published model, extended implements the cited model as published; Quantum Mind adds the capabilities listed, Published method implements the cited method as published, Software tool infrastructure rather than a model; and by category: Quantum-like quantum probability used to model people; runs on an ordinary computer, Quantum computing quantum circuits; run on the simulator or sent to quantum hardware, Classical classical statistics, decision theory or baselines, Mixed combines parts from more than one of these categories.

Perception adapters#

Published method Classical

Detector scores or logits, speech n-best lists and gaze or pointing directions as likelihoods over the same candidate objects.

        flowchart LR
    accTitle: Block diagram of Perception adapters
    accDescr: Detector scores or logits, speech n-best lists and gaze or pointing directions as likelihoods over the same candidate objects.
    det["detector scores / logits"]:::in --> a1["tempered softmax"]:::op
    asr["speech n-best"]:::in --> a2["keyword match × confidence"]:::op
    dir["gaze / pointing vector"]:::in --> a3["von Mises-Fisher"]:::op
    a1 & a2 & a3 --> L["likelihoods over objects"]:::out
    classDef in fill:#0b2a4a,stroke:#79c0ff,color:#e6edf3
    classDef op fill:#2a1b3d,stroke:#d2a8ff,color:#e6edf3
    classDef out fill:#0f2e1a,stroke:#7ee787,color:#e6edf3
    classDef hum fill:#3d1414,stroke:#ff7b72,color:#e6edf3
    classDef base fill:#262626,stroke:#8b949e,color:#e6edf3,stroke-dasharray:4 3

API: detector_likelihood(), asr_likelihood(), direction_likelihood()
Tutorial: Fusing vision, speech and gaze
Origin: Published method: implements the cited method as published.
Category: Classical: classical statistics, decision theory or baselines.
Reference: Fisher, R. A. (1953). Proceedings of the Royal Society A, 217, 295-305.

Fusion rules#

Published model, extended Mixed

Combine cues three ways and compare them on held-out data.

        flowchart LR
    accTitle: Block diagram of Fusion rules
    accDescr: Combine cues three ways and compare them on held-out data.
    L["cue likelihoods"]:::in --> B["Bayes: product"]:::base
    L --> DS["Dempster-Shafer:<br/>masses + ignorance"]:::base
    L --> Q["quantum-like:<br/>cue bases, order"]:::op
    B & DS & Q --> cmp["held-out log loss"]:::out
    classDef in fill:#0b2a4a,stroke:#79c0ff,color:#e6edf3
    classDef op fill:#2a1b3d,stroke:#d2a8ff,color:#e6edf3
    classDef out fill:#0f2e1a,stroke:#7ee787,color:#e6edf3
    classDef hum fill:#3d1414,stroke:#ff7b72,color:#e6edf3
    classDef base fill:#262626,stroke:#8b949e,color:#e6edf3,stroke-dasharray:4 3

API: bayes_fusion(), dempster_shafer_fusion(), compare_fusion()
Compared with: Bayes, Dempster-Shafer
Tutorial: Fusing vision, speech and gaze
Origin: Published model, extended: implements the cited model as published; Quantum Mind adds the capabilities listed.
Category: Mixed: combines parts from more than one of these categories.
Quantum Mind adds: a quantum-like (Kraus) fusion rule designed here, next to the Bayesian and Dempster-Shafer rules.
Reference: Shafer, G. (1976). A Mathematical Theory of Evidence. Princeton University Press.

Cue incompatibility#

Unique to Quantum Mind Quantum-like

Fit the angles between cue bases from trials presented in different cue orders.

        flowchart LR
    accTitle: Block diagram of Cue incompatibility
    accDescr: Fit the angles between cue bases from trials presented in different cue orders.
    t[("trials: cue order → choice")]:::hum --> f["maximise likelihood<br/>over angles"]:::op --> th["angle per cue"]:::out
    classDef in fill:#0b2a4a,stroke:#79c0ff,color:#e6edf3
    classDef op fill:#2a1b3d,stroke:#d2a8ff,color:#e6edf3
    classDef out fill:#0f2e1a,stroke:#7ee787,color:#e6edf3
    classDef hum fill:#3d1414,stroke:#ff7b72,color:#e6edf3
    classDef base fill:#262626,stroke:#8b949e,color:#e6edf3,stroke-dasharray:4 3

API: fit_incompatibility()
Tutorial: Fusing vision, speech and gaze
Origin: Unique to Quantum Mind: designed in this library; it builds on the methods cited with it.
Category: Quantum-like: quantum probability used to model people; runs on an ordinary computer.

Dwell-time baselines#

Published model, extended Classical

Constant switching rate (exponential dwell times) and the gamma renewal model used in the perception literature.

        flowchart LR
    accTitle: Block diagram of Dwell-time baselines
    accDescr: Constant switching rate (exponential dwell times) and the gamma renewal model used in the perception literature.
    r["rate λ"]:::in --> ex["exponential dwell times"]:::base --> d["dwell-time histogram"]:::out
    ks["shape k, scale θ"]:::in --> ga["gamma dwell times"]:::base --> d
    classDef in fill:#0b2a4a,stroke:#79c0ff,color:#e6edf3
    classDef op fill:#2a1b3d,stroke:#d2a8ff,color:#e6edf3
    classDef out fill:#0f2e1a,stroke:#7ee787,color:#e6edf3
    classDef hum fill:#3d1414,stroke:#ff7b72,color:#e6edf3
    classDef base fill:#262626,stroke:#8b949e,color:#e6edf3,stroke-dasharray:4 3

API: MarkovSwitchingModel, GammaRenewalModel
Tutorial: Bistable perception and the quantum Zeno effect
Origin: Published model, extended: implements the cited model as published; Quantum Mind adds the capabilities listed.
Category: Classical: classical statistics, decision theory or baselines.
Quantum Mind adds: bootstrap intervals, a per-person posterior updated after every answer, and the choice of the most informative condition (what Quantum Mind adds).
Reference: Brascamp, J. W., et al. (2006). Journal of Vision, 6, 1244-1256.