TinyVLA model

A tiny vision-language-action model for one motor, trained end to end in NumPy.

Three slots with explicit interfaces, so each can be replaced later:

• grounding (vision + language): ground(image, text) returns a distribution over table cells, “which place does the instruction mean”. It is a single attention step: the instruction forms a query, every image cell a key.

• belief / ask gate: lives in quantum_mind.embodied.policies; it reads that distribution, not just its argmax.

• action: act(target, theta, omega) maps the expected target and the noisy motor state to a command.

fit trains grounding and action together by behaviour cloning (cross-entropy on the intended cell, squared error on a PD demonstrator’s command); gradients are written by hand and checked against finite differences in the tests.

Classes

TinyVLA	Grounding by attention plus a small action head (about 1.3k parameters at the defaults).
BCBaseline	The usual end-to-end behaviour-cloning policy: image, instruction and state in, command out (one hidden layer).

Functions

make_dataset

Scenes plus, for each, one random motor state and the demonstrator's command.

class TinyVLA(n_cells=16, d=16, hidden=24, seed=0)

Bases: object

Grounding by attention plus a small action head (about 1.3k parameters at the defaults).

ground(image, text)

Distribution over cells for the instruction in the image. Accepts one scene or a batch.

act(target, theta, omega)

Motor command in [-1, 1] for a target position and the measured state.

class BCBaseline(n_cells=16, hidden=64, seed=0)

Bases: object

The usual end-to-end behaviour-cloning policy: image, instruction and state in, command out (one hidden layer).

No grounding bottleneck, no distribution over cells, so it can neither be calibrated per place nor ask.

make_dataset(n_scenes, seed, n_cells=16)

Scenes plus, for each, one random motor state and the demonstrator’s command.