Planning which question to ask#

The robot sees three objects (red cup, blue cup, red bowl) and hears “bring me that one”. It can ask yes/no questions (“is it red?”, “is it a cup?”, “is it on the left?”) before acting. Which question should it ask first, and when should it stop? The question planner computes the value of information of each question: the expected reduction in the cost of the final decision, minus the cost of asking.

The twist: people’s answers depend on what was asked before. A projective answer model captures this; an independent model (the classical default) ignores it. Both are compared on simulated people.

import numpy as np
import matplotlib.pyplot as plt
from quantum_mind.viz import use_mpl_style
from quantum_mind.envs.hri import default_domain
from quantum_mind.applications.questioning import QuestionPlanner
theme = use_mpl_style('dark')
person = default_domain()          # a simulated person whose answers follow a projective model
print('hypotheses:', person.hypotheses, '   questions:', person.questions)

hypotheses: ['red cup', 'blue cup', 'red bowl']    questions: ['red?', 'cup?', 'left?']

1. How informative is each question?#

planner = QuestionPlanner(person, ask_cost=0.3, error_cost=5.0)
voi = {q: planner.value_of_information(q, []) for q in person.questions}
gain = {q: planner.information_gain(q, []) for q in person.questions}
for q in person.questions:
    print('%-6s value of information %.3f   information gain %.3f bits' % (q, voi[q], gain[q]))
print('decision now:', planner.decide([]))

red?   value of information 0.476   information gain 0.123 bits
cup?   value of information 0.781   information gain 0.169 bits
left?  value of information 1.408   information gain 0.448 bits
decision now: ('ask', 'left?', 1.4076866816945368)

fig, ax = plt.subplots(figsize=(6.0, 2.8))
x = np.arange(len(voi))
ax.bar(x - 0.18, list(voi.values()), 0.36, label='value of information', color=theme['palette'][1])
ax.bar(x + 0.18, list(gain.values()), 0.36, label='information gain (bits)', color=theme['palette'][2])
ax.set_xticks(x, list(voi)); ax.set_title('Which question first?'); ax.legend()
plt.show()
Figure: Which question first?

2. Order dependence#

The probability of a “yes” to one question depends on the answer to an earlier one, and on the order. The independent model cannot represent this.

blind = person.order_free()
for h in person.hypotheses:
    a = person.likelihood(h, [('red?', 1), ('cup?', 1)])
    b = person.likelihood(h, [('cup?', 1), ('red?', 1)])
    print('%-9s P(yes, yes): red first %.3f, cup first %.3f   (independent model: %.3f both)'
          % (h, a, b, blind.likelihood(h, [('red?', 1), ('cup?', 1)])))

red cup   P(yes, yes): red first 0.097, cup first 0.019   (independent model: 0.017 both)
blue cup  P(yes, yes): red first 0.010, cup first 0.174   (independent model: 0.015 both)
red bowl  P(yes, yes): red first 0.003, cup first 0.162   (independent model: 0.004 both)

3. Does modelling the order help?#

Run both planners against the same simulated people, many times, for each target object.

rng = np.random.default_rng(0)
aware, naive = QuestionPlanner(person, ask_cost=0.3, error_cost=5.0), QuestionPlanner(blind, ask_cost=0.3, error_cost=5.0)
res = {'order-aware': [], 'independent': []}
for i in range(300):
    target = person.hypotheses[i % 3]
    for name, pl in (('order-aware', aware), ('independent', naive)):
        out = pl.simulate(person, target, np.random.default_rng(i))
        res[name].append((out['total_cost'], out['correct'], len(out['history'])))
for name, r in res.items():
    r = np.array(r, float)
    print('%-12s mean cost %.3f   accuracy %.3f   questions asked %.2f' % (name, r[:, 0].mean(), r[:, 1].mean(), r[:, 2].mean()))

order-aware  mean cost 2.217   accuracy 0.617   questions asked 1.00
independent  mean cost 2.390   accuracy 0.603   questions asked 1.36

fig, ax = plt.subplots(figsize=(6.0, 2.8))
for (name, r), c in zip(res.items(), theme['palette']):
    ax.hist(np.array(r)[:, 0], bins=np.arange(0, 6, 0.3), alpha=0.6, color=c, label=name)
ax.set_xlabel('total cost of the episode'); ax.set_ylabel('episodes'); ax.legend()
ax.set_title('Episode costs on simulated people')
plt.show()
Figure: Episode costs on simulated people

The statements below are checked by this cell every time the documentation is built:

mean_cost = {k: np.array(v, float)[:, 0].mean() for k, v in res.items()}
assert mean_cost['order-aware'] <= mean_cost['independent']
print('checked')

checked

Result. In this domain the order-aware planner has a lower average cost than the planner that treats answers as independent; the printed numbers give the size of the gain. The gain is not universal: across randomly drawn domains it depends on how strongly the questions interfere, and in some domains it is close to zero.

References: Howard (1966); Wang & Busemeyer (2013), Topics in Cognitive Science 5, 689-710.