CS188 Games

Also known as Adversarial Search Problems

Deterministic zero-sum games

Components

• Initial state
• Players
• Actions
• Transition model
• Terminal test
• Terminal values(Utility)
• State Value

State value is the best possible outcome (utility) an agent can achieve from that state

$\forall$ non-terminal states, $V(S)=ma{x}_{s\in successors(s)}{V}_s$
$\forall$ terminal states, $V(S)=known$

Minimax

$\forall$ agent-controlled states, $V(S)=ma{x}_{s\in successors(s)}{V}_s$
$\forall$ opponent-controlled states, $V(S)=mi{n}_{s\in successors(s)}{V}_s$
$\forall$ terminal states, $V(S)=known$

In implementation, minimax behaves similarly to postorder traversal depth-first search.

Alpha-Beta Pruning

$x$ is the terminal value need to be looked up
$$\beta \ge x\ge \alpha$$

Still need to reach the bottom.

Evaluation Functions

Evaluation function takes a state, and output the evaluated state value.

Most common form:

$$Eval(s)=\sum _i{w}_i{f}_i(s)$$

weights: $w_i$,
feature: $f_i$

Expectimax

Chance nodes:

Instead of considering the worst case as minimizer nodes do, considers the average case.

$\forall$ agent-controlled states, $V(S)=ma{x}_{s^{\prime }\in successors(s)}{V}_{s^{\prime }}$
$\forall$ chance states, $V(S)=mi{n}_{s^{\prime }\in successors(s)}P(s^{\prime }|s)V_{s^{\prime }}$
$\forall$ terminal states, $V(S)=known$

Mixed Layers Types

Many Players

Monte Carlo Tree Search

For games with a large branching factor

• Evaluation by rollouts
• Selective search

UCB Algorithm:
$$UC{B}_1(t)=\frac{U(n)}{N(n)}+C\times \sqrt{\frac{logN(parent(n))}{N(n)}}$$

When $N(n)$ to $+\infty$, MCTS approaches Minimax.

General Games

Multi-agent utilities: using a tuple to represent the different utility values.

Summary

• Minimax: when opponents behave optimally
• Expectimax: when opponents behave sub-optimally
• Monte Carlo Tree Search: when a large branching factor
• General game: using tuples.