sketch_epsilon_greedy.nx
buildroot/runtime/sketch_epsilon_greedy.nx
about
sketch_epsilon_greedy.nx -- Epsilon-Greedy bandit (Sutton-Barto, classic).
Simpler alternative to UCB1. At each pull:
with probability epsilon: pull a UNIFORM-RANDOM arm
with probability 1-epsilon: pull arm with highest empirical mean
Three modes via the epsilon_mode field:
NX_EG_MODE_FIXED -- epsilon constant (param: epsilon_ppm)
NX_EG_MODE_DECAY_LINEAR -- epsilon(t) = epsilon_0 / (t+1)
NX_EG_MODE_DECAY_LOG -- epsilon(t) = epsilon_0 / log(t+e)
COMPLEMENTS UCB1:
- UCB1: deterministic confidence-bound; near-optimal regret;
requires sqrt + log per arm per pull
- epsilon-Greedy: stochastic; simpler; tunable explore rate;
preferred when log/sqrt are expensive (embedded) or when
bounded exploration budget needed
Together they cover the "classic bandit" textbook coverage.
LCG-based deterministic RNG so two runs with same seed produce
byte-identical action sequences.
LOSSLESS-LANGUAGE DISCIPLINE: nx_eg_query reports the running
mean of an arm with stderr ~ 1/sqrt(count_i).
dependencies 2 imports · 1 importers
imports: syscalls.nxsketch_types.nx
imported by: sketch_epsilon_greedy_test.nx
structs
| 42 | struct EpsilonGreedy { |
consts
| 29 | const NX_EG_MIN_ARMS: i64 = 2 |
| 30 | const NX_EG_MAX_ARMS: i64 = 10000 |
| 32 | const NX_EG_MODE_FIXED: i64 = 0 |
| 33 | const NX_EG_MODE_DECAY_LINEAR: i64 = 1 |
| 34 | const NX_EG_MODE_DECAY_LOG: i64 = 2 |
| 36 | const NX_EG_LCG_A: i64 = 1103515245 |
| 37 | const NX_EG_LCG_C: i64 = 12345 |
| 38 | const NX_EG_LCG_MOD: i64 = 0x7FFFFFFF |
| 40 | const NX_EG_REWARD_MAX: i64 = 1000000 |
functions
| 54 | func nx_eg_alloc(n_arms: i64, mode: i64, epsilon_ppm: i64, seed: i64) -> *EpsilonGreedy {
called by 1: main |
| 79 | func nx_eg_rng_next(eg: *EpsilonGreedy) -> i64 {
called by 1: nx_eg_rng_below |
| 85 | func nx_eg_rng_below(eg: *EpsilonGreedy, n: i64) -> i64 { |
| 92 | func nx_eg_bitlen(x: i64) -> i64 {
called by 1: nx_eg_current_epsilon_ppm |
| 105 | func nx_eg_current_epsilon_ppm(eg: *EpsilonGreedy) -> i64 { |
| 129 | func nx_eg_best_arm_by_mean(eg: *EpsilonGreedy) -> i64 { |
| 150 | func nx_eg_select(eg: *EpsilonGreedy) -> i64 { |
| 163 | func nx_eg_update(eg: *EpsilonGreedy, arm: i64, reward_ppm: i64) -> i64 {
called by 1: main |
| 176 | func nx_eg_mean_ppm(eg: *EpsilonGreedy, arm: i64) -> i64 { |
| 181 | func nx_eg_count(eg: *EpsilonGreedy, arm: i64) -> i64 {
called by 1: main |
| 185 | func nx_eg_best_arm(eg: *EpsilonGreedy) -> i64 { |
| 191 | func nx_eg_isqrt(x: i64) -> i64 {
called by 1: nx_eg_query |
| 208 | func nx_eg_query(eg: *EpsilonGreedy, arm: i64) -> *ApproxI64 { |
| 222 | func nx_eg_memory_bytes(eg: *EpsilonGreedy) -> i64 { |