01 / Experiment
Each blue trace is a different neural network.
Network architecture
Every neuron receives values from the preceding layer, multiplies them by learned weights, adds a bias, and applies the activation function. “More neurons” and “more layers” both increase capacity, but they do not add the same kind of capacity.
The output neuron uses sigmoid to produce a value between 0 and 1. When it rises above 0.52, the bird jumps; otherwise gravity continues pulling it down.
This is a direct action score, not a prediction of height or distance. The live brain highlights the output whenever the jump threshold is crossed.
Gives that layer more parallel detectors. Several neurons can specialize in different situations—for example: falling quickly, approaching a gap from below, or already being safely centered.
Adds stages of reasoning. Later layers can combine patterns found by earlier layers, building a more composed rule before the final flap decision.
4 → 6 → 1 means four observations, six hidden neurons, and one flap probability. Entering 4 3 2 creates three hidden layers: 4 → 4 → 3 → 2 → 1.
Changing 4 → 4 → 1 to 4 → 8 → 1 adds width and grows the genome from 25 to 49 genes. Adding depth with 4 → 4 → 3 → 2 → 1 creates 46 genes.
Extra capacity can express subtler strategies, but evolution must discover and preserve more useful weights. With only four inputs and one decision, Flappy Bird often needs a small network rather than the largest one.
Parent selection
The algorithm never edits a bird because it made a mistake. It only changes how likely each completed network is to become a parent. Stronger pressure exploits current winners; weaker pressure preserves diversity.
Sample four birds at random and keep the best. Repeat whenever a parent is needed. It strongly favors good birds without letting one score monopolize every child.
Give each bird a probability equal to its share of total fitness. A bird with twice the fitness receives roughly twice the chance, so outliers can dominate.
Sort birds first, then assign probability by position instead of raw score. The distance between scores no longer matters, producing steadier pressure.
Genome crossover
Weights and biases are flattened into a genome. Both parents share the same architecture, so corresponding positions describe the same connection. Crossover combines those positions before mutation.
Flip a coin at every gene. This explores many combinations, but can separate groups of weights that only work well together.
Choose one split and copy the left side from one parent and the right side from the other. Blocks remain intact, although the split ignores network boundaries.
Set every child gene halfway between both values. Changes are smooth and conservative, but an especially useful extreme value may be diluted.
Mutation
After crossover, every gene receives an independent chance to change. Selected genes receive Gaussian noise, so small changes are common and large jumps are possible but rare.
if random() < rate: gene′ = gene + Normal(0, strength)
The percentage of weights and biases changed in each child. At 10%, a 100-gene network changes about ten genes on average.
The standard deviation of the added noise. At 0.36, roughly two thirds of changes lie between −0.36 and +0.36.
Too little can freeze the population around a mediocre strategy. Too much destroys useful inherited structure and approaches random search.
Activation functions
Without an activation function, stacked layers collapse into one linear transformation. Nonlinearity lets the network express rules such as “flap only when low, falling, and near a pipe.”
Maps values to −1…1. It is smooth and zero-centered, making positive and negative internal signals natural. Large inputs saturate near the limits.
Returns zero below zero and the original value above it. It does not saturate positively, but a neuron receiving only negative inputs becomes inactive.
Maps values to 0…1. It is interpretable as a gate but not zero-centered and becomes insensitive at both extremes. The final output always uses sigmoid so it represents a flap probability.
Fitness function
A genetic algorithm can only optimize the number it receives. The current function rewards surviving every frame and adds a much larger step whenever the bird clears a pipe.
fitness = survival frames + (pipes passed × 480)
Early random birds may clear no pipes. One point per frame still distinguishes immediate crashes from behavior that approaches the first gap.
At 60 steps per second, a pipe is worth eight seconds of survival. Clearing obstacles therefore dominates merely hovering safely.
A bird surviving 300 frames and clearing two pipes earns 300 + 2 × 480 = 1,260. There is no gap-centering reward, flap penalty, or death penalty.
Optimization outside the model
Evolution learns the network’s weights and biases. Hyperparameter optimization sits one level above that process: it runs several complete experiments with different settings, then compares how well those settings generalize.
These values live inside each brain. Selection, crossover, and mutation change them throughout a run.
Architecture, activation, selection, crossover, mutation rate, and mutation strength are chosen before evolution begins.
best setup = arg max mean unseen fitness(setup, evaluation seeds)
Compare configurations using the same population size, generation budget, and held-out pipe seeds. One lucky run is not enough; average several independent runs.
Grid search checks a fixed lattice. Random search samples combinations and often covers large spaces more efficiently. Adaptive methods use earlier trials to decide which settings to test next.
Use training runs to evolve weights, validation runs to choose hyperparameters, and a final untouched test set to estimate performance. Reusing the test set simply moves overfitting up one level.
Model compression
A dense network gives every neuron every possible incoming connection. After training, many weights may contribute little. Pruning sets selected weights to zero, creating a smaller effective network whose behavior should be measured again.
prune connection if |weight| < threshold
Remove weights closest to zero first. It is simple and often effective, but magnitude is only a proxy: a small weight can still matter when its input is large.
Unstructured pruning removes individual connections. Structured pruning removes whole neurons or layers, producing a network that is easier to make genuinely smaller and faster.
Prune gradually, test on held-out pipe patterns, then allow a short recovery evolution. Stop when additional sparsity causes an unacceptable performance drop.
Generalization
A network can become excellent on the situations it repeatedly sees while becoming worse at unfamiliar ones. Training performance alone cannot reveal this—the model must be evaluated on data or environments that were not used to evolve it.
Training uses one seeded sequence of five pipe heights, repeated forever. Evolution can gradually exploit the order and timing of that exact loop.
Testing freezes the best trained brain and replaces the loop with genuinely random pipe heights. No further selection, crossover, or mutation occurs during the test.
If the fixed-pattern score keeps improving while the unseen-pattern score stalls or falls, the growing difference is evidence of overfitting.
This simulator has its own population and state. It does not replace or reset the random-environment simulation. It uses a snapshot of the current architecture and evolution parameters when training begins.
Start training to evolve on the repeating five-pipe sequence.
Run at least one training generation to unlock the test.
TODO