Correlated Behaviours & Environment Selection — Prosser (2022)

A from-scratch replication of two computational models from:

Prosser, D. (2022). The Interaction Between Lifetime Learning and Evolution. PhD thesis, University of Southampton, School of Electronics and Computer Science.

Nearly every prior model of the Baldwin effect — including Hinton & Nowlan (1987) and Ackley & Littman (1991) — treats behavioural/genetic traits as independent: each locus, each weight, is mutated and assimilated on its own. Prosser's thesis asks what changes when learning acts on correlated groups of traits instead, with the correlation structure itself evolving. The answer, across two different causal mechanisms, is: a lot.

This repo replicates the thesis's two headline models:

• Correlated Behaviours (CB) — learning directly tests combinations of innate behaviours together, biased by their evolving genetic correlation. Produces a genuine Baldwin Optimizing Effect: with learning, evolution reliably reaches the global fitness optimum; without it, only local optima are found, and not consistently.
• Environment Selection (ES) — learning never touches behaviour directly. It only selects which of several candidate environments the individual resides in, indirectly changing which selection pressure acts on its genetically-determined behaviour. Produces a Baldwin Expediting Effect (learning accelerates evolution) but not an Optimizing Effect (evolution alone eventually gets there too, just slower).

Both models find that the classical necessary conditions for genetic assimilation — a cost of learning, a correlation between genotype-space and phenotype-space movement, a stable learnt target — are not required once learning acts on correlated traits.

The mechanism

Both models share the same substrate: a population-mean correlation matrix B (n x n, one entry per pair of traits), mutated under a strong-selection-weak-mutation (SSWM) regime — one mutation tested at a time, kept only if it doesn't reduce fitness. They're evaluated against two synthetic multi-peaked fitness landscapes borrowed from prior work (prosser/landscapes.py):

• Modular Constraints (MC) — m blocks of k traits. Strong (1.0) connections within a block, weak positive/negative (p) connections between blocks depending on whether the blocks' target signs agree. Local optima exist wherever a block's own traits are internally resolved but disagree with other blocks; there are exactly two global optima (the target phenotype and its sign-flipped complement).
• Concentric Squares (CS) — target is an image of concentric alternating-sign square rings; connections are purely local (a cell's 4 grid neighbours). No explicit building-block structure, used as a counterpoint to MC's clean block hierarchy.

Fitness is w(p) = p·C·pᵀ (quadratic in the phenotype, so it rewards correctly-signed combinations of traits, not single traits in isolation) — C is the landscape's constraints matrix.

Correlated Behaviours (prosser/correlated_behaviours.py)

• B starts as the identity matrix (no correlation beyond self).
• Each generation, n mutations are tested (not n² — the paper notes this is ~100x fewer fitness evaluations than the ESDP model it builds on).
• Learning runs η trial-and-error steps. Each trial picks a random focal behaviour c; every other behaviour y whose correlation with c exceeds a random threshold gets its sign flipped to match c ("trait flipping"), scaled by the correlation strength. With learning, a trial's change is kept only if it's fitter; without learning, every trial's change is kept unconditionally (so "no learning" still expresses correlated behaviours, it just never discards a worse combination).
• As correlations strengthen through evolution, learning naturally scales up from flipping single behaviours to flipping whole correlated blocks at once — which is what lets it escape local optima that trial-and-error on independent traits can't.

Environment Selection (prosser/environment_selection.py)

• B starts at zero (complete absence of correlation).
• Each generation, n² mutations are tested.
• Learning samples m candidate environments and picks whichever one, once the genetically-determined behaviour has developed in it (η steps of p ← (1-τ)p + tanh(pB)), gives the highest fitness. Without learning, only one candidate is available (m=1) — there's no selection to do.
• Because genetic correlations and learning operate in genuinely different spaces (correlation matrix vs. environment vector), the usual test for "has learning been genetically assimilated" — comparing genotype and learnt-phenotype configurations directly — doesn't apply here. The thesis treats the shrinking diversity of phenotypes produced over evolutionary time as the relevant signal instead.

What's matched vs. simplified

This is a much higher-fidelity replication than typical "based on the paper" reimplementations, because the thesis publishes every equation used (unlike Ackley & Littman 1991, which leaves most constants unpublished). Every mutation rule, learning-trial rule, and fitness function here is implemented directly from the thesis's own numbered equations (§4.1, §5.3, §6.2-6.3, §7.2-7.3), not reconstructed from a verbal description.

One genuine correction made during implementation: the thesis's Modular-Constraints target-phenotype formula (t_i = 1 if i mod(2k) ≥ k) is written with 1-indexed i but only produces the clean alternating block structure described in the text (and shown in Figure 4.1) when i is 0-indexed — confirmed directly (tests/test_landscapes.py) before building anything on top of it.

Not implemented: the thesis's noise-in-fitness experiments (§5.3.4's ψ term — present in landscapes.fitness but not exercised by any run script here), the rHN-G connection-density variant beyond the basic version tested in correlated_behaviours.evolve(connection_density=...), and the Environment Selection model's Figure 7.11-style analysis of which correlation type (within-block vs. between-block) learning favours over evolutionary time.

Running it

pip install -r requirements.txt

# Correlated Behaviours, Modular Constraints problem (paper's own default scale)
python run_cb.py --n-blocks 10 --block-size 10 --generations 10000 --with-learning --seed 1
python run_cb.py --n-blocks 10 --block-size 10 --generations 10000 --no-learning --seed 1

# Environment Selection, Modular Constraints problem
python run_es.py --n-blocks 10 --block-size 6 --generations 10000 --with-learning --seed 1

Run the test suite with:

pip install -r requirements.txt pytest
pytest tests/ -q

Performance note: Correlated Behaviours is cheap (n mutations/ generation) and runs at paper scale in minutes. Environment Selection is expensive (n² mutations/generation, each requiring m fitness evaluations) — the paper's own scale (n=60, up to 25,000 generations, 30 seeds) is a multi-hour undertaking; run_es.py's defaults are tuned smaller for a quick, still-qualitatively-correct run. See RESULTS.md for the reduced-scale validation runs actually performed, and the paper-scale estimate for reproducing the published figures exactly.

Results

See RESULTS.md. Summary: both models' headline qualitative findings reproduce cleanly at reduced scale — Correlated Behaviours reaches the exact global optimum and holds it, while its no-learning control stays noisy and well below optimum; Environment Selection shows learning reaching a high-fitness plateau far faster than its no-learning control, without a hard guarantee of optimality either way.

Relation to other repos

• HintonNowlan1987 — Hinton & Nowlan (1987), the theoretical precursor both this thesis and Ackley & Littman build on.
• AckleyLittman1991 — Ackley & Littman (1991), the per-agent genome/network architecture this thesis's correlation-structure question was originally motivated by (see that repo's README for the connection).
• PredPreyGrass's predpreygrass/evolutionary/discrete_space/eco_evolutionary_erl_baldwin/ module is where the architecture these ideas point toward gets pushed into a real multi-agent ecology, rather than a synthetic landscape. This repo is the faithful, standalone replication baseline — not a component of that module.

References

• Prosser, D. (2022). The Interaction Between Lifetime Learning and Evolution. PhD thesis, University of Southampton.
• Watson, R. A., Mills, R., & Buckley, C. L. (2011). Global adaptation in networks of selfish components: Emergent associative memory at the system scale. Artificial Life, 17(3), 147-166. (Source of the rHN-G generative-dynamical-system concept the Correlated Behaviours model adapts.)
• Kounios, L., Clune, J., Kouvaris, K., Wagner, G. P., Pavlicev, M., Weinreich, D. M., & Watson, R. A. (2016). Resolving the paradox of evolvability with learning theory: How evolution learns to improve evolvability on rugged fitness landscapes. arXiv:1612.05955. (Source of the Concentric Squares problem.)
• Hinton, G. E., & Nowlan, S. J. (1987). How Learning Can Guide Evolution. Complex Systems, 1(3), 495-502.
• Ackley, D. H., & Littman, M. L. (1991). Interactions Between Learning and Evolution. In Artificial Life II, 487-509. Addison-Wesley.