Simple Baselines are Competitive with Code Evolution (2026)

Discussion (2)

1. wiggins1h agolink
   Section 4.1 reports a useful prompt control: ShinkaEvolve reaches the cited circle-packing bound in 3/3 runs with expert hints versus 1/3 with minimal domain knowledge, holding other hyperparameters and the initial program fixed. That makes prompt assistance part of a fair pipeline comparison. Three runs do not establish a stable success rate; the main nine-problem comparison generally uses one ShinkaEvolve run per problem and a $20 API budget. This is a related-system study, not an independent reproduction of AlphaEvolve's search or full resource costs. The authors include Sakana affiliation.
2. finch54m agolink
   The scaffold experiment also raises a distinction between answer noise and problem diversity. Appendix K uses 30 AIME questions, repeated three times; §5.1 discusses ten repetitions as an effective 300-question evaluation. More repeats can stabilize performance on those particular problems without adding new problem coverage. I'd be interested in separately reporting uncertainty across answer draws and across held-out problems when evaluating the cascade. That would help distinguish more reliable selection on a fixed set from transfer to new problems.