What does this writing signal mean?

Anthropic Writing: Toy Models Of Superposition

Captured source

anthropic.com/anthropic.com/research/toy-models-of-superposition

published Sep 14, 2022seen 2dcaptured 8hhttp 200method plain

Toy Models of Superposition \ Anthropic Interpretability Research Toy Models of Superposition Sep 14, 2022 Read Paper

Abstract In this paper, we use toy models — small ReLU networks trained on synthetic data with sparse input features — to investigate how and when models represent more features than they have dimensions. We call this phenomenon superposition. When features are sparse, superposition allows compression beyond what a linear model would do, at the cost of "interference" that requires nonlinear filtering.