[Paper] Stringological sequence prediction I
First major step in 'compositional learning' program offers efficient algorithms for sequence prediction, bypassing deep learning.
A new research paper, the first in a planned series, marks a significant advance in the 'compositional learning' program, aiming to close the long-standing gap between abstract agent foundations theory and practically relevant algorithms. The work proposes novel sequence prediction algorithms inspired by 'stringology'—the study of strings and sequences—that are both computationally efficient and come with theoretical guarantees. Their performance is bounded by specific complexity measures of the data sequence, such as the size of the smallest straight-line program that can generate it. This directly targets a core criticism of fields like learning-theoretic alignment (LTA), which have historically been split between simplistic toy models and computationally infeasible ideals like AIXI.
The algorithms are designed to exploit compositional patterns in data to achieve efficiency in highly expressive hypothesis classes. The research focuses on sequences with low 'stringological' complexity, which includes rich classes studied in combinatorics of words, like automatic and morphic sequences. The implications are multi-faceted: conservatively, it enables more realistic 'toy' models of agents that reason with Occam's razor; it might provide a testable mathematical model for the generalization power of deep learning; and, most ambitiously, it charts a potential path to building practical AI that doesn't rely on deep learning. Notably, this breakthrough was enabled by synthesizing computational learning theory with disparate fields like stringology and combinatorics on words.
- Proposes first practical algorithms bridging agent foundations theory (like AIXI) with computational feasibility, a major gap in the field.
- Algorithms predict sequences based on 'stringological' complexity measures (e.g., smallest straight-line program size) and come with efficiency guarantees.
- Represents the inaugural step in the 'compositional learning' program, with potential to model deep learning or create AI that bypasses it.
Why It Matters
Provides a concrete, efficient bridge from abstract AI theory to practical algorithm design, potentially unlocking new paths to machine intelligence.