Compute is the fuel of system evolution, and data plus algorithm provide the
direction.
Language models might mark the beginning of a new programming paradigm.
Traditional programming tightly couples code and system behavior, allowing us
to directly infer system decisions by examining the code. However, language
models differ greatly.
Instead, we use benchmarks to roughly judge and understand the
model's
behavior. The intelligence and actions of these models
are largely independent
from their code, emerging naturally
during the training process. I believe this
approach to system
programming may represent the early stages of a
fundamentally
new programming paradigm, where systems evolve
autonomously.
In this paradigm, we provide only high-level signals and metrics to align the
system broadly with human intentions. Even these high-level directions should
ideally be simple, granting the system greater flexibility to evolve independently.
Consider the future of games as an example. Game settings, environments,
physics laws, and even the purpose of the game itself might no longer be
explicitly defined in the game's programming logic, but rather exist in an
evolved state.
Computational power may become the essential resource fueling this evolution.
Today's language models may thus represent just the start of a new programming
approach, with future GPU resources or other methods continuing to drive
system evolution.