As it happens I’m working on a chessdb.nu, which is supposed to be a set of tools and skills for llms to annotate, even Socratically coach chess positions and games.

Two observations, 1) I think the post is really just trying to explain that the LLM is not limited to the next token in its planning. It can have an idea of where it is going all the way to the last period and token prediction is just the step by step to get there. Even more interesting is that the LLM can change plans midstream (cf: seahorse emoji). And 2) without going so far as saying it is existentially making choices, it is showing complex reasoning and planning which is a few steps above a Markov Chain.

As for myself, I like the analogy of french curves and splines being fixed and added to with the LLMs internal reasoning and then walked token by token. But LLM’s don’t actually have to be that continuous and smooth, although in fairness when it isn’t, it’s usually a failure mode.

But if you are even a little bit educated as to what the textbooks say about LLMs, they are exactly next-token predictors. There’s no ifs-or-buts about it.

Your chess analogy is misinformed too. Your description of neural networks trained on past master games reads like the machine is given pictures of the board and simply learns from that. It also assumes that is how modern day chess engines are written which is not completely true. It is called the concept of “Book” which records past moves in certain positions as a cache. And people don’t call that “Next Token Prediction”. It’s not as sophisticated… it’s simply a lookup table or a cache.

Actually good Chess Engines are search engines walking over a tree or graph of possible next moves. One can argue this is actually a prediction engine… but “Prediction Engine” as a description feels like the machine is rolling the dice far more than a search… which is the part that’s misleading.

Here is my claim: LLMs as they stand today are fundamentally less sophisticated than a Chess Engine since the search part is absent from LLMs. This is not because LLMs are less powerful, no. Their domain is simply different from search engines. Token prediction IS the job description of an LLM. Which is why LLMs are not very good at chess.