No matter how esoteric AI literature has become, and no matter how thoroughly the intellectual origins of AI's technical methods have been forgotten, the technical work of AI has nonetheless been engaged in an effort to domesticate the Cartesian soul into a technical order in which it does not belong. The problem is not that the individual operations of Cartesian reason cannot be mechanized (they can be) but that the role assigned to the soul in the larger architecture of cognition is untenable. This incompatibility has shown itself up in a pervasive and ever more clear pattern of technical frustrations. The difficulty can be shoved into one area or another through programmers' choices about architectures and representation schemes, but it cannot be made to go away.

From Phil Agre's 1995 article The Soul Gained And Lost.

If one were to continue the genealogy in this article from 1995 to present, one would find many of the same issues inherent in Cartesian dualism present in large language models. Like the STRIPS system Agre surveys, LLMs also generate sequences. They also must make choices among many available options at each step of sequence generation. They also use heuristics to guide this process that would otherwise explode intractably. The heuristics, or what Agre dubs "determining tendency", are random number generators and "guardrails" in LLMs instead of the tree-structured search of previous-generation AI systems. But otherwise the systems are structured similarly.

It's fascinating, but not coincidental, that the determining tendency of AI systems like these is so often perceived to have mystical or even God-like qualities. Breathless predictions about the endless potential of tree-structured search in early writing on GOFAI resembles modern proclamations of imminent AGI or superintelligence among generative AI boosters because both of these mechanisms---tree search or random number generation---are situated where the Cartesian soul would be. These mysterious determining tendencies, homunculuses of last resort, or souls are timeless, acausal factors that choose a single path from an infinite space of possibilities, and thereby direct the encompassing agent's behavior in an intelligent manner.

This is one reason why I posted the other day that if you removed the random number generation from LLMs, the illusion of their intelligence would more than likely quickly evaporate. You'd be excising their soul, leaving behind a zombie!

#AI #GenAI #GenerativeAI #LLM #GOFAI #search #heuristics #CartesianDualism #IntelligenceAsRandomNumberGeneration