ChatGPT is a Blurry JPEG and We Need That

Blurry JPEG is a thought-provoking article by science fiction legend Ted Chang. Ted makes an interesting analogy between large-scale language models (LLMs) like ChatGPT and lossy compression algorithms like blurry JPEG images. Ted argues that LLMs provide only a superficial understanding of a topic and can ultimately hinder the pursuit of proper knowledge. I disagree with the conclusion of this article. We need to recognize that even blurry images have value for learning and knowledge acquisition because taking data from the real world and processing it is very time-consuming. Sometimes we need less detailed but more efficient representations to facilitate faster understanding and progress.

If we squint, learning is ultimately a process of fetching data from the world, just like a computer retrieves information from data storage. If that's the case, then the speed at which information is fetched from any storage medium significantly impacts learning. I once did an exciting thought experiment where I visualized the calendar scale of a CPU. Imagine that one cycle of a CPU takes one second in CPU terms. (That's 4 trillion times slower.) So how long do everyday computer tasks take?